Beware claims of edtech ‘sweet spot’, there’s compromised data at play: expert

As schools across the OECD reach tech saturation status, Dr Jared Cooney Horvath has raised a critical question: how can we now assess if classroom devices actually improve learning outcomes?

Drawing on PISA data to illustrate his point, Horvath warns that the historically high performing ‘no-technology’ student group is shrinking towards zero in some countries.

This means the comparison group is now severely compromised, the former teacher and author of The Digital Delusion suggests.

Across previous PISA cycles, the relationship between classroom technology use and academic performance has been remarkably consistent. That is, as technology use increases, student performance largely declines.

“But I predict PISA 2025 will look different,” Horvath wrote ahead of the 2025 results release date.

“This time, I expect students who use some technology at school will outperform students who use none. We will see graphs suggesting an edTtech ‘sweet spot’ – likely somewhere between one and 180 minutes per day.

“And then the flood will begin.

“Fifteen years of previous data will be pushed aside, and we will hear that the relationship between screen use and learning is ‘nonlinear’.

“We may even be told that previous concerns about educational technology were overblown – that the problem was never technology itself, merely too much technology.”

Before anyone starts celebrating, we should be questioning what’s causing this apparent ‘sweet spot’ of tech use that has been emerging in the data for the last decade, Horvath argued. 

“When almost every student uses technology regularly in school, discovering that the ‘highest performing group’ uses technology tells us surprisingly little about whether technology caused their success,” he noted.

This leaves us with a critical distinction, Horvath said.

“‘Performing better than the remaining non-users’ is not the same thing as ‘performing better than the average student would perform without technology’."

“Unfortunately, we are rapidly losing the data needed to distinguish between the two.”

The expert has been outspoken about the negative impact that the large-scale adoption of digital tech in classrooms is having on student learning.

Speaking recently on CBS Mornings in the US, Horvath said that when the screens come out, learning is immediately hindered.

“One of the good mechanisms of human learning is, as humans, we can synchronise with one another. Human biology can actually link up. And you can feel that. We call that ‘affective empathy’.

“And in those moments, learning goes up. Any time you bring a screen then into the mix, it tends to cut off that synchrony. So, if you’ve ever been at a dinner table and everything’s going great, and suddenly someone brings out a phone.

“You can almost feel that moment where everything kind of drops. It’s that same thing in classrooms. When screens come out, that synchrony drops. Learning starts to go down,” he explained.

Despite the markedly increased tech use amongst Gen Z, digital literacy in the US is trending down for this cohort, Horvath says.

In January, Horvath submitted a written testimony to the US Senate Committee on Commerce, Science, and Transportation, warning that if federal policy kept incentivising widespread digital adoption in schools without demanding “independent efficacy evidence, privacy protections, and developmental safeguards”, then we risk causing long-term educational harm and may compound the damage already done to children’s brains.

“Over the past two decades, the cognitive development of children across much of the developed world has stalled and, in many domains, reversed.

“Literacy, numeracy, attention, and higher-order reasoning have declined despite increased school attendance and expanded public investment,” he flagged.

International test scores reveal a clear pattern here, Horvath told his CBS Mornings hosts.

“When you look at [the data], the more time kids spend on tech for learning purposes at school, the more learning starts to go down. So, any school that maybe introduced it as a peripheral, it may have been okay.

“But when it becomes central, when kids are using it in every class, we’re just seeing drops in literacy, numeracy, to the point where right now in the US our literacy scores are lower than they’ve ever been this century and our math scores are lower than they’ve ever been since the 80s.”

We know that the brain remembers more when reading from print compared to reading from a screen, Horvath added.

“A big aspect of human memory is space. Spatial location. Where do things exist? Paper has an unchanging, clear three-dimensional location. If you read a book, every word has a very clear space. It’s going nowhere. That becomes part of your memory.

“(If) you read the same thing on a screen, the words will scroll through...

“There is no three-dimensional location, so an entire aspect of your memory just gets dumped. So, if you changed nothing with the classroom or your home other than print things out versus look at them on a screen, you will learn more and comprehend better just by the medium you’re using.”

When pressed as to whether it is technology and not other factors that is driving declining educational achievement, Horvath indicated the link was obvious.

“What we can [see] through statistics is the drops really started around 2010, 2012, so pre-COVID. So, we were seeing this move before COVID hit…”

If you remove health issues from the equation, we still see the same decline. The same goes for SES factors, Horvath said.

“So [that] tends to say that, yeah, tech is one of the drivers of this big move.”

In the latest PISA testing, Australian students scored the strongest in the new domain Learning in the Digital World (ranking 9th overall), which measured how well they use digital tools to source and evaluate information, solve problems and to continuously learn through reflection and feedback.

Australia achieved a score of 532 points compared with the OECD average of 500 points, with more than three-quarters of students attaining the National Proficient Standard and some 38 per cent classified as high achievers.

ACER research director Dr Jonas Bertling told EducationHQ these figures were “very encouraging”.

“This indicates that Australian students are very well prepared to work in digital, complex environments, which, as we know, will become more important in a digital AI-based world,” he said.

Yet Horvath warned that despite their markedly increased tech use, digital literacy in the US at least is trending down in the younger generation.

“Believe it or not … from 2013 to 2023, the percentage of kids using tech every day in school, being trained on it, is up about 650 per cent.

“But the percentage of kids who are minimally digitally literate is down 11 percent.

“So they’re using it more and more, yet they’re worse at it. Gen Z is less digitally literate than Millennials and equal to Gen X now.

“Why? Because tech is a tool, and the tool is not the skill. The tool is just the way you amplify or demonstrate your skill.”

This is a good rule of thumb for teachers, he suggested.

“Do the skill first. Bring the tech in when you no longer need it. If you know how to do math, now you can safely use a calculator. If you know how to format an essay, now you can bring in a word processor.

“So, develop the skill first and then move to the tech when you’re just ready to make that skill easier.”

Learning by its nature needs to be difficult because it ‘goes against our biology’, Horvath concluded.

“Our biology wants to take the easiest route, use the least amount of effort and energy. That is the law of least effort. But at the end of the day, learning has to push back against your biology. You have to say, ‘no, we’re going in a different direction’.

“So if you’re not struggling, if you’re not fighting, you’re probably not learning. And tech removes all of that friction from a lot of [the learning process].”