At present there is limited robust research examining AI and student learning outcomes in school settings, the organisation has flagged, noting most available studies to date have yet to hone in on AI’s use and impact in primary schools.  

AERO suggest Australian teachers and school leaders should be aware of several other important limitations to the evidence base on AI:

  • most studies are recent, short-term, exploratory and conducted outside Australia;
  • there’s limited research on the durability of student learning when AI is removed;
  • many studies focus on student engagement or perceptions rather than robust measures of learning;
  • evidence about the long-term impact of AI on student learning remains limited or mixed; and
  • AI technologies continue to evolve rapidly, meaning research findings may become outdated as new tools and capabilities emerge.

In light of this, AERO recommends school leaders and teachers ensure that a sound understanding of how students learn – and particularly the process for acquiring, retaining, and applying knowledge – be used to inform decisions about whether, when and how AI is used in the classroom.

“If AI is being used, it should scaffold learning and create opportunities for students to acquire, retrieve, practise and apply knowledge, rather than replace the cognitive work students need to do themselves,” one of the guides reads.

Building knowledge in long-term memory is central to this process, AERO adds, by providing the foundation for understanding, reasoning and applying learning to new situations.

“This has particular implications for novice learners and students who are still developing foundational knowledge and the metacognitive strategies needed to regulate their learning.”

AERO CEO Dr Jenny Donovan says educators are understandably seeking guidance on the rapidly developing technology.

“Students are already using AI for help with their schoolwork, including using it to find information, support learning or translation, and for writing or editing,” Donovan says.

“Emerging evidence on AI points to both potential opportunities, such as supporting teachers with planning, scaffolding and workload reduction, but risks, too.

“While AI can generate content, solve problems and complete tasks on demand, the flipside is that it can lead to students outsourcing the essential cognitive processes that lead to learning.”

AERO says that teachers and school leaders should monitor for the potential risks that come with AI, and particularly with GenAI, that could hinder effective teaching and learning.

Beware a ‘performance paradox’ when students use AI to generate responses, solve problems and produce content on cue, AERO warns.

Here students might perform better on a task when using AI but ultimately learn less in the process.

“The key concern isn’t simply whether AI improves students’ immediate performance, but whether its use supports or displaces the cognitive processes required for learning.”

This is closely tied to the concept of cognitive offloading – a concern experts have consistently raised since GenAI arrived on the scene.

“While cognitive offloading can sometimes be beneficial when it reduces unnecessary demands on working memory and allows students to focus on learning tasks (e.g., using a calculator for complex calculations), it can become detrimental when students outsource cognitive processes and expend less cognitive effort, undermining learning,” AERO states.  

We know that knowledge stored in long-term memory provides the foundation for subsequent thinking, retrieval, reasoning and application. It follows that when students use AI to retrieve information, generate ideas or produce written responses, it may limit their opportunities to demonstrate, extend and apply their learning, AERO adds.

 School leaders and teachers should also be aware of other potential risks, including:

  •  Students may become over reliant on AI, and particularly GenAI, to plan, monitor and evaluate their learning, which may reduce opportunities to develop the metacognitive strategies and self-regulated learning techniques needed to independently manage their learning.
  •  Students may develop ‘illusions of knowledge’, where they believe they understand content because they can quickly obtain answers, despite not having developed the foundational knowledge, metacognitive strategies or self-regulated learning techniques needed to apply or transfer their learning on their own.  
  • Students without sufficient prior knowledge and metacognitive strategies may not be able to recognise inaccurate, biased or fabricated AI outputs.

Primary schools should exercise greater caution if and when introducing AI, the guide states.

And if AI is being used, educators should explicitly teach students about how it works, its capabilities and limitations, and how to use it safely, critically and responsibly across curriculum areas without replacing the thinking required for learning.

“They should also clarify when students are expected to work independently, when and how AI can be used appropriately, and how teachers will maintain oversight of student learning.”

AERO has also put forward three broad recommendations for policymakers, including developing clear, evidence-informed policies and guidance to schools to inform decisions about AI; supporting effective implementation through access to quality AI tools and building capability; and strengthening the evidence base through ongoing research, monitoring and evaluation.

“The emerging evidence suggests that AI doesn’t have a single, uniform effect on learning.

“Rather, its impact appears to depend on the design of the technology, including whether it incorporates pedagogical guardrails – that is, features that guide students toward engaging in the learning process rather than simply obtaining the answer…” the organisation concludes.