Developed by researchers at the Technical University of Munich (TUM) and the University of Cologne, the system enables teachers to provide significantly more students with individualised support.
According to research by Professor Achim Lilienthal an up-to-date PC, a good graphics card and a standard webcam is all that will be required for teachers to identify many of pupils’ issues.
The principle is simple and very straightforward. A webcam tracks the student’s eye movements and depending on the task, specific patterns emerge that can be displayed digitally on a heatmap, with red indicating areas where the children look frequently and green the areas where they glance only briefly.
“The AI system classifies the patterns,” Lilienthal, a robotics professor, says.
“On this basis, the software selects learning videos and exercises for the pupil.”
Identifying learning strategies via heat maps
Maike Schindler is a Professor of Mathematics in Inclusive and Special Education Contexts at the University of Cologne and has worked with Lilienthal for ten years.
“Tracking eye movements in a single system using a webcam, recognising learning strategies via patterns and offering individual support, and finally creating automated support reports for teachers is completely new,” she says.
Schindler also headed up the recently completed KI-ALF research project, which was funded by the German Federal Ministry of Education and Research (BMBF), and within which the webcam-based eye-tracking system was developed.
Her research focuses on pupils “who have great difficulties in learning mathematics”.
Lilienthal believes that “individually customised lessons” for high-achieving children are also possible in the future.
Schindler and her team have defined hundreds of tasks in which children add, subtract, multiply and divide numbers, or have to recognise or represent them.
“Tasks involving visually presented, digital learning materials are particularly suitable for this approach,” she says.
For example, the children are asked to count the dots in a ten-row table with several dots missing only in the bottom row.
The pupils who catch on quickly jump to the bottom row and only count backwards.
Those who count the rows and dots individually are among the ones who need support.
The digital system uses a heat map to show where the children look and the AI translates the patterns into individual practice programs.
Simplified, high-precision eye tracking
To develop the simplified eye tracking system, Lilienthal says he benefits from the fact that he also works with corresponding systems in robotics research.
In that work, he currently uses eye trackers with the small humanoid robot, Nao. This enables it to communicate better with humans. However, very precise systems like these cost many thousands of euros.
To find a more cost-effective solution for schools, the researchers combined technical expertise with knowledge from mathematical didactics.
While advanced systems work with a maximum deviation of one degree, webcams have a lower accuracy of three to four degrees.
Thankfully there has been a solution:
“With the AI-ALF math tasks, we know that the students are ultimately looking at the on-screen display of the problems,” Lilienthal, who earlier in her career was a classroom teacher, says.
“We use this to automatically readjust the eye tracking with the webcam. The system has gradually learned to deal with inaccuracy.
“Today it makes no difference to our application whether we work with our webcams or high-end eye trackers,” the academic says.
This makes the AI system developed with Schindler affordable and, therefore, increasingly important for school use.
Wulfen Comprehensive School the first to use the system
At the Wulfen Comprehensive School in Dorsten, North Rhine-Westphalia, a standardised maths test revealed that one-third of 180 children at the start of Year 5 had ‘arithmetic difficulties’.
“We are delighted that we can now support significantly more children in their basic math skills with the help of the AI-based learning system,” Lilienthal says.
This means we can help more learners improve their maths performance than in the past due to a lack of teachers.”
In the comprehensive school, five pupils can work with the KI-ALF system simultaneously in individual remedial lessons and are supported and accompanied by a maths teacher.
Typically teachers can give individual support to only one child at a time.
“Especially in times of scarce resources and teacher shortages, our system for promoting basic maths skills is simply an excellent support for schools,” Schindler says.
