Outsmarting the machine: the puzzle game showing kids how they’re smarter than AI

With more than 100 AI companions available now in Australia, including character.ai, Replika, talkie.ai and others listed in The eSafety Guide, users are spoilt for choice.

Many are free, advertised on mainstream platforms, and designed to look attractive and exciting for young users – however they often lack mechanisms to enforce age restrictions and other safety measures.

Children and young people are obviously particularly vulnerable to mental and physical harms from AI companions – their age means they are still developing the critical thinking and life skills needed to understand how they can be misguided or manipulated by computer programs, and what to do about it.

While the current generation of AI chatbots still stumble over basic facts, the systems answer with such confidence that they’re often more persuasive than humans.

Adults regularly fall for this, but spotting errors in text is especially difficult for children, since they often don’t have the contextual knowledge to effectively investigate or uncover falsehoods.

In the United States, University of Washington researchers have developed the game AI Puzzlers to show kids an area where AI systems still typically and blatantly fail: solving certain reasoning puzzles.

In the game, users get a chance to solve ‘ARC (Abstraction and Reasoning Corpus) puzzles by completing patterns of coloured blocks.

They can then ask various AI chatbots to solve the puzzles and have the systems explain their solutions – which they nearly always fail to do accurately.

The tasks are presented as input-output examples, and the goal is to infer the underlying rules and apply them to new, unseen inputs.

ARC is designed to be challenging for AI but relatively easy for humans, highlighting the gap between current AI capabilities and human-level intelligence.

The team tested the game with two groups of children.

They found the kids learned to think critically about AI responses and discovered ways to nudge the systems toward better answers.

Researchers presented their findings on June 25 at the Interaction Design and Children 2025 conference in Reykjavik, Iceland.

The explainer above goes through a simple example of how AI Puzzlers work and why ARC can be challenging for AI, but relatively easy for humans. SOURCE: Aayushi Dangol

“Kids naturally loved ARC puzzles and they’re not specific to any language or culture,” lead author Aayushi Dangol , a UW doctoral student in human centred design and engineering, says.

“Because the puzzles rely solely on visual pattern recognition, even kids that can’t read yet can play and learn.

“They get a lot of satisfaction in being able to solve the puzzles, and then in seeing AI – which they might consider highly smart – fail at the puzzles that they thought were easy.”

ARC puzzles were developed in 2019 to be difficult for computers but easy for humans because they demand abstraction: being able to look at a few examples of a pattern, then apply it to a new example.

Current cutting-edge AI models have improved at ARC puzzles, but they’ve not caught up with humans.

Researchers built AI Puzzlers with 12 ARC puzzles that kids can solve. They can then compare their solutions to those from various AI chatbots; users can pick the model from a drop-down menu.

An “Ask AI to Explain” button generates a text explanation of its solution attempt. Even if the system gets the puzzle right, its explanation of how is frequently inaccurate. An “Assist Mode” lets kids try to guide the AI system to a correct solution.

“Initially, kids were giving really broad hints,” Dangol says.

“Like, ‘Oh, this pattern is like a doughnut.’ An AI model might not understand that a kid means that there’s a hole in the middle, so then the kid needs to iterate. Maybe they say, ‘A white space surrounded by blue squares.’”

The researchers tested the system at the UW College of Engineering’s Discovery Days last year with more than 100 children from Grades 3 to 8.

They also led two sessions with the KidsTeam UW , a project that works with a group of children to collaboratively design technologies. In these sessions, 21 children ages 6-11 played AI Puzzlers and worked with the researchers.

“The kids in KidsTeam are used to giving advice on how to make a piece of technology better,” co-senior author Jason Yip , a UW associate professor in the Information School and KidsTeam director, says.

“We hadn’t really thought about adding the Assist Mode feature, but during these co-design sessions, we were talking with the kids about how we might help AI solve the puzzles and the idea came from that.”

Through the testing, the team found that children were able to spot errors both in the puzzle solutions and in the text explanations from the AI models.

They also recognise differences in how human brains think and how AI systems generate information.

“This is the internet’s mind,” one child said.

“It’s trying to solve it based only on the internet, but the human brain is creative.”

The researchers also found that as children worked in Assist Mode, they learned to use AI as a tool that needs guidance rather than as an answer machine.

“Kids are smart and capable,” co-senior author Julie Kientz , a UW professor and chair in human centred design and engineering, says.

“We need to give them opportunities to make up their own minds about what AI is and isn’t, because they’re actually really capable of recognizing it. And they can be bigger skeptics than adults.”

Through projects like AI Puzzlers, the UW’s CHiLL Lab says it is continuing to push forward how humans think about technology, education, and child development, while equipping the next generation to understand and question the role of AI in their lives.