This school built its own chatbots. Students rate them highly as a learning tool

That’s right, they’re chatbots – yet as curated and finely-tuned learning assistants, they don’t work in the way you might first expect.

So suggests Alice James, Biology teacher and Head of Teaching and Learning at the Melbourne school, who along with fellow teacher Brent Charlton, worked with a group of teachers to build specialised chatbots that could help students to sharpen their own thinking processes, rather than just spit out a tidy answer when prompted.

James believes that AI is an ‘amazing tool’ that, when used carefully, can deepen the learning experience for students without short-cutting the fundamental cognitive work required to master key concepts and skills.

She says the chatbot project started as a means of providing a safe and trusted AI tool for students to freely access.  

“It comes back to trying to give teachers some control over what students are accessing in the AI world, rather than having this problematic situation where they can routinely access all sorts of information that may be inaccurate, unreliable or not aligned with the study design.

“And so [we thought], ‘this is something that they are routinely using, so let's try and get them using something that we know is safe and reliable.”

Drawing on the technical expertise of AI consultant Chris Bush, James and her team set about defining some in-build guardrails that would direct the chatbots’ modis operandi.

The school leader says it really pays to bring in an expert to guide the design and implementation process.

“I'm a biology teacher, I've got no real technical understanding [of how to build a chatbot]. Being able to get Chris to walk us through the process of actually how we needed to set these up, and what lessons he'd learned along the way, was really invaluable.”

The chatbots were woven into teachers’ pedagogy last year, and so far there’s been a positive response to their arrival. 

James reports that of 42 students surveyed, the chatbots scored an average rating of 4.32 out of five stars in terms of their usefulness in class and their ease of use.

“Sometimes students use them as a personal tutor. So, the teachers upload their resources, and the kids have the chatbot link on their Schoolbox class page and can access it at any time.

“They can ask questions about [the task or project] and it tries to guide them through the answer, without just giving them the answer - that is how we tried to set them up.

“And we also use them as a kind of a resource in class. So for example, the Year 6s used them for the very specific activity of building their writing responses and linking that to the rubric.

“They have also been used to generate examples of exam questions…”

Edwin uses layered questioning to guide students from basic understanding to deeper analysis. SOURCE: supplied

While it is tricky to isolate the direct impact the chatbots have had on learning outcomes, James says, feedback from teachers and students has been generally positive.

James collated some data around this as part of her Masters study.

“Students really appreciated having immediate, on-demand support. It was particularly important to them that it was reliable - as the teacher had built the chatbots, they knew they could trust it.”

James and Charlton recently shared Kilvington’s chatbot project at an Informa’s AI in Education conference held in Geelong.

The educator says they both got a lot out of the event.

“I found it really useful to hear what other schools are doing in this space.

“I think we're in a very murky world at the moment where we're all trying to out how AI can be used in Education, and the more we can learn from each other and network and have that opportunity to share ideas, the better we are.”

Inspired by some ideas put forward by other presenters, James says she now wants to look more into how micro-credentials could be used in order to build up students’ AI knowledge and skills, and how students could also be granted more autonomy with the tech.

While wider concerns around cognitive outsourcing in schools remain, James says the in-house chatbots are working to show students that just seeking an answer via Co-Pilot or ChatGPT and the like is not always the wisest move.

Edwin, for example, acts as a private Economics tutor. He has been built to engage students in rounds of Socrative questioning and refuses to offer outright solutions without getting a meaningful response first.

If met with vague or surface-level answers from students, he craftily asks probing questions like ‘why do you think this might be the case?’ or ‘what evidence do you have to support this view’ to prompt deeper analysis and engagement.

“We know AI can replace [their] thinking, and by removing the struggle, students are able to short-cut the learning process. With the chatbots, we tried to get students away from that, from being able to just put in the question and get an answer straight away,” James adds.

The educator is openly optimistic about AI’s future in the classroom.

“Like I said in the presentation last week, I firmly believe that we can collaborate with AI as a partner in learning. It provides a wealth of opportunities for teaching and learning, however we must ensure that it something that we use to help the learning, rather than  replacing it.

“But for the chatbot to be really effective, it has to be in-built into the pedagogy. And so it has to be part of the classroom experience - I had one set up where it was a bit of a personal tutor in the background, and it just really didn't get that much traction from the students.

“But where it was actually used in class and I directed them to it, that's where it made the difference.”