Teachers globally are embracing creativity as an essential 21st century skill, particularly because it is a skill that cannot be automated, however a lack of valid and reliable creativity tests mean schools struggle to assess student achievement.
A new machine-learning model developed by the University of South Australia (UniSA), however, is finally providing teachers with access to high-quality, fit-for-purpose creativity tests, that can score assessments in a fraction of the time – and at a fraction of the cost.
Lead researcher, UniSA’s Professor David Cropley started looking at applying a machine learning model approach to assessing student achievement in creativity in late 2021.
“I realised that one of the problems with creativity tests, good creativity tests, in terms of accuracy and so on, is that they tend to be slow and expensive and resource-intensive to score,” Cropley, an internationally recognised expert on creativity and innovation, tells EducationHQ.
“So what often happens is that both in practical settings, but also amongst researchers, people tend to kind of default to less accurate, less high-quality tests, like self-report measures that are quicker and easier.”
Cropley wondered whether AI could help solve the problem, and hunted for ‘drag and drop’ tools applicable to the machine learning process.
He discovered that not only was there an existing image-based test of creativity – the Test of Creative Thinking – Drawing Production (TCT-DP) – but that it could also easily be linked to machine learning, given its inclusion of image classification and object recognition.
“So I put two and two together and thought ‘I’ll give this a try and see if it shows any sort of strong capability with regard to scoring these creativity tests’.
Initially, he used a small-scale version of a machine learning tool to do this, and it showed results that were promising enough that he then moved on to a more sophisticated tool and built a bigger model.
In his research, Cropley gathered a number of already scored examples of the TCT-DP test and used those to train the machine learning model to recognise different examples, in terms of their classification of being ‘far below average’, ‘below average’, ‘average’, ‘above average’, and ‘far above average’.
“I showed hundreds of examples in each of those categories and basically let the model run its process, and then showed it new images.
“[In that process] you see if it gets those correct, and you can build up a clearer picture of how accurate it is on the images it’s never seen before.”
Until now, when students have been given the TCT-DP test, someone like Cropley would collect those tests, sit down, manually score them over 10 or 15 minutes for each test (because there are a set of 14 criteria to look for) and then double check them with another assessor to iron out any differences.
Professor Cropley says a manually scored test for a school with 1000 students would cost approximately $25,000 and require about 10-weeks to receive test results; with his algorithm, the same testing could be conducted for approximately $1000 with results delivered in 1-2 days.
With the new algorithm, however, test forms are fed into a photocopier for scanning, they’re turned into a PDF, and then are uploaded to the model and it literally spits out the result of each individual test form in a matter of seconds.
The development could save teachers thousands of hours in an already overloaded schedule.
“I can get scores for 100 of these test forms in one hour,” Cropley explains.
“Whereas in the past, that would take two people 15 minutes for each test, it’s a matter of days of work.”
Co-researcher, UniSA’s Dr Rebecca Marrone says the capacity to test and measure creativity has additional benefits for students who are sometimes overlooked.
“Testing for creativity opens up an avenue beyond more traditional intelligence testing,” Marrone says.
“Testing for creativity helps identify students who may have abilities that do not show up on traditional approaches to testing in school.
"For example, a child who does poorly on traditional IQ tests, but is highly creative, could easily slip through the cracks.
“Developing creativity also protects children on the lower end of the achievement spectrum by training them in a skill that is not vulnerable to automation, which can help buffer them against the effects of digital transformation.”
Despite there being limited opportunity until now to assess creativity, Cropley says he is encouraged by the amount of creativity learning happening in schools.
“For decades, there has been a lot of lip service around the world regarding creativity, but in the last, maybe four or five years, I think we are really seeing some action that follows that up, and that’s particularly coming about because of the impact of automation and the future of work that for the first time, we have an almost existential need for creativity rather than just as a nice thing to have.
“A few years ago ACARA developed creativity as a general competency – so it’s officially recognised in the national curriculum, but also I’ve seen in my own work and experience, a lot of schools, whether it be independent schools, Catholic schools, and state schools, are all doing things in various ways to respond to this desire to do something about creativity and to try and build it into what they’re doing.”
Recently EducationHQ covered a UK story where teachers and school leaders have formed what’s called the ‘Penryn Partnership’ in West Cornwall, involving eight schools, along with the University of Exeter, and created a toolkit to support British educators in developing teaching for creativity in the classroom.
Head of humanities at Penryn College, Liz Westhead, who was part of the team that developed the toolkit, said schools have a moral imperative to embed creative pedagogy and skills into classroom practice.
She said creative pedagogy deepens understanding so students are engaging in more memorable learning, whilst also developing the creative skills they need for their future.
Cropley concurs and says there’s a relationship between creativity and academic outcomes.
“If we want to educate students more completely, then we’ve got to focus not just on one narrower slice of what it means to think and to be successful and so on.
“It’s not just a matter of learning that two plus two equals four, as I often frame it, but we have to also teach students to answer the question, ‘what are different ways that you can come up with the answer four?’"
So, Cropley says, it’s flipping it around from a convergent ‘one right answer’ mode to many possible answers.
“And that’s important, because in every activity and in our lives, we face problems all the time that are uncertain, that have no correct answer – and so kids have to be equipped to deal with that.”
The other related pressure, he says, relates to a growing wave of AI and automation replacing numerous jobs, however the one thing that AI can’t perform as well as humans, is creativity.
“And so it’s likely that more and more human jobs in the future will be based around our ability to think creatively.
“So if we’re not teaching kids how to think like that, then we’re setting them up for failure.”
Cropley remains optimistic about AI and its implications for education, particularly regarding generative AI.
“There are lots of very useful things about the current generative AI that we’re seeing, there’s no doubt about it, however we have to be nimble and quick on our feet to respond to that and to teach kids how to use it effectively,” he says.
“It’s one of those things that you can’t put the genie back in the bottle, it’s going to happen one way or the other.”
Cropley believes teachers are ‘pretty pragmatic people’ and good at responding to these types of things.
“But we definitely can’t sit on our hands, we have to be thinking about these things, thinking about how they’re affecting education and devising solutions.”
The algorithm is at present being developed as a desktop app for teachers to use in the classroom. Ahead of this, classroom teachers interested in using the TCT-DP are invited to contact the UniSA team to discuss their needs.
