Dr Marty Ross, who blogs regularly on maths education, told EducationHQ it was both appalling and incomprehensible that the last time a NAPLAN test was made available was back in 2016.

“The major issue at the moment is the lack of transparency, the lack of accountability of ACARA – that simply they can conduct these tests online and no one sees them.

“As far as I can tell … I do not know how anybody outside of ACARA can see, for example, any of the questions from last year’s tests.

“So, we simply have no way of knowing whether these tests are mathematically correct, let alone good.”

Ross contends that because the authority functions as a “kind of quasi government” entity, it’s really answerable only to education ministers.

“They’re not accountable enough. You cannot see the questions, so you cannot critique the questions,” he adds.

EducationHQ contacted ACARA for direct comment on Ross’ claims.

An ACARA spokesperson said the authority release recent sample questions across all domains tested in NAPLAN.

“This is in line with other assessments such as the National Assessment of Educational Progress (NAEP) in the United States and the Programme for International Student Assessment (PISA).

“The language in NAPLAN tests is kept simple and age appropriate, and an audio file is available for every numeracy question, so students can also choose to listen to the questions being read to them,” they added.

The best place to get familiar with the assessment was the NAPLAN public demonstration site, ACARA suggested, while more information about the development of the tests can be found here.

Tired of the media blindly accepting NAPLAN as gospel, a decade ago Ross ‘fought some battles with ACARA’ to have past tests released to the public under freedom of information laws. 

After close analysis, he concluded the numeracy tests were “just crap”.

Then, before NAPLAN switched fully online in 2022, Ross managed to source a few papers from the 2017-2021 tests.

He says while some aspects of the numeracy questions look to have improved, a few glaring problems stand out in terms of their wording and focus.

For starters, ridiculous ‘real world’ scenarios laid out in ‘verbose’ and ‘messy’ language appear to reign, Ross suggests.

“It’s true that the real world application of mathematics is important. But also, it turns out that in the questions, there’s so much about the scenario, and so much work ... in the parsing of the language, that the actual arithmetic they’re testing, the actual mathematics they’re testing, is almost always trivial.

“So, they dumbed down the maths to be tested in order to pump up emphasis on the real world.”

Plenty of questions were entirely ambiguous and badly written, he adds.

“NAPLAN writers are just not very good writers…”

Compounding the scene is the harsh fact that nobody seems to care, Ross maintains.

Education reporting has generally churned out the same stories year after year, honing in on results being up or down, reworking ACARA media releases, or pitting private school results against those of public schools, he argues.

A former writer with The Age himself, Ross understands the constraints now facing the profession but says NAPLAN coverage ought to step up.

“You don’t have sub-editors the way you used to have, it’s just not the way it used to be.

“So, I understand when it’s someone like ACARA puts out a media release, it’s an easy story.

“But I think all you can do, and it’s hard, (but) you have to talk to knowledgable critics. 

“It’s very easy to swallow the status quo – and I think education is in a disastrous state at the moment, not just in maths,” Ross says.

The mathematician’s other main beef with ACARA’s numeracy test is that he believes it’s giving testing a bad name.

“I think testing is important. I’m a pro-test guy, I like the idea of a national test.

“And even more than that, I think the lack of testing within primary schools is a good reason why primary schools are failing – but the national test has to be good.

“[NAPLAN] precludes the possibility of a good test. You’re not going to have a second national test while NAPLAN is there.”

According to the National Assessment Program (NAP), NAPLAN numeracy tests measure students’ achievement in “mathematical knowledge, skills and understanding as outlined in the Australian Curriculum: Mathematics”.

“The numeracy tests assess the proficiency strands of understanding, fluency, problem-solving and reasoning across the three content strands of mathematics: number and algebra, measurement and geometry, and statistics and probability,” NAP states online.

Yesterday, the annual NAPLAN exams started for 1.3 million students across the country.