From vision to reality: The journey of computer-adaptive testing

A November 1986 edition of the ACER Newsletter lauds the ‘the development of microcomputers and their introduction into schools’ as a great step forward for introducing CAT in the classroom.

The objective of ACER’s original CAT project was to develop a computer-based instrument that could more accurately assess a student’s achievement than a static paper-based assessment by individualising the test as the student enters their responses. Although students would experience a different question set, student outcomes could be plotted on a common scale. The results of each student’s assessment would therefore equip teachers with a far more precise idea of how well a student was doing and where to go from there. While the understanding of the theory was clearly present in the 1980s, it did take several decades for the technology and school readiness to catch up.

An illustration of pathways through an adaptive test.

The original work in the 1980s demonstrated an understanding that standardised testing doesn’t always provide teachers with an accurate enough picture of individual students’ knowledge and skills. With research showing students in the same year level can vary in achievement by the equivalent of up to 6 school years, it’s no surprise that in standardised testing, high-achieving students and students requiring support are under-served.

Released in 2021, ACER’s PAT Adaptive assessment is the successful realisation of an innovative project that was started back in the 1980s. It’s a fairer, more engaging, and accurate way to assess students’ progress and map out their personalised learning needs, so no one is overlooked.

The 1986 article notes the potential for time-saving innovation in the classroom, declaring that CAT ‘could free the teacher from having to administer tests to the class as a whole at the same time – individual students could take the test when they were ready to do so; students could then be provided with instant feedback by the computer about how well they were progressing in the course; the computer could automatically update information about students’ performance’ and deliver ‘tailored reports to teachers and school’.

PAT Adaptive has now achieved the 3 key aims outlined in the 1986 project, as we waited for technology to catch up to the idea:

1. Create a CAT program that could be run on ‘portable computers’.
2. Convert paper-based maths test to CAT.
3. Assess the differences between paper tests and CAT, and the extent to which CAT improves test-taking performance.

It's fair to say that the changing technology landscape has meant ACER exceeded the original goal of providing CAT on ‘floppy diskettes’ that can ‘handle items with significant pictorial content’.

The key advantage of CAT is that it helps teachers identify a starting point on the learning journey that’s appropriate for each student. Adaptive testing establishes what a student understands and is able to demonstrate at a point in time. Armed with this more exact information, teachers can then more accurately identify the strengths and areas for improvement of each individual student; diagnose student needs; differentiate their teaching; and monitor individual progress over time.

Computers have become such an integral part of teaching and the classroom experience – and so quickly – that it’s hard to remember a time when students had to put pencil to paper and demonstrate their proficiency in one-size-fits-all models of assessment. But less than 40 years ago, ACER was at the cutting edge of computer-adaptive testing – steering the education sector away from inequitable testing models and pushing the technology to catch up to our vision.