Digital assessments of early learning

ACER has developed a computer-based monitoring and assessment tool for the Northern Territory Department of Education and Training (DET) that will be used to assess Year 1 students’ literacy and numeracy skills from 2013.

While other computer-based assessments for early years do exist, they generally involve one-to-one administration by teachers. The ACER assessment pioneers an approach where early years students independently interact with the program by navigating the site, tackling the questions and recording their own responses.

From the very early stages of the test development, an attempt to replicate or replace the teacher-student interview style was felt to be counterproductive so the focus of design became about ‘what are the good points about using a computer’?

Many benefits were discovered for the computer-based model. 

Firstly, research shows that children, including those from migrant and Indigenous backgrounds, engage strongly with the technology. They enjoy the independence and interactivity of using computers and computer games, even with software that does not reflect their own backgrounds.

The audio features of a computer enable standardised delivery to all students. Instructions are recorded by professional voice actors who can be directed to give emphasis to intonation where the question’s intent requires. Students can listen to the instruction as many times as they need to, which saves the teacher reading the same text over and over again.

Due to the autonomous nature of engaging with a computer program, students are less likely to feel pressure when they realise they have made a mistake. Clicking the Undo button allows for easy self-correction and a larger scope for thinking through a variety of answers before committing to one. This feature, combined with an engaging format, has the ability to allow students to feel less likethey are being assessed and reduce performance anxiety that can be created in other test environments.

The model also enables the capacity to contribute to student profiles by recording computer behaviours such as number of seconds spent on a question, use of the pop-up text before recording an answer, and number of times an instruction was listened to.

Finally, only those questions that can be asked in a manageable way onscreen were considered. This is the same for response styles also. Hence, while the main assessment is mapped against the new Australian Curriculum with additional diagnostic modules reflecting the Northern Territory’s own T-9 Diagnostic Net, the components of writing and speaking from the Australian Curriculum are not covered. To reduce confusion, there are only two item format styles, ‘clicking on’ and ‘drag and drop’.

ACER will also deliver a practice program so that teachers can troubleshoot the administration and students can familiarise themselves with the navigation prior to the trial test. To make the most of computerbased features and avoid the navigation itself becoming an obstacle, students will be able to engage with the practice program year-round. This will allow students to become comfortable with the technology, style and language.

In commissioning the work, DET was very clear that the assessment should feel local and familiar, and reflect the students’ context. The assessment features remote scenery and bush images, and one of the texts has an Indigenous family as the main characters and is read by an Indigenous voice actor. ACER secured a range of actors to read the stories for the comprehension section, and Julie Nihill – from Blue Heelers and now a teacher at Collingwood College in Melbourne – is voicing all the audio. Every instruction and question is available as an audio and text option.

ACER piloted the assessment in several Northern Territory schools, including a remote Indigenous school in West Arnhem Land, in May 2012. Feedback from the pilot indicated that the young learners enjoyed the assessment and were able to navigate the computerbased materials. The teachers who participated were enthusiastic about the potential of the tool to inform their own planning.

A larger trial of the assessment will take place at the end of 2012, and will reveal important information about this new form of testing including how well six-year-olds cope with multiple-choice questions and how well they can work independently on computers.