Assessment by stealth in a digital learning environment

Schools have long made use of assessments of and reports about their students’ learning to inform decisions about teaching. However, reports from national assessment programs, or even end of course assessments usually represent a snapshot in time, often at the conclusion of instruction. According to Dr Michael Timms, Director of Assessment and Psychometric Research at the Australian Council for Educational Research (ACER), digital learning environments now provide educators with an opportunity to undertake continuous assessments of learning through embedding them into the instructional materials.

‘Schools can analyse the data from assessment snapshots to infer the learning growth of individual students, particular classes or year levels. They can also analyse the data to evaluate the impact of their teaching and learning programs, and monitor trends over time. Nonetheless, there is always some amount of time lag between a student undertaking an assessment task and the report from that assessment task becoming available,’ Dr Timms said.

Dr Timms will speak at Research Conference 2015 on the opportunities offered by the growing use of digital learning environments to track, analyse, report on and even adapt to the characteristics of students as they respond to a task in real time.

According to Dr Val Shute, the Mack and Effie Campbell Tyner endowed professor of education at Florida State University, the data we can obtain through digital learning environments is similar to the continuous stream of information generated by barcodes that retailers can use to monitor and manage stock inventory and identify patterns in customer behaviour.

A keynote speaker at Research Conference 2015 in August, Dr Shute notes that the traditional approach in schools usually divorces assessment from learning. ‘The typical educational cycle is: teach; stop; administer test; teach, with new content,’ Dr Shute said, speaking ahead of the conference.

‘Digital learning environments enable us to weave assessment seamlessly into the fabric of the learning experience so that it is virtually invisible, blurring the distinction between learning and assessment,’ she said. Dr Shute calls this stealth assessment.

‘Stealth assessment is intended to be invisible and ongoing, to support learning, and remove or seriously reduce test anxiety while not sacrificing validity and consistency,’ Dr Shute said.

‘With stealth assessment, schools no longer have to interrupt the teaching and learning process to administer tests. Instead, assessment is continual and invisible to students, supporting real-time, just-in-time instruction.

‘Stealth assessment also reduces the time spent administering tests, reduces “cramming” so students retain more of what they learn through continuous and ubiquitous assessment, and focuses on actual achievement in terms of competencies.’

According to Dr Timms, digital learning environments generate a variety of data beyond simply achievement data by recording not only a student’s response to a task, but the actions they took during the task and how much help they needed to reach a successful conclusion.

‘Simulations and game-like digital learning environments generate data on the actions students take, including the type of interaction, the frequency of interaction, the sequence of interactions and the timing of those interactions,’ Dr Timms said.

Research that brings together: a) models of learning; b) domain models that characterise the content to be learned; and c) optimal instructional sequences for learning and learning software, is enabling educators to benefit from the ‘big data’ collected from students’ interactions in digital learning environments.

As more digital learning materials are developed that contain stealth assessments of learning, the less we will have to rely on snapshot types of assessment that are not blended with instruction. This will enable teachers to have more continuous information about where students currently are in their learning and be able to act immediately.

Further information:
Research Conference 2015, on the theme Learning Assessments: Designing the future, will be held from 16 to 18 August in Melbourne. Visit <www.acer.edu.au/rc> for more information.