Assessment of online learning: A world of possibilities

Teachers in traditional classrooms use a variety of assessment methods to obtain a picture of how well their students are progressing in their learning. Common types of assessment include selected response items, in which a student makes a choice from responses that are suggested to them, and constructed response items, in which the students must write their own response from scratch.

When teachers move to online learning they often, as a first step, simply move their selected response and simple constructed response items into an online format. While this is a good place to start, it ignores the fact that digital environments offer a greater range of item types than traditional paper-and-pencil tests.

Types of online assessment

A taxonomy of digital assessment formats developed in 2009 by Kathy Scalise at the University of Oregon, USA, provides interactive examples of 28 innovative item types, arranged by level of complexity and constraint.

The term ‘constraint’ refers to how restricted the student is in how they can respond to the question. In the most constrained types of items the student can only select from pre-determined choices. The less constrained the item, the greater the student’s scope and control over what is submitted in their response.

In general, constrained types of items like true/false or multiple choice are more limited in the types of knowledge and skill that they can assess.

For example, multiple choice items may only ask questions that require students just to recall facts, like ‘When did the First World War take place?’, while a moderately constrained drag and drop item enables students to place a list of events leading up to the outbreak of the First World War into chronological order.

An even less constrained task might be to have students type a response to the question, ‘Which of the events leading up to the outbreak of the First World War do you think contributed most to starting the war?’

Often, however, we are trying to promote the acquisition of skills by students, rather than just filling their heads with a series of facts. To see if students can apply these skills, new formats of online items are being developed that involve scenarios and/or simulations to measure not only the learning outcomes but also the learning process.

Figure 1: An example of an online assessment item from ACER’s Dynamic Assessment and Learning Environment. Image ©ACER

Developing and selecting assessments

Teachers needing to assess their students have to choose between sourcing an existing assessment or creating their own.

There are many applications available on the Internet that allow teachers to create simple multiple choice assessments. For more sophisticated development of complex questions a better choice is to work within a learning management system that offers an assessment feature, such as Moodle or Blackboard.

Teachers developing their own online assessments should, however, exercise caution with some of the more sophisticated features of these assessment systems, such as using partial credit scoring or random number generation, as they can negatively affect the fairness of the assessment. Similarly, teachers should avoid using a language level that is above the reading level of the students, to prevent introducing an additional layer of difficulty to the item that is unrelated to what might be being tested.

The important thing in developing or selecting assessments for online learning, as with classroom based learning, is to have a clear idea of what the learning goals are for students and to choose assessment types that:

• are aligned to the learning goals
• produce responses that will provide evidence that the learning goals have been achieved, and
• require students to perform tasks that are as authentic as possible applications of the knowledge and skills being assessed.

If these criteria are met, you will probably find that you are developing or selecting assessment tasks that cover a variety of levels of constraint and complexity.

Find out more:
This article is based on the book chapter ‘Assessment of Online Learning’ by Dr Mike Timms, published in the Handbook on Digital Learning for K-12 Schools, by Ann Marcus-Quinn and Tríona Hourigran (Eds.), Springer (2017).