Taking higher education assessment from the boutique to eBay

Want to study flight vehicle aerodynamics at MIT? Maybe fly to Yaoundé in Cameroon.  Animal behaviour at the University of Melbourne? You might be in Sydney, Nova Scotia. Calculus at Ohio State University? Not a problem, even if you’re on the International Space Station.

Beyond our bricks-and-mortar universities, large fields of study in higher education have, over the last 20 years, become quite freely accessible online. Sophisticated virtual robots driven by powerful people analytics now help experienced humans teach millions of students. In just the blink of an eye, the key challenge for higher education has switched from selecting who gets in, to grading who gets out.

In just the blink of an eye, the key challenge for higher education has switched from selecting who gets in, to grading who gets out.

Some of the issues are political. Who owns and governs this ‘open’ learning? And what do competing stakeholders mean by ‘open’? Some of the issues are practical. How will innovative educators find effective and credible new ways for assessing what learners know and can do? Can the cloud or crowd address higher education’s deep productivity and quality challenges?

Innovators, institutions, and larger networks in higher education now proffer a suite of ideas and practices for curriculum and teaching. Progress in the important matter of assessment, however, appears modest. Unlike the eBay approach to curriculum and teaching through massive open online courses, or MOOCs, assessment remains a boutique enterprise.

Unlike the eBay approach to curriculum and teaching through massive open online courses, or MOOCs, assessment remains a boutique enterprise.

In Australia alone, we lack pretty basic details on assessment resources and practices, even in highly regulated professional fields. Projects have been advanced in recent years to moderate conceptions of what learners should know and how they are performing. Related work has stimulated the creation and sharing of assessment materials. Quality reviews of assessment tasks show that about a third of that in common use meets acceptable technical standards. This important work treads gently and gradually, weaving around tacit academic cultures and conventions, and more tangible industrial and professional matters.

Imagine a more scalable model of assessment. What might it look like? First, learning outcomes and assessment materials would be defined and created by broad collaborations of scholars, practitioners and technical experts. The materials would be carefully reviewed, with about a third being discarded. Institutions would share materials, underpinning standards and benchmarking. Technology would be used for production, storage, review, delivery and reporting. Educators would moderate processes and outcomes.

This powerful vision is triggered by online education, but its implications are far wider.

The rudiments of the architecture for such a scalable model of assessment are already established. Through the OECD’s Assessment of Higher Education Learning Outcomes study, researchers at the Australian Council for Educational Research showed it is feasible to reach consensus internationally on what undergraduate students should know, and to invoke large international collaborations to review materials. Running an online assessment across 250 universities internationally also revealed that technology is easily managed compared with complex matters like human ethics, individual identification and organisational compacts. Through the Australian Medical Assessment Collaboration we’ve shown how medical schools can work together to produce very large numbers of shared materials for filtering into practice or for use as a stand-alone test.

Yes, research projects like these take years to progress, but they enable us to transform tomorrow given proof of concept and feasibility, and not least given cost pressures eating into university budgets. Let’s get moving, fuelled by all the urgency of new massive online participation. Let’s at least test the waters to find out what’s going on.

This article was published in The Australian on Wednesday 14 August 2013.