Healthy prognosis for medical assessment

The pilot phase of the Australian Medical Assessment Collaboration (AMAC) was completed in May 2012.  The project focused on the feasibility of generalisable assessments for final-year medical students and was conducted by a consortium of staff from the University of Queensland, Monash University and ACER.

Seven medical schools and approximately 500 students participated in the pilot. All students received individual reports which outlined their achievement in key content areas of the assessment framework relative to the other participating students. Participating schools also received institution reports illustrating their cohort’s performance.

The project successfully gathered both substantial data on assessment items for psychometric evaluation and feedback from participating students.  The processes have also provided insight into the possibility of benchmarking strategies for participating institutions.

In an exit survey, over 80 per cent of students stated that the pilot assessment was either ‘very’ or ‘quite’ relevant to their medical degree and to their future professional practice. Only two per cent selected ‘not at all’ or ‘very little’.

Student feedback highlighted the appropriateness of the assessment. One student responded: ‘I thought the test was one of the most reasonable ones I’ve done as a med student. Generally they are so abstract that there is no point studying. This test was very appropriate’.

The pilot assessment was built with specific reference to the AMAC Assessment Framework—a blueprint for the possibilities of final-year assessment of medical students, separated into content and process domains.  The pilot assessment measured students’ knowledge and understanding of clinical problems and conditions in a clinical context. It is envisaged that future incarnations of AMAC will also assess medical skills and procedures, and aspects of professional practice (such as communication, clinical management and professionalism), coupled with processes of demonstrating and implementing in a clinical context.

Assessment consistency, fairness and standardisation were achieved through the development of high quality assessment items, validated by qualified and experienced medical education experts and clinicians. Items for possible inclusion were submitted from nine Australian medical schools and were subjected to a rigorous quality assurance process. Revised items were then discussed at workshops before being validated and selected for the pilot.

Preliminary analyses suggest that the pilot assessment was statistically reliable. Further psychometric analysis will be conducted over the coming months.

AMAC was funded by a grant from the Australian Government’s Office for Learning and Teaching (OLT). A national report will soon be released for the OLT and planning is underway for a new phase of AMAC.  This next phase will focus on development, implementation and benchmarking approaches.

In an engagement forum held at the University of Queensland in June 2012, Professor David Wilkinson, Dean of Medicine and Head of the School of Medicine at the University of Queensland, opened the invitation for the next phase of AMAC to every medical school in Australia. ACER Higher Education Research Director, Associate Professor Hamish Coates, similarly emphasised that AMAC aims to be a truly collaborative exercise between Australian medical schools.

Find out more:
Further information on the Australian Medical Assessment Collaboration (AMAC) is available at <www.acer.edu.au/amac>