Designing assessment for radiopharmaceutical scientist training

The Australian Council for Educational Research (ACER) has been engaged by the Australasian College of Physical Scientists and Engineers in Medicine (ACPSEM) to evaluate the assessment methodologies in the College’s training programs and to make recommendations on how assessment practices may be improved.

The ACPSEM oversees certification and registration in programs in Radiation Oncology Medical Physics, Diagnostic Imaging Medical Physics and most recently, Radiopharmaceutical Sciences (RPS). The RPS program receives funding from the Australian Government Department of Health to aid the growth of this newly-developed but critical workforce. As part of its work for ACPSEM, ACER worked closely with radiopharmaceutical scientists to develop a new program of assessment for the recently established RPS Training Education and Assessment Program.

RPS is a multidisciplinary field, requiring expertise in chemistry, physics, biology and pharmacy. These scientists are involved in the design, manufacture and analysis of radiopharmaceuticals — diagnostic and therapeutic drugs that incorporate suitable radionuclides into biologically active molecules. They use scientific knowledge and analytical skills to ensure that radiopharmaceuticals are safe and effective for clinical use. Due to the physical characteristics of the radionuclides used in medicine, most of this is carried out at the clinical site. Thus, skilled radiopharmaceutical scientists have become increasingly important in the medical profession.

Radiopharmaceutical scientists manufacture both the radioactive tracers that are essential for diagnostic imaging such as PET and SPECT scans, as well as the radioactive drugs used in radionuclide therapy, such as molecular radiation therapy. In those hospitals with cyclotrons, they will also be responsible for the in-house production of the radionuclides.

ACER worked closely with the RPS Working Group to develop an assessment framework, which would allow individual assessments to be developed and adjusted over time. The framework details the key areas of the program, as well as Progressive Assessment Pathways that trainees and their supervisors should follow. The aim was to build an efficient and effective program of progressive assessment that would be consistent, transparent, defensible and robust.

Using the medical education model of ‘programmatic assessment’, a variety of well-chosen assessment tools are being used where appropriate, which allows for adequate sampling across the breadth and depth of the curriculum. With the programmatic approach, progressive assessment information is collected about trainees until sufficient information is available to be able to make informed judgements about their achievement. Individual data points can be used for targeted feedback of learning, but high-stakes decisions can still be made by assessors, based on the aggregation of multiple data points.

In order to develop a high-quality program of assessment, each specific skill and competency required of the trainee was discussed. Fit-for-purpose assessments could then be designed that are mapped to the RPS Clinical Training Guide. Some of the competencies require written work, while others will require trainees to undertake a customised assessment task.

In undertaking this work, ACER was able to bring its expertise and previous experience in assessment in collaboration with disciplinary experts. This experience includes work for a number of specialist colleges in medicine to evaluate the quality of assessment and data collection processes, and of test item development. ACER’s past work with the Medical Deans of Australia and New Zealand and members of the UK’s Medical Schools Council – Assessment Alliance to implement common assessments across medical schools also informed the approach to the RPS project.

In contrast with these previous projects, in which ACER worked with an existing assessment model, ACPSEM asked ACER to help design the initial program of assessment for its new RPS training program. This allowed an assessment program to be designed, developed and implemented using best-practice assessment principles that are up-to-date.