Answer to "What is programmatic assessment?"


Editor: Lambert Schuwirth


Programmatic assessment is an approach to collecting and collating information in a more integrated and meaningful way than traditional assessment programs. Although programmatic assessment and assessment for learning are often used in conjunction they are actually different principles.

The following elements are innate to programmatic assessment:

  • Collection of information about the learner’s performance and progress is done through the use of various assessment instruments.

  • Collation of information about the learner and their progress does not happen simply within one assessment instrument. Information from multiple different assessments and multiple different assessment instruments are triangulated to provide information about how a learner is performing. For example, poor performance on an abdominal examination OSCE station is compared with information from other assessment instruments – for instance sections of a multiple-choice test or aspects of a mini-CEX - to better understand why the performance on the abdominal examination station was poor. This could be because of, for example, lack of practice /poor technique or because of an inability to correlate the clinical symptoms with the right examination to be done.

  • Collection and collation of information happens on a longitudinal and continuous basis to allow better prediction and prognosis of a learner.

  • Information is collected and collated, for example in a portfolio, which is periodically discussed between the learner and their supervisor/mentor/coach. The learner would prepare for these discussions through short written self-reflections.

  • The principle of proportionality governs decision-making in that limited information about a learner’s performance and progress can only lead to low stakes decisions and for high-stakes decisions rich longitudinal information is needed.

  • Assessments are typically assessed according to reliability and validity. However, reliability and validity are not purely governed by traditional psychometric methods but are also achieved by organisational measures, such as transparency, documentation, member checking and saturation of information.

  • In order to successfully implement and sustain programmatic assessment supervisor development must occur. Much of the assessment and decision making in workplace based training, such as the rural generalist program relies on human judgement. It is important that the faculty who are involved in that assessment and decision-making are sufficiently “assessment – literate” to ensure the rigour in the process. Therefore, the most critical aspect in a successful implementation of programmatic assessment is the aligning staff development and capacity building.