Answer to "What is the cost versus value of selection?"

Funding/resources for medical education are becoming more constrained at the same time as accountability in medical education is increasing.  In this constricting environment, medical schools need to consider and justify their selection procedures in terms of costs and benefits as well as fairness and psychometric robustness such as predictive validity and reliability.

Some work has been done on the cost-effectiveness of the Multiple-Mini Interview.

  • Rosenfeld JM, Reiter HI, Trinh K, Eva KW. A cost efficiency comparison between the multiple mini-interview and traditional admissions interviews. Adv Health Sci Educ Theory Pract. 2008;13(1):43-58.

  • Hissbach JC, Sehner S, Harendza S, Hampe W. Cutting costs of multiple mini-interviews - Changes in reliability and efficiency of the Hamburg medical school admission test between two applications. BMC Medical Education 2014: 14(1): 54.

However, not everyone has the resources to stage an MMI.  The following papers suggest an alternative for medical schools seeking to enhance measurement properties of interviews within resource and recruitment constraints.

  • Hanson MD, Woods NN, Martimianakis MA, Rasingham R, Kulasegaram, K.  Multiple independent sampling within medical school admission interviewing: an “intermediate approach”.  Perspectives in Medical Education 2016: 5; 292–299.

  • Hanson MD, Kulasegaram, K, Woods  NN, Fechtig L, Anderson G.  Modified Personal Interviews: Resurrecting Reliable Personal Interviews for Admissions?  Academic Medicine 2012: 87; 1330–1334.

Moreover, given that many medical schools selection processes involve a number, or combination, of selection tools such as prior attainment, an aptitude test and an MMI, it is critical to look at the cost-effectiveness of selection as a whole, and also the value of broad engagement in the process (e.g., as MMI interviewers).  We were aware of only one published study examining the cost and value of medical school selection:

  • Schreurs S, oude Egbrink MJA, Muijtjens AMM, Cleland J Cleutjens K. A cost-benefit analysis of a multi-tool selection procedure into medical school contrasted with a lottery procedure.  In press, Medical Education.

Despite increasing calls to examine cost and value in medical education generally, this remains a relatively under-evidenced area. More research is needed on this topic.

Selection into medical school is not an easy matter. We regard selection as complex and very context dependent (see Cleland, Patterson and Hanson, in press Medical Education), and thus is it hard to provide simple answers or simple ways to address issues which might look the same but are actually subtly different depending on your country and system.  However, we hope that the above overview and references will help you formulate your own evidence-informed approaches to selection and widening access!  A final potentially useful reference is the 2018 update to the Ottawa consensus statement on selection:

  • Patterson F, Roberts C, Hampe W, et al.  2018 Ottawa Consensus Statement: Selection and Recruitment in the Healthcare Professions.  In press, Medical Teacher.