Replicating the results of experiment, investigators found that examiners’ ratings of candidates in two very different high-stakes behavioral testing situations were affected by the performance of prior candidates. The first group of scores were from 2,272 takers of a UK 16-station OSCE required for any medical school graduate who has been out of medical school for more than 2 years and who wishes to pursue subspecialty training. The second were the multiple mini-interview (MMI) scores of 3,016 applicants to the University of Alberta Medical School. Consistent negative correlations were found between the index score and those of preceding examinees. That is, a candidate was graded somewhat higher if he or she was preceded by less-ranked candidates, and vice versa. The effect accounted for 5% to 11% of total score variance. This appears to be a fairly robust effect. Now the difficulty will be in deciding how to correct or ameliorate it. — Laura Willett, MD
Yeates P1, Moreau M, Eva K. Are Examiners’ Judgments in OSCE-Style Assessments Influenced by Contrast Effects? Acad Med. 2015 Jan 27.