I’m a hospitalized patient who wants to survive. Please send me that female doctor with high USMLE certifying exam scores.

Researchers assessed mortality and length of stay (LOS) for nearly 200,000 patients hospitalized in Pennsylvania for five common diagnoses who were cared for by physicians certified in either family medicine or internal medicine.  Average inpatient mortality was about 2%.  After adjustment for multiple patient variables, physician female gender (RR 0.82) and higher composite USMLE score (RR 0.95) were significantly associated with lower mortality.  For each 1 standard deviation (SD) improvement in USMLE score, the odds of mortality improved by 5.5%.  Physician characteristics had minimal effect on LOS.  We have yet more evidence that the US licensing exam results can predict important outcomes in the real world – in this case, in-hospital mortality. — Laura Willett, MD

Link To Article

Norcini, John PhD; Grabovsky, Irina PhD; Barone, Michael A. MD, MPH; Anderson, M. Brownell MEd; Pandian, Ravi S. MA; Mechaber, Alex J. MD. The Associations Between United States Medical Licensing Examination Performance and Outcomes of Patient Care. Academic Medicine 99(3):p 325-330, March 2024. | DOI: 10.1097/ACM.0000000000005480

More On The Problems of US News & World Report Rankings

A student and a faculty member from Duke have written a succinct and practical critique of US News and World Report (USNWR) rankings.  Duke University School of Medicine, along with many other institutions recently, has withdrawn from participation in the rankings.  The authors point out that very little in the rankings have an influence on the actual quality of education received by a typical medical student, and that institutional focus on rankings may have unintended and nefarious consequences.  In a nutshell, the “current rankings are misleading, problematic, and unhelpful and have permeated board of trustee meetings, admission discussions, and evaluations of an institution’s success for too long.” — Laura Willett, MD

Link To Article

Kelly-Hedrick, Margot MBE; Antiel, Ryan M. MD, MSME. The Rise and Fall of Medical School Rankings. Academic Medicine ():10.1097/ACM.0000000000005425, August 02, 2023. | DOI: 10.1097/ACM.0000000000005425

From Community College to Medical School

A study of 2.1 million medical school applications from 2018-2020 using data from the American Medical College Application Service (AMCAS) found that applicants who attended or took courses in community college (about 35% of all applicants) were less likely to be accepted to medical school than those who did not. The magnitude of the association decreased but remained significant after accounting for applicant characteristics such as MCAT scores, GPA, research and other experiences (adjusted odds ratio 0.73, 95% CI 0.70-0.74), as well as medical school characteristics such as geographic region, private versus public, and admissions policy regarding CC coursework (adjusted odds ratio 0.78, 95% CI 0.76-0.81). While theses findings may reflect residual confounding not adequately accounted for by their regression models (for example, no information regarding the interview was included), the authors of the study suggest that professional development for admissions officers focused on understanding the CC pathway and potential biases related to CC attendance would likely be beneficial in terms of increasing diversity in the medical school applicant pool. — Sarang Kim, MD

Are ACGME Milestones Predictive of Early Career Clinical Performance?

Maybe not, according to a recent retrospective study based on Medicare claims for common high-risk inpatient general surgical procedures (N=12,400 cases) from 2015-2018 by non-fellowship trained US general surgeons (N=701). In contrast to prior validity evidence for milestones to predict performance during residency, the authors found no significant association between the surgeon’s milestone ratings during their final year of residency training and subsequent early career clinical performance. The authors suggest the problem may lie with the milestone framework itself and/or with the process by which the milestone ratings are generated, including lack of precision in individual faculty evaluations, error generated by the Clinical Competency Committee that then uses those evaluations to generate milestone ratings, among others factors. — Sarang Kim, MD

Link To Article

Kendrick, D. , Thelen, A. , Chen, X. , Gupta, T. , Yamazaki, K. , Krumm, A. , Bandeh-Ahmadi, H. , Clark, M. , Luckoscki, J. , Fan, Z. , Wnuk, G. , Ryan, A. , Mukherjee, B. , Hamstra, S. , Dimick, J. , Holmboe, E. & George, B. (9900). Association of Surgical Resident Competency Ratings With Patient Outcomes. Academic Medicine, Publish Ahead of Print , doi: 10.1097/ACM.0000000000005157.

Teaching Clinical Reasoning Early

Clinical reasoning is either not explicitly taught or taught to more experienced learners.  In this study, educators at Weill Cornell medical school assessed the impact of a theory-informed clinical reasoning curriculum for first year medical students before they’ve had much clinical experience. The curriculum consisted of a lecture on concepts of dual process theory, script theory and cognitive biases, and 3 days of clinical reasoning exams where students reviewed History and Physical write ups (without the assessment and plan) and generated problem lists, summary statements, and leading and alternative diagnoses with rationale and illness scripts.  The study found that 80% of students had a complete problem representation, 63% of problem representations were concise, and 92% of problem representations included semantic qualifiers, suggesting that clinical reasoning can be successfully taught to inexperienced learners. Whether this translates into improved performance during clerkships remains to be seen. — Sarang Kim, MD

Link To Article

Justin J. Choi, Jeanie Gribben, Myriam Lin, Erika L. Abramson & Juliet Aizer (2023) Using an experiential learning model to teach clinical reasoning theory and cognitive bias: an evaluation of a first-year medical student curriculum, Medical Education Online, 28:1, DOI: 10.1080/10872981.2022.2153782

Unmatched and Unwanted?

A study of 206 comments on the online discussion forum Reddit reveals that between 2021 and 2022 US and Canadian medical students who did not match into residency programs showed signs of regret for having attended (and paid for!) medical school, thinking of themselves as failures and considering alternate careers.  These outcomes occurred even though medical schools likely provided (to varying degrees) mental health support, feedback on residency applications, and de-stigmatization of being unmatched.  — The Editorial Team

Link To Article

Morrissette, Anita MD1; Morrissette, Matthew MD2. “The Feeling of Failure Is Immense”: A Qualitative Analysis of the Experiences of Unmatched Residency Applicants Using Reddit. Academic Medicine ():10.1097/ACM.0000000000005120, December 12, 2022. | DOI: 10.1097/ACM.0000000000005120

Are the Increasing Numbers of Out-of-State Students at State-“Supported” Medical Schools a Good Thing?

This nationwide study of US MD-granting schools reports two concerning and possibly related trends.  Over the past 15 years or so, the number of out-of-state applicants and matriculants to state-sponsored schools has gone up substantially.  This is generally helpful to the schools’ financial status, as out-of-state students usually pay a lot more tuition than in-state students.  Over the same time period, mean medical student debt has gone up by about 75%, to eye-popping levels.  The authors surmise that in-state applicants displaced by accepted out-of-state students may attend schools which are either private or out-of-state for them, markedly increasing their cost of med school attendance. — Laura Willett, MD

Kraus, Molly B. MD1; Hasan, Shaina H. MD2; Buckner-Petty, Skye A.3; Files, Julia A. MD4; Hayes, Sharonne N. MD5; Habermann, Elizabeth B. PhD6; LeMond, Lisa M. MD7. Out-of-State Students at State Medical Schools and Increasing Medical Debt. Academic Medicine: November 08, 2022 – Volume – Issue – 10.1097/ACM.0000000000005079
doi: 10.1097/ACM.0000000000005079

Link To Article

What Do Teaching Evaluations Evaluate?

In this readable commentary about the value of teaching evaluations(TEs) by students, the authors summarize a lot of evidence suggesting that TEs measure a lot of things other than teaching, for example attractiveness and charisma, and that they should not be used as the sole or main arbiter of faculty personnel decisions.  The authors do not suggest getting rid of student-generated TEs, as students are often the only observers who can give input on items such as the learning environment and TEs may serve as a valuable source of formative feedback for faculty.  However, many weaknesses are summarized here:  TEs correlate poorly or not at all with other measures of learning;  TE ratings may be biased against female or minority race faculty;  student anonymity protects the occasional seriously unprofessional comments received by faculty; and TE ratings are adversely affected by poor grades received by students, providing a disincentive for faculty to give students negative feedback.  Overall advice on TEs:  handle with care, and develop/validate more competency-based assessments of faculty. — Laura Willett, MD

Ginsburg, Shiphra MD, MEd, PhD1; Stroud, Lynfa MD, MEd2. Necessary But Insufficient and Possibly Counterproductive: The Complex Problem of Teaching Evaluations. Academic Medicine: October 04, 2022.

Link To Article

It’s A DIY World for Medical Education Researchers

Medical education research lab is an emerging model for conducting medical education research. Defined as a distinct research team led by single or multiple PIs to study specific problems, medical education research lab is a strategic attempt at overcoming the well-recognized challenges in medical education research, such as lack of support and funding. Guided by the vision, passion, and political will of well-established investigator(s), a med ed research lab can catalyze research by developing an infrastructure for research, collaborative support, and competing for extramural funding. Using case examples, the authors of this study describe various medical education research lab models, highlighting advantages, disadvantages, and challenges, and also include a well-organized start up guide for those seeking to start a research lab. A notable shared phenotype among medical education research labs: the primary mission of the PI is research rather than teaching. The authors believe that this is a critical difference that distinguishes research labs from educational centers or teaching collaboratives. Given the typical underfunded and under-supported landscape of med ed research that is often driven by faculty who are primarily educators and not researchers, this reviewer feels that this model, however innovative and promising, is unlikely to take off in most institutions. — Sarang Kim, MD

Gisondi Michael A. MD; Michael Sarah DO, MS,; Li-Sauerwine Simiao MD, MSCR; Brazil Victoria MD, MBA; Caretta-Weyer Holly A. MD, MHPE; Issenberg Barry MD; Giordano Jonathan DO, MEd; Lineberry Matthew PhD; Olson Adriana Segura MD, MAEd; Burkhardt John C. MD, PhD; Chan Teresa M. MD, MHPE, The Purpose, Design, and Promise of Medical Education Research Labs, Academic Medicine, June 2022. 

Link to Article

MCAT, Grades Are Predictive Of Important Outcomes After All!

The investigators examined Medical College Admission Test (MCAT) scores and undergraduate grade point averages (UGPAs) from a large representative sample of students in US and Canadian MD-granting schools, and looked for correlations to multiple educational outcomes.  These outcomes included:  numeric grade in pre-clerkship courses, first attempt USMLE Step 1 grade, clerkship examination scores, clerkship GPAs, and first attempt USMLE Step 2 CK grade.  Probably the most important outcome, from the standpoint of the student, was “progress in medical school” which was defined as “completion of coursework on time or within one extra year; passing relevant licensure exams on the first attempt; and experiencing no withdrawal or dismissal from medical school for academic reasons.”  MCAT scores were a moderate-to-strong predictor of all measured outcomes, including those occurring during clerkship year or after.  For most outcomes, MCAT scores were a slightly stronger predictor than UGPA, and the combination of MCAT and UGPA was more predictive than either measure alone.  Importantly, MCAT scores were quite predictive of medical student outcome across a range of student demographics, including gender, race, and parental education. — Laura Willett, MD

Hanson JT, Busche K, Elks ML, Jackson-Williams LE, Liotta RA, Miller C, Morris CA, Thiessen B, Yuan K. The Validity of MCAT Scores in Predicting Students’ Performance and Progress in Medical School: Results from a Multisite Study. Acad Med. 2022 May 24. doi: 10.1097/ACM.0000000000004754. Epub ahead of print. PMID: 35612915.

Link To Article