MCAT, Grades Are Predictive Of Important Outcomes After All!

The investigators examined Medical College Admission Test (MCAT) scores and undergraduate grade point averages (UGPAs) from a large representative sample of students in US and Canadian MD-granting schools, and looked for correlations to multiple educational outcomes.  These outcomes included:  numeric grade in pre-clerkship courses, first attempt USMLE Step 1 grade, clerkship examination scores, clerkship GPAs, and first attempt USMLE Step 2 CK grade.  Probably the most important outcome, from the standpoint of the student, was “progress in medical school” which was defined as “completion of coursework on time or within one extra year; passing relevant licensure exams on the first attempt; and experiencing no withdrawal or dismissal from medical school for academic reasons.”  MCAT scores were a moderate-to-strong predictor of all measured outcomes, including those occurring during clerkship year or after.  For most outcomes, MCAT scores were a slightly stronger predictor than UGPA, and the combination of MCAT and UGPA was more predictive than either measure alone.  Importantly, MCAT scores were quite predictive of medical student outcome across a range of student demographics, including gender, race, and parental education. — Laura Willett, MD

Hanson JT, Busche K, Elks ML, Jackson-Williams LE, Liotta RA, Miller C, Morris CA, Thiessen B, Yuan K. The Validity of MCAT Scores in Predicting Students’ Performance and Progress in Medical School: Results from a Multisite Study. Acad Med. 2022 May 24. doi: 10.1097/ACM.0000000000004754. Epub ahead of print. PMID: 35612915.

Link To Article

Who Are Our Medical Students (By Family Income)?

Students from high-income households are very over-represented in US medical schools.  This is a conclusion from a comparison of US Census subjects to matriculating US allopathic medical students who reported race/ethnicity and parental income on the 2017-19 AAMC Matriculating Student Questionnaires (response rates 65-71%).  A representation index (RI) was calculated for subgroups of medical students defined by race/ethnicity and family income as compared to the representation of these subgroups in the general US population.  About half of medical students come from the top 20^th percentile of family income.  Twenty-four percent of medical students come from the top 5^th percentile of family income, for a RI of 4.8.  Taken on its face, this means that young people from the top 5^th percentile of family income have approximately a 5 times higher chance of matriculating into medical school than the average US resident.  At the other end of the financial spectrum, only 25% of our students come from the lowest 60^th percentile of family income.  This under-representation was most marked for non-Hispanic white students: in this group, only 18% of students come from the lowest 60^th percentile of family income — Laura Willet, MD.

Shahriar AA, Puram VV, Miller JM, Sagi V, Castañón-Gonzalez LA, Prasad S, Crichlow R. Socioeconomic Diversity of the Matriculating US Medical Student Body by Race, Ethnicity, and Sex, 2017-2019. JAMA Netw Open. 2022 Mar 1;5(3):e222621. doi: 10.1001/jamanetworkopen.2022.2621. PMID: 35289863.

Link To Article

Widespread But Mostly Mild – Internal Residents Report Mistreatment

In this survey study of 21,931 US internal medicine residents (81% response rate), 48% reported experiencing and 62% reported witnessing “inappropriate comments or actions based on identity markers” at any time during their residency.  Fortunately, most of these incidents were reported as occurring “infrequently” or “sometimes”.  Reports were made more frequently by residents who were female, US medical grads, or native English-speakers.  The most common sources of these comments or actions were patients and their families, followed by nurses, faculty, and other residents.  These results are similar to those from prior studies of surgical residents. — Laura Willett, MD

Finn KM, O’Connor AB, McGarry K, Harris L, Zaas A. Prevalence and Sources of Mistreatment Experienced by Internal Medicine Residents. JAMA Intern Med. 2022 Feb 28:e220051. doi: 10.1001/jamainternmed.2022.0051. Epub ahead of print. PMID: 35226048; PMCID: PMC8886451.

Link To Article

Measuring Clinical Reasoning in Admission Notes?

This study examined the use of a revised assessment tool (IDEA–(I)interpretive summary, (D) differential diagnosis with commitment to most likely diagnosis, (E) explanation of reasoning for most likely diagnosis, (A) alternative diagnosis with explanation of reasoning, plus descriptive anchors) to evaluate clinical reasoning in admission notes written by internal medicine residents and fellows at a large academic medical center in New York City.  A panel of clinician educators reviewed 252 randomly selected, de-identified admission notes written by 30 different trainees.  They found that only 53% of the notes were considered high quality (score of 6 or higher on a scale of 0-10).  Intraclass correlation was high (0.84). The use of a relatively straight forward rating tool with a high reliability to assess clinical reasoning (something educators are all struggling to measure) with clinical notes that are already being generated as part of existing workflow makes this study’s findings noteworthy. The major downside would be the amount of faculty time and effort required to grade each note, as well as the fact that clinical notes were not really designed to measure clinical skills. In fact, with the transition to EHR, many have lamented the perceived deterioration of clinical notes driven by ease of copy and pasted text without much thought given to accuracy, relevance, or internal consistency, let alone documentation of clinical reasoning. Until we get buy-in from our trainees that clinical notes should reflect accurate data and sound clinical reasoning, the effort required to evaluate notes may seem to outweigh possible benefits, especially if adapted widely and routinely across a whole program. However, this study’s findings may lend support to selected use, such as facilitation of feedback for individual trainees requiring targeted remediation for clinical reasoning. — Sarang Kim, MD

Schaye, V., Miller, L., Kudlowitz, D. et al. Development of a Clinical Reasoning Documentation Assessment Tool for Resident and Fellow Admission Notes: a Shared Mental Model for Feedback. J GEN INTERN MED 37, 507–512 (2022). https://doi.org/10.1007/s11606-021-06805-6

Go To Article

Point-Of-Care Ultrasound: Progress And Problems

Ultrasound: The good news is ultrasound usage in the medical school curriculum has experienced a two-fold increase since 2014, according to a recent survey of clinical directors and curricular deans at 154 medical schools.  The bad news is there is no standardized approach to integrating POCUS into the curriculum with only 57% of schools reporting an approved POCUS curriculum and 10% a longitudinal 4-year curriculum.  Despite improvements in ultrasound technology and demonstrated benefits from its use in the clinical setting, barriers identified in previous surveys are stubbornly persistent today: lack of curriculum time, lack of funds to purchase equipment, and lack of faculty time to deliver the content. — The Editorial Staff

Link To Article

Frances M. Russell, MD, Bita Zakeri, PhD, Audrey Herbert, MD, Robinson M. Ferre, MD, Abraham Leiser, and Paul M. Wallach, MD, The State of Point-of-Care Ultrasound Training in Undergraduate Medical Education: Findings From a National Survey, Academic Medicine, 2021.

Who, And Where, Are Our Residents?

In the US, there are about 145,000 residents, of whom more than 25,000 are in subspecialty training. About 46% of them are female, in contrast to entering medical students who have been majority female since 2017.   Resident race and ethnicity, by relative frequency compared to the US population from the Census, is: White 0.69, Asian 4.9, Hispanic 0.48, Black 0.44, Multiracial 1.2, and American Indian/Alaska Native/Native Hawaiian/Pacific Islander 1.8.  All of these ratios are likely somewhat under-estimated, as 12% of residents were listed as of Other/unknown race.  They graduated from US/Canadian allopathic schools (61%), international medical schools (23%), and osteopathic schools (16%).  US allopathic graduates are over-represented in the higher-paying specialties.  International medical graduates are over-represented in pathology (49%), neurology (32%), and internal medicine (39%) and its subspecialties.  Osteopathic physicians are over-represented in family medicine (32%) and emergency medicine (26%).On average, the US has 43 resident physicians per 100,000 population, but these are not evenly distributed across state populations.  Three states (New York, Massachusetts, and Rhode Island) have more than 80 residents per 100,000 population, while three others (Alaska, Montana, and Idaho) have fewer than 10 per 100,000.  This may represent resources available to rural vs urban health care, or historical graduate medical education financing trends. — Laura Willet, MD

Sarah E Brotherton, Sylvia I Etzel, Graduate Medical Education, 2019-2020, JAMA 2020 Sep 22;324(12):1230-1250. doi: 10.1001/jama.2020.14635

Link To Article

To Signal Or Not To Signal, That Is The Question For Residency Applicants And Programs

This study describes the initial findings from the preference signaling program that was implemented for the 2021 otolaryngology residency match which allowed applicants to identify up to 5 programs of particular interest at the time they submitted their initial application, allowing them an additional opportunity to stand out to their favored program. The findings, had they been available a few weeks earlier, would have been of high interest to students applying to General Surgery, Internal Medicine, and Dermatology this year as these specialties also started a preference signaling program, and students and programs were uncertain about the potential value or use of such a program. While the survey response rate in this study was low (42% (N=233) of applicants and 52% (N= 62) of program directors), there are interesting findings to take note of. First, the rate of receiving an interview offer was significantly higher from signaled programs (58%) than from non-signaled programs (14%; P < .001), and the association was consistent across the spectrum of applicant competitiveness. Program directors most commonly reported using signals as a tiebreaker for similar applications and as part of an initial application review process. One program required a signal to offer an interview, and 18% (11/61) of program directors reported using signals to rank candidates. The most common theme in applicants’ comments was a recommendation to increase the transparency and consistency of how programs use signals. The majority of applicants (178, 77%) and program directors (53, 91%) strongly favored continuing the program.  

Whether this program results in a “better” match between applicants and programs, reduces the burden of application inflation, or if there might be differences in how signals are used across racial or other demographic groups remain to be seen for otolaryngology now on its second year of this program, or for the 3 new specialties that started the program this year.  — Sarang Kim, MD

Link To Article

Pletcher, Steven D. MD; Chang, C.W. David MD; Thorne, Marc C. MD, MPH; Malekzadeh, Sonya MD, The Otolaryngology Residency Program Preference Signaling Experience, Academic Medicine October 5, 2021 doi: 10.1097/ACM.0000000000004441

Exams Might Actually Work

Link To Article

Maybe the American Board of Internal Medicine certification exam actually works — at least as far as determining how often internists prescribe opioids for back pain is concerned. In a study of 10246 mid-level physicians, researchers looked at 55,387 low back pain visits (average patient age 76.2) between 2009 and 2017 and found that internists who scored in the upper quartile on the exam prescribed opioids less often than those in the lowest quartile between 2015 and 2017– a time when guidelines for opioid use changed.  Prior to that time period, there was no distinction between scorers in the two quartiles.  The findings of this study seem to suggest that, when necessary, the top scorers on the certification exam may be more likely than their peers to make the proper adjustments to prescribing behavior — an indication, perhaps, that the exam can predict some real-world outcomes.  – The Editorial Team  

Link To Article   

Bradley, Vandergrift, Weng et al Clinical Knowledge and Trends in Physicians’ Prescribing of Opioids for New Onset Back Pain, 2009-2017, JAMA Netw Open. 2021;4(7):e2115328. doi:10.1001/jamanetworkopen.2021.15328

Should Pre-Med Students Be Required To Pay Tuition To Do Useful Work As Medical Scribes?

Medical Scribes — Lin et al presents a compelling program for pre-medical students that serves to both provide students with strong clinical experiences as well as provide physicians with practical assistance in documentation required in the health care setting.  The main focus of the program is having students trained and serve as scribes, yet the program also includes faculty teaching at the bedside, giving feedback on applications, and mentoring on scholarly projects.  The program does charge a tuition and so allows the school to use this resource to hire two program managers that serve as chief scribes and communicators of opportunities.  Data was presented to attest to student satisfaction, production of abstracts, and reflection on personal growth.  Yet, the one in three acceptance to a health professions school does not seem as high as what might be expected with the amount of hands on support students’ obtain.  The data on physician/provider outcomes has been presented in other publications with an additional measure of self-report improved joy of practice presented here.  As admissions becomes more competitive with increased applications, the clamor for access to clinical experience has increased. This program provides one method of serving both the physician and pre-med population.  Whether it provides adequate outcomes for the pre-meds in terms of their goal to be accepted to a health professions school and whether it would meld with different practice structures and faculty needs, are considerations each school needs to consider within their own environment.  — Liesel Copeland, PhD

Lin S, Duong A, Nguyen C, Teng V. Five Years’ Experience With a Medical Scribe Fellowship: Shaping Future Health Professions Students While Addressing Provider Burnout. Acad Med. 2021 May 1;96(5):671-679. doi: 10.1097/ACM.0000000000003757. PMID: 32969839.

Link To Article

Out Of Sight, But Not Out Of Mind

Depression: Willingness among medical trainees to seek help for a mental health issue gets a boost when faculty members share their own struggles.  At least that is the finding from a limited study of 39 residents who attended a closed conference on the subject given by three self-selected faculty at Mayo Clinic Rochester.  On an anonymous survey given to residents who attended the event, 89.7% stated their knowledge of available mental health resources increased, 84.7% said they were more likely to pursue mental health resources, and 97.4% replied that faculty sharing their challenges de-stigmatizes mental health.  These are impressive stats, but a post-conference afterglow effect might be at play here.  The authors acknowledge as much by emphasizing the need to study whether residents retain their enthusiasm over time, and more importantly, they seek that help they need.  Despite its limitations, this study opens a window on depression among residents which remains too often out of sight, but not out of mind. — The Editorial Team

Brianna E Vaa Stelling, Colin P West, Faculty Disclosure of Personal Mental Health History and Resident Physician Perceptions of Stigma Surrounding Mental Illness, Academic Medicine January 2021. 

Link To Article