If You Can’t Say Something Nice, Don’t Say Nothing At All: Thumper’s Rule And How Faculty Interpret Narrative Comments On Resident Evaluations

evaluations  In this qualitative study using constructivist grounded theory analysis, investigators describe how faculty interpret narrative comments on residents’ in-training evaluation. De-identified narrative comments of 126 residents were distributed to 24 faculty who were each asked to review 15-16 documents and sort them into four categories ranging from A for outstanding to D for unacceptable, and then rank-order the residents within each category. Semi-structured interviews were conducted to assess how faculty decided to categorize and rank-order residents. Results show that all faculty read between the lines to understand narrative comments. “Excellent” was often interpreted as “middle of the road,” and “good” was considered to actually mean “below level, needs work.” Faculty reported scanning for “red flags” to identify the most relevant cues among all the comments- examples of positive red flags included “chief resident material, future colleague, superstar,” and examples of negative red flags included “solid, good, improving, functioning at expected level.” While faculty also considered consistency and specificity of comments to categorize residents, it’s interesting to note that faculty assume what is written is often not what is meant, and instictively and actively search for hidden meaning. So why do we not just say what we mean? Maybe our feelings about politeness may have something to do with it. The saying “if you can’t say something nice, don’t say nothing at all,” (also known as Thumper’s rule from the Disney movie Bambie), may impede our ability to describe and interpret trainee performance accurately. — Sarang Kim, MD

Medical Student’s Perspective: This article is valuable in exploring how faculty interpret narrative comments on resident evaluations. I was not surprised by the need for illustrative examples to make residents stand out nor by many evaluations blending together. Scanning for positive and negative red flags in such letters seems a natural method for differentiation. I was surprised, however, to see generalized skepticism towards many commonly used descriptive words. It was also surprising that personality trait praises were the most common commentaries but also some of the most disregarded. The most concerning idea revealed was the differing opinions on the purpose of resident evaluations and the resulting variation in commentary content. Some believed they were for resident feedback while others felt their purpose was for program directors to use in generating recommendation letters for fellowship or job applications. Since these goals can seem misaligned, I believe the best standardized approach would be a separate forum of honest feedback documented during rotations along with a summative commentary to highlight exemplary qualities or concerns. This way, the program director stays better abreast of resident development and can evaluate residents based on fuller commentary. — Victoria Behrend, MD Candidate 2015 Rutgers Robert Wood Johnson Medical School

Resident’s Perspective: As one progresses through medical training, less emphasis is placed on objectively based knowledge assessments and shifted towards largely narrative-based assessments of clinical performance.  Although these narrative-based assessments serve many purposes, one core objective is to provide residents constructive feedback to promote professional growth. Faculty physicians face a troubling dilemma when attempting to provide this feedback effectively. They must provide honest and constructive criticism relating to areas where individual residents can improve. However, they must do this in a way that preserves a working teacher-student relationship, protects the psyche of the resident, and not compromise future fellowship/job opportunities for the resident.  The “hidden code” referred to by the authors of this study allows faculty physicians to address many of these issues. Unfortunately though, resident physicians have little experience deciphering the “hidden code”.  The residents may believe they are meeting expectations and get a false sense of security when in fact the faculty might perceive them as struggling. This raises the question of whether the current use of narrative-based assessments, embedded with hidden codes, is ultimately doing residents- and possibly even their patients- a disservice. I would argue, yes. — James Penn, MD (PGY 3) Rutgers Robert Wood Johnson Medical School, Internal Medicine Residency Program

Ginsburg S, Regehr G, Lingard L, Eva KW. Reading between the lines: faculty interpretations of narrative evaluation comments. Med Educ 2015;49:296-306

Link To Article

Advertisements
This entry was posted in Graduate Medical Education and tagged , , . Bookmark the permalink.