Training in Neurology: Diagnostic Accuracy Among Neurology Residents

Discussion

Morning report is an essential venue for clinical learning in residency. Over 6 academic years, we have used this forum to categorize cases assessed by neurology residents and to critically evaluate patterns of diagnostic acumen. Nearly one-third of cases presented by neurology residents were ultimately deemed to be non-neurologic, most of which were medical or psychiatric in etiology. The majority of neurologic cases localized to the CNS, with the most common diagnoses being stroke and seizure. This is similar to prior work in which single residents tracked all cases seen during their residency training.^1,3,4

Overall junior neurology resident initial diagnostic accuracy when presenting their initial impressions at morning report was 64%, which is congruent with 2 prior studies.^5,6 Residents tended to overcall cases presented in morning report as neurologic but rarely missed true neurologic disease. Despite the significantly higher quantity of CNS disorders presented, resident accuracy in morning report cases was similar between CNS and PNS localizations. Patterns where error was likely to occur varied between etiologies. For example, headache was more likely than ischemic stroke to be missed at the more nuanced etiologic categorization, while stroke was more likely than headache to be missed as non-neurologic.

Several limitations of our study warrant consideration. First, a resident's true independent diagnostic assessment is difficult to capture, as residents frequently interact with coresidents, ED or other attendings, and radiologists in the course of initial consultations. We note also that the amount of supporting clinical data (e.g., laboratory studies, imaging) available at the time of first diagnostic impression varied across cases and the natural diagnostic evolution that occurs during a hospitalization may hinder determination of true accuracy. Furthermore, although we sought to determine diagnostic accuracy, there is often no gold standard for final diagnosis. These final determinations reflect best clinical judgment at the conclusion of the hospitalization or evaluation of each case. There is also challenge in categorizing complex cases into our prespecified diagnostic localizations and etiologies (e.g., vasculitis could be categorized as ischemic stroke, but also as a neuroinflammatory or headache syndrome). Furthermore, not every diagnostic category could be parsed with identical levels of granularity. Mount Sinai is a large tertiary care hospital located in New York City and thus the cases seen by residents in this urban setting may not match those seen at community hospitals or in suburban or rural settings. Finally, our data include inpatient consultations only, thus diagnoses more often seen in our outpatient clinics are necessarily underrepresented.

Importantly, analyses based on morning report presentations are subject to selection biases, as the cases residents choose for presentation may be more complex and may overrepresent neurologic disease, as junior residents are presenting to neurology attendings with the aim of learning from their expertise in localization and management. Our results and conclusions need to be considered with this limitation in mind—the cases presented in morning report reflect only a subset of all cases seen and assessed—thus patterns of clinical judgement captured in this study may not reflect diagnostic acumen more broadly. Although our data do not capture all patients evaluated by residents, we have included the entirety of the cases used for clinical teaching in the morning report venue. The analysis spans 6 academic years and more than 1,000 cases with consistency across years, which supports the internal reliability of the results. Future studies could take our CTL approach forward to address the issue of selection bias, for instance by performing a shorter study of several months' duration closing the loop on all cases, seen by all residents, in every clinical venue. This approach would complement our current study, which evaluated longitudinally all the cases presented in one educational venue, morning report, over several years.

Systematically closing the loop on morning report cases has provided important short-term feedback for residents in the group setting of morning report, allowing for expanded educational discussion facilitated by faculty and senior residents and fostering a culture of continuity and diagnostic humility. Data on a per-resident basis are used in feedback sessions between individual residents and the resident program director to develop a personalized educational plan. More broadly, the CTL methodology in morning report was a component of our adult learning theory–based curriculum, which was found to have a positive effect on Residency In-service Training Examination scores.⁷ Further work is needed to assess both subjective (e.g., survey) and objective measures of improved diagnostic acumen in future postgraduate years. We also plan to analyze cases by specific etiology, patient demographics, time of year, patient disposition, and consult venue to pursue multivariate predictors of resident errors that could inform curricular changes. Qualitative examination of incorrect cases may also allow for assessment of the role of cognitive biases (e.g., anchoring or framing) on initial diagnostic accuracy on the group level. CTL methodology is simple to employ, low cost, and can be replicated and expanded upon in other programs. Our findings can guide educational and quality initiatives to improve clinical acumen among neurology residents.