Education Research: Can my electronic health record teach me something?

DISCUSSION

Given the nascent technology, adoption of fully functional EHRs with integrated knowledge management systems is currently low.^9,10 Therefore, little is known regarding how and to what extent these tools can be implemented in academic training settings. Perhaps most important are the time challenges in a real-world practice setting. We sought to assess clinicians' perceived utility, user preferences, and barriers to implementation regarding a novel knowledge management tool developed at UMMSM.

The major limitation of our study was the low response rate. We attempted to minimize this by sending several e-mails over several weeks. However, only 18% of those initially e-mailed actually participated in this study and the response rate was even lower (7.4%) for the second simulation. While this response rate is similar to one published industry average for medical services, our goal was higher and may have been improved with a more optimal “call-to-action” subject heading.¹¹ Variability in recruitment style by institution likely also contributed. Time pressures may also have played a role. The most conservative interpretation would be that the nonresponders were not interested in this type of technology and would have rated its utility poorly. However, the response rate might have been higher if the software had sent e-mails about clinicians' real patients rather than mock patients. We postulate that the lower response rate for the second case represented responder fatigue.

A vast majority of respondents agreed that there is a need for better EHR-based tools to improve medical education and clinical decision-making. The software consistently identified resources that “could change practice,” demonstrating the ability of a basic search algorithm to identify valuable clinical resources. There was also clinician demand for innovative education approaches such as EHR-linked examination preparation and EHR-linked continuing medical education. Interestingly, faculty were more likely to express interest in using the technology for learning outside their own specialty. Also, despite the continuing trend toward using handheld devices as medical schools adopt these devices in teaching and patient care, clinicians preferred desktop to device access. In general, clinicians worried that too much information would be presented. Results were similar across specialty and institution, lending support to the generalizability of these findings.

Several prior studies have demonstrated the efficacy, and the potential impact on practice, of knowledge management technologies in health care. The KnowledgeLink Infobutton at Partners Healthcare answered 84% of clinical questions, and altered 15% of patient care decisions.⁶ At a different institution, 74% of Infobutton users felt it had a positive impact on patient care, with 20% reporting a specific positive impact.⁵ In one study, Infobutton technology retrieved pertinent papers to over 55% of online discussion threads.⁶

Our study has other limitations. Our survey was not validated, but we used closed-ended questions, with simple wording and balanced rating scales, and attempted to provide logically ordered nonleading questions. With 154 unique respondents, our margin of error was approximately 8%. Stratified random sampling, which improves the representativeness of the sample population to the population as a whole, was not performed. Furthermore, when asked hypothetically outside of a busy clinical setting, clinicians might optimistically rate the utility of available resources. However, click tracking showed that those who rated a given resource highly (4 or 5 stars) also took the time to click on the resource. Finally, participation rates varied among programs and we did not collect information about nonresponders. This would contribute to a nonresponder bias if differences in recruitment approaches led to differences in characteristics of responders. Strengths of the study include the multi-institutional design and the inclusion of more than one medical specialty.

Our findings suggest that information technology tools should be integrated, with protected time, into graduate and continuing medical education, yet wisely and sparingly. We identified that time is the most important barrier to implementation. The low response rate may have represented this aforementioned time barrier, or may have represented a lack of perceived utility, or a combination of the two. If this software were delivered during protected time, relating to real cases with active knowledge gaps, even more favorable survey results might have ensued. Identifying the most important types of cases (e.g., those with common patient safety issues) or targeting clinical weaknesses might be another way to address time challenges.

There is value in having a venue, or a social space, that enables both explicit (e.g., research articles) and tacit (e.g., clinical experience) knowledge sharing to take place.¹² Mobley and Rosenberg¹³ recently predicted that “trainees will benefit by search engines specifically targeted for the care of neurology patients.” In future steps, we hope to assess the effectiveness of this tool, vs comparable knowledge technologies and vs independent research, in the context of protected time. We hope to identify whether these tools can improve medical knowledge and clinical decision-making, as measured by the Internal Medicine and Neurology Script Concordance Tests.¹⁴ We might find that, indeed, these tools can achieve important quality-of-care goals by freeing clinicians from spending time looking for relevant patient information, and allowing them instead to spend time delivering optimal care.