Abstract
Objective: To assess the feasibility and accuracy of telemedical concussion evaluations (teleconcussion) for real-time athletic sideline assessment of concussion, as such assessment may address the gap in access some populations of athletes have to providers with expertise in concussion evaluation.
Methods: A cohort of 11 consecutive male collegiate football players with suspected concussion was assessed using Standardized Assessment of Concussion (SAC), King-Devick test (K-D), and modified Balance Error Scoring System (mBESS). A remote neurologist assessed each athlete using a telemedicine robot with real-time, 2-way audiovisual capabilities, while a sideline provider performed a simultaneous face-to-face assessment. After the assessment, a remove-from-play (RFP) determination was made. The remote and the face-to-face providers were blinded to each other’s examination findings and RFP decision until the end of the assessment.
Results: The teleconcussion and face-to-face SAC were in agreement 100% of the time (6/6; 95% confidence interval [CI] 54%–100%). The mean (SD) difference between remote and sideline K-D times was 0.7 (1.4) seconds. Remote and sideline K-D times were within a 3-second difference 100% of the time (11/11; 95% CI 72%–100%). Remote and sideline mBESS scores were within 3 points 100% of the time (6/6; 95% CI 54%–100%). RFP decisions were in agreement 100% of the time (11/11; 95% CI 72%–100%).
Conclusions: The aim of this study was to investigate the feasibility of teleconcussion for sideline concussion assessments. These data suggest a high level of agreement between remote and face-to-face providers with regard to examination findings and RFP determinations.
GLOSSARY
- CI=
- confidence interval;
- K-D=
- King-Devick test;
- mBESS=
- modified Balance Error Scoring System;
- RFP=
- remove-from-play;
- SAC=
- Standardized Assessment of Concussion;
- TBI=
- traumatic brain injury
An estimated 1.6–3.8 million traumatic brain injuries (TBIs) occur annually in the United States among 38 million children and adolescents and 170 million adults engaged in organized physical and sports-related activities.1 To help improve concussion and TBI identification and postinjury care, some sports organizations have employed unaffiliated neurotrauma consultants to provide sideline evaluation for suspected injuries.2
This same level of care is not available for most youth and collegiate-level athletes. Of more than 4,000 US colleges and universities,3 most have qualified medical staff providing care for elite-level sports teams; however, most have no access to neurotrauma experts on the sideline. Moreover, among the 37,000 US public and private secondary schools,4 58% do not have access to any sideline medical personnel (including athletic trainers).5 This problem may be compounded by geographic disparities as rural colleges, universities, and high schools may have even less access to specialty care than schools in major metropolitan areas.
Telemedicine has been shown to be a safe and effective means to evaluate and treat numerous acute neurologic conditions, including stroke,6 in populations with limited access to subspecialty care. Similarly, telemedicine evaluation for concussion (hereafter referred to as teleconcussion) may also allow the higher-level care seen in professional sports to be delivered to underserved populations including high school, collegiate, and rural athletes.3,7 The present study aimed to assess the feasibility and accuracy of teleconcussion evaluations of male collegiate football players evaluated for suspected concussion.
METHODS
A cohort of adult male collegiate athletes was recruited and enrolled from the active roster of the Northern Arizona University football team during preseason sports physical examinations. All consenting athletes received face-to-face baseline examinations from the study team, including a symptom severity checklist, Standardized Assessment of Concussion (SAC), King-Devick test (K-D), and modified Balance Error Scoring System (mBESS). Over the course of 2 seasons of competition, all athletes with suspected concussion were evaluated by the Northern Arizona University medical team in the athletic training room or on the sideline. For the duration of the face-to-face history and examination, a remote neurologist was present and independently scored the same examination via telemedicine robot (VGo; VGo Communications, Inc., Nashua, NH) (figure 1). During each football game, the remote neurologist was logged on continuously to the telemedicine robot so as to provide rapid and efficient evaluations of the athlete while simultaneously monitoring athletes for possible injuries through live game broadcast (figure 2).
VGo remote telepresence robot at Northern Arizona University's J. Lawrence Walkup Skydome. Image used with permission from Vecna Technologies, Inc.
(A) Remote neurologist logged into the telepresence robot using a laptop computer while simultaneously watching live game broadcast. Image of King-Devick Test used with permission. (B) Remote neurologist point of view, showing an on-field perspective from sideline telepresence via tablet device and a secondary on-field perspective via live game broadcast. Image reproduced with permission of Springer, © Springer Science+Business Media New York 2015. (The emerging role of telemedicine in the evaluation of sports-related concussion. In: Tsao JW, Demaerschalk, BM, editors. Teleneurology in Practice: A Comprehensive Clinical Guide. New York: Springer; 2015:159 –165.)
After the face-to-face examination, the remote neurologist had the ability to ask additional questions and repeat any portion of the physical examination. Final, blinded remove-from-play (RFP) determinations were made by the face-to-face evaluator and the remote evaluator. Both the face-to-face and the remote examiners were blinded to each other’s examination findings and RFP determinations until the face-to-face medical staff rendered a final diagnosis and disposition.
For the purposes of this study, the face-to-face examination and clinical decisions of the primary medical team were both final and the gold standard against which teleconcussion assessments and decision-making were compared. Confidence intervals (CIs) were calculated by using the exact binomial method.
Standard protocol approvals, registrations, and patient consents.
Before participant recruitment, the Mayo Clinic Institutional Review Board provided approval for this study.
RESULTS
In total, 123 athletes were enrolled in the study, 50 of whom participated through 2 athletic seasons. Over these 2 seasons, 11 consecutive athletes with suspected concussion were evaluated. The specific tests and testing protocols used for each assessment were made by the face-to-face providers. Because of the efficiency needed for game time assessments, some athletes were removed from play based only on history and K-D performance. K-D scores and RFP decisions were obtained from remote and sideline assessments for all 11 athletes evaluated for suspected concussion. SAC and mBESS scores were obtained for 6 of the 11 athletes who were not removed from play on the basis of history and K-D scores.
The mean (SD) difference between remote and sideline K-D times was 0.7 (1.4) seconds and the times were within a 3-second difference in 100% of cases (11/11; 95% CI 72%–100%). The teleconcussion and face-to-face SACs were in agreement 100% of the time (6/6; 95% CI 54%–100%). Remote and sideline mBESS scores were within 3 points 100% of the time (6/6; 95% CI 54%–100%). RFP decisions were in agreement 100% of the time (11/11; 95% CI 72%–100%).
DISCUSSION
This aim of this study was to objectively determine the accuracy of teleconcussion assessments compared with the gold standard of face-to-face assessments in real-world scenarios during competition. This feasibility study suggests that teleconcussion evaluations are safe and effective for providing accurate and rapid assessments of athletes and RFP determinations at the time of suspected injury.
The authors acknowledge some limitations in this study. The sample size is small; however, the aim of the study was to provide pilot data for formulating larger, more comprehensive future studies, which will be necessary to reach more definitive conclusions. The remote and face-to-face examiners were not the same for every assessment (examinations were conducted among a pool of 2 remote and 2 face-to-face examiners). Another limitation is that the examiners were not compared with each other at baseline to demonstrate standardization and agreement of examination interpretation and scoring. However, this lack of comparison did not likely affect the outcome because the agreement level on examination scoring was high (even on subjective measures such as mBESS), with full agreement on final RFP decision making. This study did not address whether using a robotic unit vs less-expensive, more portable technology influences the ease and accuracy of assessments. Although the authors infer that teleconcussion assessments can be conducted in the absence of face-to-face personnel, this hypothesis was not meant to be supported with these data.
Previously, telemedicine has been shown to be potentially effective for the remote neurologic evaluation of TBI.8 A body of evidence now exists that telerehabilitiation is potentially effective at directing functional recovery in patients with TBI.9 Videoconference-based telemedical neuropsychological assessments have been shown to be as accurate as face-to-face interactions.10 Furthermore, the first published teleconcussion evaluation noted an ability to perform a neurologic examination of the athlete in the emergency department while remotely reviewing pertinent imaging studies and performance on after-injury computerized cognitive testing3 (figure 3).
Remote evaluation of additional clinical data, including computerized cognitive testing and imaging studies.
The conclusions from these studies suggest that teleconcussion is an accurate tool that may be used throughout the course of an athlete's concussion care—from the hyperacute and acute stages (on the sideline or in the emergency department) through recovery (with clinical assessments of neurologic and neuropsychometric function paired with physical and cognitive rehabilitation). The authors hypothesize that eventual return-to-play decisions can be made telemedically because these decisions are frequently based on interpretation of histories and physical examination components similar to those used for RFP decision-making. The accuracy of teleconcussion for return-to-play decisions is beyond the scope of this study and should be examined in greater detail with future research.
This study quantitatively examines the feasibility and accuracy of telemedicine for sports-related concussion at the time of injury. It addresses possible solutions for existing gaps in medical care for youth athletes in particular who are not only more susceptible to the effects of concussion but who also have fewer resources than elite-level athletes for proper concussion education, identification, and care. In addition to the need for more research demonstrating the safety and efficacy of teleconcussion for RFP and return-to-play decisions, the cost-effectiveness of these assessments, and the need for a face-to-face provider to assist with the examination, should also be appraised critically.
AUTHOR CONTRIBUTIONS
Bert Vargas: study concept and design, acquisition of data, and drafting/revision of manuscript for content. Morgan Shepard: acquisition of data, critical revision of manuscript for intellectual content. Joseph Hentz: analysis and interpretation of data, critical revision of manuscript for intellectual content. Cherisse Kutyreff: study concept and design, acquisition of data, and drafting/revision of manuscript for content. L. George Hershey: study concept and design, acquisition of data, critical revision of manuscript for intellectual content. Amaal Starling: study concept and design, acquisition of data, and drafting/revision of manuscript for content.
STUDY FUNDING
No governmental sponsorship or funding was used for this study. This study was funded by the Mayo Clinic Center for Innovation.
DISCLOSURE
B. Vargas received honoraria for serving on scientific and clinical research advisory boards for Pernix Therapeutics and Alder Biopharmaceuticals. He has also agreed to receive honoraria from Eli Lilly and Company and Avanir Pharmaceuticals for clinical research consulting and participation on clinical research advisory boards. He serves on the speakers' bureau for Avanir Pharmaceuticals. M. Shepard, J. Hentz, C. Kutyreff, G. Hershey, and A. Starling report no disclosures relevant to the manuscript. Go to Neurology.org for full disclosures.
Footnotes
Go to Neurology.org for full disclosures. Funding information and disclosures deemed relevant by the authors, if any, are provided at the end of the article.
- Received July 17, 2016.
- Accepted in final form January 24, 2017.
- © 2017 American Academy of Neurology
Letters: Rapid online correspondence
REQUIREMENTS
You must ensure that your Disclosures have been updated within the previous six months. Please go to our Submission Site to add or update your Disclosure information.
Your co-authors must send a completed Publishing Agreement Form to Neurology Staff (not necessary for the lead/corresponding author as the form below will suffice) before you upload your comment.
If you are responding to a comment that was written about an article you originally authored:
You (and co-authors) do not need to fill out forms or check disclosures as author forms are still valid
and apply to letter.
Submission specifications:
- Submissions must be < 200 words with < 5 references. Reference 1 must be the article on which you are commenting.
- Submissions should not have more than 5 authors. (Exception: original author replies can include all original authors of the article)
- Submit only on articles published within 6 months of issue date.
- Do not be redundant. Read any comments already posted on the article prior to submission.
- Submitted comments are subject to editing and editor review prior to posting.
