Abstract
Objective To determine the accuracy of automated detection of generalized tonic-clonic seizures (GTCS) using a wearable surface EMG device.
Methods We prospectively tested the technical performance and diagnostic accuracy of real-time seizure detection using a wearable surface EMG device. The seizure detection algorithm and the cutoff values were prespecified. A total of 71 patients, referred to long-term video-EEG monitoring, on suspicion of GTCS, were recruited in 3 centers. Seizure detection was real-time and fully automated. The reference standard was the evaluation of video-EEG recordings by trained experts, who were blinded to data from the device. Reading the seizure logs from the device was done blinded to all other data.
Results The mean recording time per patient was 53.18 hours. Total recording time was 3735.5 hours, and device deficiency time was 193 hours (4.9% of the total time the device was turned on). No adverse events occurred. The sensitivity of the wearable device was 93.8% (30 out of 32 GTCS were detected). Median seizure detection latency was 9 seconds (range −4 to 48 seconds). False alarm rate was 0.67/d.
Conclusions The performance of the wearable EMG device fulfilled the requirements of patients: it detected GTCS with a sensitivity exceeding 90% and detection latency within 30 seconds.
Classification of evidence This study provides Class II evidence that for people with a history of GTCS, a wearable EMG device accurately detects GTCS (sensitivity 93.8%, false alarm rate 0.67/d).
Glossary
- CI=
- confidence interval;
- EDDI=
- Epileptic seizure Detector Developed by IctalCare;
- EMU=
- epilepsy monitoring unit;
- FAR=
- false alarm rate;
- GTCS=
- generalized tonic-clonic seizures;
- STARD=
- Standards for Reporting of Diagnostic Accuracy;
- SUDEP=
- sudden unexpected death in epilepsy
Generalized tonic-clonic seizures (GTCS) may lead to injuries, and constitute the major risk factor for sudden unexpected death in epilepsy (SUDEP), especially in unattended patients.1,2 Wearable seizure detection devices were suggested to prevent SUDEP.1,–,4 The unpredictability of seizure occurrence is distressing for patients and caregivers.3 It contributes to social isolation and decreased quality of life, and surveys showed a need for wearable seizure detection devices.5,–,10
Therapeutic decisions and clinical trials use self-reporting of seizures. However, this is unreliable: studies in video-EEG monitoring units demonstrated that 61% of seizures remain unrecognized by patients, even secondarily generalized tonic-clonic seizures, which remained completely unrecognized.11 Wearable devices could provide objective data about seizure occurrence.
Signals from muscles, recorded with surface EMG, seem to be a promising modality for detecting motor seizures. A recently published review summarizes the specific pathophysiologic changes in muscle activity during seizures.12 Quantitative EMG parameters differentiate epileptic from physiologic muscle activation and from nonepileptic seizures.13,–,15 Based on characteristic changes in frequency and amplitude,16 we have developed a generic seizure detection algorithm.17 High sensitivity and specificity of EMG-based seizure detection algorithms were found in 2 retrospective studies using offline analysis of EMG recorded with conventional amplifiers.17,18 A pilot study showed promising results for a wearable EMG-based device implementing our algorithm.19
We present a prospective, multicenter study on the diagnostic accuracy of real-time detection of GTCS using the wearable EMG device. We report our findings according to the Standards for Reporting of Diagnostic Accuracy (STARD) criteria.20
Methods
Patients
Seventy-one consecutive patients (32 female; age 10–62 years, mean 34.1 years, median 34 years) were recruited to the study: 33 patients at the Danish Epilepsy Centre, Dianalund; 5 patients at Copenhagen University Hospital Rigshospitalet, Denmark; and 33 patients at the National Centre for Epilepsy, Oslo University Hospital, Norway, between October 21, 2014, and January 23, 2017. The median baseline monthly frequency of GTCS according to the patients' seizure logs was 1.75.
Inclusion criteria were (1) admission to noninvasive long-term video-EEG monitoring in the epilepsy monitoring unit (EMU) in one of the participating centers; and (2) historical data suggesting that the patient had GTCS. Exclusion criteria were (1) age under 3 years; (2) pregnancy; (3) allergy to the self-adhesive patch; or (4) implanted vagal nerve stimulator or deep brain stimulator.
Patients underwent video-EEG monitoring using wireless amplifiers (at the Danish Epilepsy Centre) or connected with a 10-meter-long cable (in the other centers). The mobility of the patients was not restricted in the EMUs, where patients were free to perform their habitual physical activities and participate in activities such as using an exercise bicycle, preparing coffee or food, and playing home video games involving body motion (Nintendo [Kyoto, Japan] Wii).21 Patients were under continuous surveillance and no injuries occurred during monitoring.
Standard protocol approvals, registrations, and patient consent
The project has been approved by the regional ethics committees (in Denmark: SJ-380; in Norway: 2013/2354). All patients (or guardians of patients) gave their written, informed consent prior to the study.
Wearable EMG-based seizure detection device
We tested the diagnostic accuracy of the Epileptic seizure Detector Developed by IctalCare (EDDI). The device measures surface EMG signals and was placed on the brachial biceps muscle (figure 1, A–D). The device is relatively small, and is easily hidden under the patient's clothes. The choice of side for the placement was decided based on historical data where the patient's seizures habitually were most strongly expressed. When there was no known side preponderance, the device was placed on the left.
(A) The wearable device placed on the brachial biceps muscles. (B, C) The wearable device, which is connected to the self-adhesive patch, containing the recording electrodes and the ground electrode. (D) Remote control of the device.
The wearable device is attached to a self-adhesive hypoallergenic hydrogel patch (Axelgaard Manufacturing Co. Ltd., Fallbrook, CA) with 3 embedded surface EMG electrodes (bipolar recording electrodes and ground electrode), developed specifically for this device (figure 1C). The distance between the recording electrodes was 20 mm. A lithium polymer battery was placed into the wearable device.
Sampling frequency was 1,024 Hz. A modified version of the previously reported algorithm was implemented into the device.17,19 Briefly, the electrophysiologic biomarker of the initial phase of GTCS consists of an increase in amplitude and high-frequency (>150 Hz) oscillations in the EMG signal. This was expressed as the number of zero-crossings, where high frequency oscillations (high-pass filtered at 150 Hz) exceeded a hysteresis of ±35 μV. The count of zero-crossings was calculated for a window of 1 second, with an overlap of 75% with the previous window. When this exceeded the predefined threshold, the alarm was triggered (figure 2). The measures are generic (not patient-specific), they were prespecified before this study, and they were applied to all patients. The algorithm requires minimal computational power.
(A) The EMG signals recorded by Epileptic seizure Detector Developed by IctalCare (EDDI) during a generalized tonic-clonic seizure (high-pass filter: 150 Hz). (B) The number of zero-crossings calculated from the signal in (A). The horizontal axis in both (A) and (B) is the time (in seconds). When the number of zero-crossings exceeds the threshold (dotted horizontal line) for the prespecified time window (yellow line), then the seizure alarm is triggered (red vertical arrow).
The device gives real-time seizure alarms and a different type of alarm for device deficiency, when electrode impedance is out of bounds. However, the study design implied blinding of the personnel for any data from the device, and therefore these alarms were disabled in this study. Instead, the device logged real-time all-seizure alarm time points, for comparison with the reference standard, and the periods with high impedance and 50-Hz artefacts were considered device deficiency periods. Detected seizure time points were read blinded to any other data. In addition, the EMG recorded by EDDI was transmitted via radio signals to a computer that stored the signals. The intention was to use these signals for further optimization and fine-tuning the measures of the algorithm. However, that is not reported in this study. All data presented in this study are based on real-time seizure detection using the predefined measures and the algorithm described above, with all calculations done on the wearable device.
The self-adhesive patch was changed once per day (standard procedure). The battery was changed 3 times per day; this is more than the standard procedure for the device, and it was necessary because in the study EMG data were transmitted to and saved on a computer. This increased significantly the power consumption of the device. For standard operation of EDDI (real-time computation, giving alarms and logging time points, without transmitting EMG data), change of the battery is required once per day.
Reference standard
The reference standard for identifying seizures was long-term video-EEG monitoring, reviewed by trained, board-certified clinical neurophysiologists and epileptologists, with more than 10 years’ experience with EMU (S.B., O.H., and M.F.). The experts were blinded to all data from EDDI, until they completed the analysis of the video-EEG recordings. For GTCS, they identified the start time point at the beginning of the tonic phase and the end time point at the end of the clonic phase. The time points of the reference standard were then compared with the logs of the real-time automated seizure detection. At the beginning of each monitoring, the times in the device and in the video-EEG monitoring equipment were synchronized.
Study outcome measures
The primary aim was determining the diagnostic accuracy of the device. Sensitivity was expressed as the percentage of the detected GTCS. Specificity was expressed as the false alarm rate (FAR), calculated from the entire patient population.
In addition, we documented the following: adverse events (any patient concern related to the device, reported by the patient or observed by the personnel), device-off time (total time when the device was turned off during monitoring—for example, for changing patch and battery, time in bathroom, ictal SPECT), device-deficiency time (periods with high impedance and artefacts preventing the functioning of the device; this was expressed as percentage of the total time when the device was turned on—i.e., the total recording time), and detection latency (measured from the start time point identified by the experts to the alarm time point from the device).
Classification of evidence
The primary research question was the accuracy (sensitivity and false-alarm rate) of a wearable EMG device for detecting GTCS. This study provides Class II evidence that for people with a history of GTCS, the wearable EMG device, EDDI, detects GTCS with a sensitivity of 93.8% and a false-alarm rate of 0.67/d.
Sample size estimation
For the expected sensitivity of 95%17,19 and expected prevalence of 28% in the studied population,22 with a confidence interval (CI) of 10%, we needed 18 patients with GTCS and 65 recruited patients.23 GTCS is the seizure type with the highest risk for the patients and therefore considered an undesired event in EMUs.24 Diagnostic tapering of antiepileptic medication is aimed at increasing seizure frequency though avoiding GTCS unless this is the targeted seizure type, and these occur relatively rarely in EMUs. Our intention was to record GTCS from 20 patients, with a total of at least 30 GTCS. Since the quantitative EMG changes during GTCS have a high intraindividual and interindividual consistency, this number seemed to be sufficient to evaluate the performance of the device.14
Results
The mean recording time per patient was 53.18 hours (median 53.12; range 2.3–130 hours). The device was turned on for 3,928.6 hours. Device deficiency occurred in 193.1 hours (4.9%). The total recording time was 3,735.5 hours. No adverse events occurred.
Figure 3 shows the STARD flowchart of the study. Thirty-two GTCS occurred during the recording time. Twenty of the 71 included patients had GTCS (11 patients had 1 seizure each; 6 patients had 2 seizures; 3 patients had 3 seizures).
DBS = deep brain stimulation; EDDI = Epileptic seizure Detector Developed by IctalCare; GTCS = generalized tonic-clonic seizure.
The sensitivity of the device was 93.8% as it detected 30 out of the 32 GTCS (95% CI 86%–100%). All GTCS were detected in 18 patients (90% of all patients with GTCS). Two patients with 2 seizures had only 1 seizure detected. Calculated per patient, the mean sensitivity was 95% (median 100%; range 50%–100%). Four seizures from 3 patients with idiopathic generalized epilepsy were primary GTCS; all of them were detected. The other seizures were secondarily GTCS.
Both missed GTCS occurred as the second seizure after a detected GTCS, with short interseizure periods (19 minutes and 98 minutes, respectively). Retrospective inspection of the EMG signals during the missed seizures showed low amplitude of the EMG signal in one case, and very short tonic phase in the other case, so that the number of zero-crossings did not exceed the predefined detection threshold for any of the missed seizures.
The median detection latency of the device was 9 seconds (range −4 to 48 seconds; mean 12.3 seconds), measured from the start of the tonic phase of the seizures.
The false discovery rate was 0.02/h (i.e., 0.67/d). Two-thirds of the patients (47/71 [66%]) did not have false alarms. In 3 patients, the number of false alarms was ≥10. The most common reason for false alarms was physiologic muscle activation: physical exercise (68% of false alarms). The second most common reason was device malfunction: induction artifacts (18%) and detaching electrodes (2.5%). The vast majority of false alarms occurred during daytime. Only 2 nocturnal false alarms occurred: one was triggered by an episode classified as paroxysmal arousal, and the other one was triggered by stretching the arms during an arousal.
The ratio between false alarms and true alarms was ≤1 for 17 of the 20 patients with GTCS (85%), and in 13 patients with GTCS (65%) the ratio was zero (i.e., no false alarms recorded). The range of the ratio was between 0 and 14 (median 0; mean 1.3). A total of 161 seizures other than GTCS occurred during the recording: 59 simple partial seizures (11 patients), 55 complex partial seizures (14 patients), 26 myoclonic jerks (5 patients), 6 absences (2 patients), 1 nonconvulsive status epilepticus, 14 convulsive psychogenic nonepileptic seizures (5 patients). None of these triggered seizure alarms.
Discussion
In this prospective, multicenter, blinded study on the accuracy of real-time detection of GTCS using an EMG-based wearable device (EDDI), we found that it had high sensitivity (93.8%) and short detection latency (9 seconds). Specificity of the device was reflected by the low number of false alarms (0.67/d). None of 161 seizures other than GTCS triggered an alarm.
As the study was blinded, the alarm function of the device was turned off in this study. Nevertheless, the short detection latency suggests that timely warning is possible using this device. Large field studies, with long-term, ambulatory use of the device, are necessary to evaluate its potential in reducing the number of seizure-related injuries and ultimately the number of SUDEP.
Both undetected seizures occurred shortly after a detected GTCS. A possible explanation is that the mechanism of muscle activation during the tonic phase of recurring GTCS is different (smaller amplitude, shorter duration). Multiple GTCS occurred in 7 other patients in whom all GTCS were detected. Recurring seizures (i.e., with short interseizure period) occurred in the group of detected seizures too. Two of the detected, recurring seizures had interseizure periods (73 and 74 minutes) within the range of the interseizure periods of the undetected seizures (19–98 minutes). Thus, in at least some patients the second GTCS might remain undetected, when the interseizure period is short.
Surveys of large numbers of patients and their caregivers addressed the users' requirements for wearable seizure detection devices.7,–,10 Patients did not want to use implanted and EEG-based devices. Patch electrodes at invisible body sites were considered acceptable.7,8,10 Detection of seizures involving heavy movement and falls was considered most important by patients and caregivers, while for physicians, SUDEP prevention was the most significant goal.8 This emphasizes the importance of detecting GTCS, the seizure type that is targeted by our device. The most important feature for users was sensitivity, with most users requiring sensitivity over 90%. Our device provides this. The required detection latency was under 10 seconds8 or 30 seconds.9 The median detection latency in our study was 9 seconds. In 2 cases, the muscle activation triggering the alarm occurred 3–4 seconds before any movement was observable in the video-EEG recordings, giving negative values for detection latency. Concerning the FAR, the results of the surveys varied broadly, but in all surveys users agreed that sensitivity is more important than FAR. In one of the surveys, the acceptable FAR was considered less than 1/wk or up to 1 false alarm/detected seizure.8 Our device fulfilled the latter requirement, but the overall FAR was more than 1/wk (4.7/wk). However, throughout the study, only 2 false alarms occurred during sleep, which is important for the quality of life of patients and caregivers.
The baseline frequency of GTCS prior to admission to the EMU was lower than what we recorded in the EMU. This means that in ambulatory use, the ratio between false alarms and true alarms would be higher than what we calculated in the EMU, and FAR may need to be improved for ambulatory use in medicated patients. Nevertheless, the baseline seizure frequency of GTCS was derived from the patients' seizure logs, and previous studies showed that many GTCS remain undetected by the patients.11
Device deficiency time was <5%. The device gives alarms when the electrode impedance is too high, so that the users can correct this. However, due to the blinding, this feature was disabled throughout the study. Thus, it is reasonable to assume that device deficiency time would be shorter when users are alerted about this.
Two different algorithms for unimodal EMG-based seizure detection have been developed by 2 groups. Both algorithms have been tested retrospectively, by offline analysis of data recorded with conventional amplifiers.17,18 Our algorithm is generic and targets the initial phase of GTCS.17 The retrospective offline analysis gave a sensitivity of 100%. A modified version of this algorithm was implemented in the wearable device we have tested. In this prospective study, based on real-time seizure detection, 2 of the 32 seizures were missed. This demonstrates the importance of validating seizure detection devices using this design, rather than by offline, retrospective analysis. The other algorithm, developed by Brain Sentinel (San Antonio, TX), targets a later phase: the tonic–clonic interface.18 The algorithm is individualized and it utilizes the Hotelling T2 power analysis. The retrospective, offline analysis showed a sensitivity of 95% (20 out of 21 GTCS from 11 patients were detected) and only 1 false alarm occurred. The average detection latency was within 20 seconds.18 Both algorithms have been registered for medical application: in Europe, EDDI obtained CE registration in 2013, and Brain Sentinel was granted de novo clearance by the Food and Drug Administration in 2017.
In spite of numerous attempts to develop noninvasive seizure detection,3,5 there are only 4 prospective studies on clinical validation of wearable devices using real-time seizure detection.22,25,–,27 All 4 studies used accelerometers. Two articles addressed the same device: SmartWatch.26,27 Although the pilot study showed promising results (7 out of 8 seizures were detected),26 in a large prospective study, only 31% of GTCS were detected.27 Another algorithm correctly identified 20 of 22 (91%) seizures from 15 patients, and had a detection latency of 17 seconds.25 Although real-time, the analysis was done on a computer placed within 10 meters from the patient. A wearable accelerometer device (Epi-Care Free; Tunstall, Astoria, NY) detected GTCS with a sensitivity of 89.7%.22 Although the FAR was low (0.2/d), this was achieved by instructing patients to avoid movements that could trigger alarms with the arm where the sensor was placed. The detection latency was significantly longer (55 seconds) compared to EDDI. Accelerometers trigger the alarm in the later, clonic phase of the GTCS, while EDDI targets the initial part of the tonic phase, thus providing much earlier detection (9 seconds) of the seizure and the possibility of much earlier intervention.
The main limitation of our study is the intrinsic challenge of any validation study of wearable seizure detection devices: the testing was done in an EMU, which differs from the patients' home environment. This was necessary in order to have the reference standard. However, mobility of the patients was not restricted during monitoring, and they were encouraged to perform indoors physical activities similar to their habitual lifestyle.21
Using the data recorded in this study, further optimization of the algorithm is planned, especially for decreasing the FAR. An alternative approach can be using individualized thresholds. It is noteworthy that the device was not triggered by any of the 14 convulsive psychogenic nonepileptic seizures that occurred during this study, considering that all 5 patients who had these were referred to the EMU on suspicion of GTCS. This raised the possibility of using the device for differential diagnostic purposes, as indicated previously.15
Author contributions
S.B., I.C., and P.W. contributed to the conception and design of the study. S.B., O.H., and M.F. acquired and analyzed data. I.C. monitored the logs from the detection device and provided technical support. S.B. drafted the manuscript and I.C. the figures. All authors contributed to editing the manuscript.
Study funding
This was an investigator-initiated study and no funding was granted to it. IctalCare, the manufacturer of the EDDI device, granted the investigators free-of-charge use of the device throughout the study period.
Disclosure
S. Beniczky reports no disclosures relevant to the manuscript. I. Conradsen was an employee of and is currently a consultant for IctalCare. She monitored the logs from the detection devices but did not have any influence on the logs with the time points generated by the real-time seizure detection of the device. She also gave technical assistance in form of training the personnel in the EMUs in using the device, but she was not present in the EMUs during the study and she was blinded to all clinical data, including the reference standard, prior to completion of the analyses. IctalCare, the manufacturer of the EDDI device, granted the investigators free-of-charge use of the device throughout the study period. O. Henning, M. Fabricius, and P. Wolf report no disclosures relevant to the manuscript. Go to Neurology.org/N for full disclosures.
Acknowledgment
The authors thank the patients and their caregivers who participated in this study and the personnel of the epilepsy monitoring units of involved centers.
Footnotes
Go to Neurology.org/N for full disclosures. Funding information and disclosures deemed relevant by the authors, if any, are provided at the end of the article.
The Article Processing Charge was funded by Brain Sentinel.
Editorial, page 207
- Received July 3, 2017.
- Accepted in final form October 9, 2017.
- © 2018 The Author(s). Published by Wolters Kluwer Health, Inc. on behalf of the American Academy of Neurology.
This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND), which permits downloading and sharing the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.
References
Letters: Rapid online correspondence
REQUIREMENTS
If you are uploading a letter concerning an article:
You must have updated your disclosures within six months: http://submit.neurology.org
Your co-authors must send a completed Publishing Agreement Form to Neurology Staff (not necessary for the lead/corresponding author as the form below will suffice) before you upload your comment.
If you are responding to a comment that was written about an article you originally authored:
You (and co-authors) do not need to fill out forms or check disclosures as author forms are still valid
and apply to letter.
Submission specifications:
- Submissions must be < 200 words with < 5 references. Reference 1 must be the article on which you are commenting.
- Submissions should not have more than 5 authors. (Exception: original author replies can include all original authors of the article)
- Submit only on articles published within 6 months of issue date.
- Do not be redundant. Read any comments already posted on the article prior to submission.
- Submitted comments are subject to editing and editor review prior to posting.
You May Also be Interested in
More Online
Use of Whole-Genome Sequencing for Mitochondrial Disease Diagnosis
Dr. Robert Pitceathly and Dr. William Macken