A population-based study of acute stroke and TIA diagnosis

Methods.

This project compared the diagnosis provided by the physician staffing the ED who saw the patient with the diagnosis given by a board-certified study neurologist who did not see the patient but reviewed the patient’s source documentation. The study neurologist’s diagnosis was based on rigorous stroke definition criteria.

The methods of the Brain Attack Surveillance in Corpus Christi (BASIC) Project were previously reported.5,6⇓ In brief, this project uses both active and passive surveillance to capture all acute strokes and TIA in subjects over age 44 in the biethnic community of Nueces County, TX. Nueces County is located on the Texas Gulf Coast and is at least 150 mi from both Houston and San Antonio, making complete capture for first medical contact excellent. Nueces County has a population of 313,645; 56% are Mexican American and 38% are non-Hispanic white.7 The city of Corpus Christi is urban and contains 95% of the county’s population. There are seven acute care hospitals, all with ED with 24-hour CT capability and neurologists on staff. There are no academic medical centers. There are 41 physicians who practice in ED in this community. Of these, 19 are board certified in emergency medicine, 14 in family practice, 5 in internal medicine, 2 in general surgery, and 1 is not board certified. Information regarding individual physicians’ residency training was not included in the database.

The BASIC database includes all patients suspected of having stroke or TIA who pass through the ED regardless of whether they are admitted or sent home. As the purpose of this report was to compare the ED physicians’ diagnosis with the BASIC neurologists’ diagnosis, we excluded all patients seen only as outpatients, directly admitted, or those with in-hospital strokes. Strokes and TIA are captured from ED screening and admission logs through active surveillance utilizing a set of previously validated stroke-screening terms.8 Passive ascertainment using International Classification of Diseases-9 (ICD-9) codes is used to ensure complete case capture. Cases are abstracted by trained abstractors, and source documentation is copied. Board-certified neurologists utilize source documents from the ED and hospital course, blinded to subject age and ethnicity, to determine if the case is a stroke/TIA or not. Criteria for this decision are based on published international consensus9: the acute onset (minutes to hours) of a focal neurologic deficit specifically attributable to a cerebrovascular distribution that persists for >24 hours (except in cases of sudden death or if the development of symptoms is interrupted by a surgical or interventional procedure) and not attributable to another disease process (seizure, brain tumor, hypoglycemia, metabolic encephalopathy, or hysteria). Demographic, risk factor, and clinical presenting information is abstracted from the chart. Ethnicity is based on information in the chart. We have observed 97% agreement (κ = 0.94, p < 0.001) between interview and medical record for ethnic classification.

The ED physician impression/diagnosis was abstracted from the chart. If the diagnosis contained any mention of stroke (e.g., “CVA, cerebral ischemia, rule out CVA”), it was counted as a stroke diagnosis. Further, if emergency physicians gave a diagnosis of a focal neurologic symptom (e.g., “right hemiparesis”), they were given credit for a stroke diagnosis. Diagnoses other than stroke were grouped into four categories: 1) focal neurologic disease (e.g., “Bell’s palsy”), 2) generalized neurologic disease (e.g., “dementia”), 3) medical disease (e.g., “pneumonia”), and 4) cardiac disease (e.g., “congestive heart failure”).

Frequencies and percentages were tabulated for all categorical variables. With the BASIC-validated diagnosis considered the gold standard, sensitivity and positive predictive value of the ED physician’s diagnosis of stroke/TIA were calculated. Analyses were first performed using the total study population and then separately by gender and race/ethnicity. Univariate and multivariable logistic regression analyses were performed to estimate odds ratios (OR) and 95% CI for factors that predicted disagreement between the BASIC neurologists and the ED physicians. Factors investigated included demographic variables (age, gender, race/ethnicity, insurance status), stroke symptoms (motor, sensory, visual, and language), number of stroke risk factors (including hypertension, diabetes, atrial fibrillation, coronary artery disease, high cholesterol, heavy alcohol consumption, and current smoker), severity of neurologic deficits measured by the NIH Stroke Scale abstracted from the chart,10 history of stroke/TIA, neurologist consultation in the ED, and hospital location (seven hospitals). Only abstractions on or after June 1, 2000, included information regarding whether the neurologist consultation was performed in the ED, and therefore analyses including this variable were limited to the time period June 1, 2000, through August 31, 2002.

Age was modeled as a dummy-coded variable for age 60 or older, with <60 years as the referent group. Race/ethnicity was modeled as two dummy-coded variables for Mexican American and African American, with non-Hispanic white as the referent group. Number of stroke risk factors was modeled as two dummy-coded variables for one to two risk factors and more than two risk factors, with zero risk factors as the referent group. Neurologic deficit severity was also modeled as two dummy-coded variables for moderate (9 to 15) and severe (>15), with mild (0 to 8) as the referent group. Hospital location was modeled as six dummy-coded variables, with location 1 as the referent group. The two scenarios for disagreement (including 1) the BASIC neurologist diagnosed stroke or TIA, but the ED physician did not diagnose stroke or TIA; and 2) the BASIC neurologist did not diagnose stroke or TIA, but the ED physician did diagnose stroke or TIA) were considered separately in two series of logistic models. Wald tests and likelihood ratio tests were used to assess the significance of the factors in univariate analyses. Factors with a p value of <0.20 in univariate analyses were included in the multivariable models.

A subanalysis was performed on the subset of cases with validated strokes or TIA (n = 1,800) to determine the association of stroke type (ischemic stroke, TIA, intracerebral hemorrhage [ICH]/subarachnoid hemorrhage [SAH]) with disagreement (BASIC neurologist diagnosed stroke, but the ED physician did not diagnose stroke or TIA). Stroke type was modeled as two dummy-coded variables for TIA and ICH/SAH, with ischemic stroke as the referent group. This analysis was performed separately to avoid exclusion of the 259 cases validated as not a stroke or TIA in the larger analysis.

This project was approved by the University of Michigan and University of Texas at Houston Institutional Review Boards, the Corpus Christi/Nueces County Health District, and all the hospitals.

Results.

A total of 13,015 cases had a positive stroke screening term and/or ICD-9 code for stroke and were abstractor screened between January 1, 2000, and August 31, 2002. Of these cases, 2,634 met criteria for the study5,6⇓ and were abstracted. Patients were excluded from the analysis if they were not seen in the ED (n = 339), had an in-hospital stroke (n = 91), or had left against medical advice before evaluation was complete (n = 10). The remaining subset of 2,194 potential stroke/TIA cases were seen in the ED and were eligible for the analysis regardless of ED disposition (admitted, transferred, deceased, or not admitted). Cases with a missing ED diagnosis (n = 93), with a missing ED or medical record (n = 20), or seen at an out-of-area ED (n = 10) were excluded. An additional 12 cases (7 Asian Pacific Islander and 5 unknown race/ethnicity) were excluded based on race/ethnicity because the sample sizes were too small to analyze, leaving 2,059 eligible cases. Of the 2,059 cases, 1,800 were validated as stroke/TIA by a BASIC neurologist. Eighty-two percent of the cases were admitted to the hospital, 16% were sent home, and 2% were sent to a nursing home (n = 31), died (n = 6), transferred to another hospital (n = 4), or went to assisted living (n = 4).

Emergency physicians were given credit for a stroke/TIA diagnosis in 61 cases with documentation of focal neurologic symptoms for the purposes of this analysis. Inclusion of these cases as stroke/TIA in the analysis resulted in a 3% increase in the sensitivity of the ED physician’s diagnosis of stroke, with this increase being approximately uniform across the gender (male 2.8%, female 2.9%) and race/ethnicity (non-Hispanic white 3.8%, Mexican American 3.0%, African American 3.0%) subgroups.

Table 1 provides the demographic and clinical characteristics of the study subjects. Table 2 compares the ED physician diagnosis with the BASIC-validated diagnosis. Overall sensitivity was 91.5% and positive predictive value was 89.1%. Sensitivity exceeded 89% for both male and female patients and all race/ethnic subgroups. The observed sensitivity implies a high rate of suspicion for stroke/TIA. Of the 1,848 patients the physician practicing in the ED diagnosed as stroke/TIA, 201 (10.9%) were validated as “no stroke” by the BASIC study neurologist.

Table 3 provides the univariate analyses for potential predictive factors of disagreement between the ED physicians and the BASIC-validated diagnoses. Clinical symptoms, number of stroke risk factors, history of stroke or TIA, and neurologic severity were associated with disagreement. Table 4 demonstrates the multivariable models of predictors of disagreement. Model 1 presents predictors of disagreement where BASIC validates stroke/TIA, but the ED physician considers the diagnosis something else. Motor symptoms protected from this type of misdiagnosis. Model 2 presents predictors of disagreement where BASIC validated the case as no stroke/TIA, but the emergency physician considered the diagnosis a stroke/TIA. This was less likely to occur when patients had sensory or motor symptoms and when their neurologic deficits were categorized as severe compared with mild, but more likely to occur when the patient had a history of stroke or TIA.

Among validated stroke/TIA cases, emergency physicians were four times more likely to concur with the BASIC-validated diagnosis if the stroke was a hemorrhage compared with ischemic stroke (OR = 0.24; 95% CI 0.10 to 0.60). TIA was not predictive of disagreement (OR = 1.02; 95% CI 0.71 to 1.46).

Table 5 shows the categorized diagnosis (focal neurologic, generalized neurologic, medical, or cardiac), given disagreement between the BASIC validation and the emergency physician. Generalized neurologic and medical conditions predominated in the disagreements as alternative diagnoses to stroke/TIA.

Discussion.

This study demonstrates that emergency physicians are good at detecting patients with stroke/TIA, with >90% sensitivity in this community. This suggests that few patients who are candidates for acute stroke interventions will be missed if they are seen only by emergency physicians. This is consistent with retrospective studies at academic medical centers reporting that ED physicians are highly sensitive in their diagnosis of stroke.1,2⇓

We were unable to calculate specificity because we did not capture all true negatives (cases where the ED physician and BASIC neurologist thought “no stroke”). However, the positive predictive value was 89.1%, suggesting that in 10.9% of the patients for whom the physician in the ED gave a final diagnosis of stroke/TIA, the BASIC neurologist did not concur. This may be due to the fact that we considered focal neurologic symptoms as stroke diagnoses and we accepted any mention of stroke, including “rule out stroke,” as a stroke diagnosis. ED physicians may have also charted their diagnosis before all radiologic and clinical data were available to hone their differential diagnosis. To minimize this bias, we included only diagnoses listed as “final diagnosis” in the ED chart. It is possible that the physician practicing in the ED may have recorded a “worst case scenario” final diagnosis rather than a diagnosis of certainty.

Disorders inappropriately diagnosed as stroke by ED physicians were primarily generalized neurologic and medical conditions. This finding is consistent with past research demonstrating that postictal states, systemic infections, tumors, and toxic/metabolic disturbances account for 60% of stroke mimics in the ED.11 The improper diagnosis of stroke has important implications for patient morbidity and mortality as a false-positive diagnosis of acute ischemic stroke puts the patient at risk of receiving inappropriate, and potentially hazardous, thrombolytic therapy. Time sorts out much of the clinical dilemma in the ED; however, the constraints of acute stroke therapy do not allow that luxury and force immediate and committed decisions. Physicians practicing in the ED are caught in this difficult situation. This study speaks to this issue and lends itself directly to professional interventions to remedy the situation.

Stroke center designation is an attempt to facilitate prompt triage and treatment of acute stroke patients.4,12⇓ The emergency medicine community is vocal in their request for neurology and institutional back-up in the care of stroke patients.13,14⇓ ED physicians are critical participants in the care of acute ischemic stroke. Current emergency medicine consensus statements for the treatment of acute stroke recommend hospital systems approaches that include a physician with expertise in stroke and availability for acute phone or bedside neurology consultation.13 The current study showed that neurologic consultation reduced disagreements by one-third to one-half, but this finding was not significant, perhaps owing to the infrequent neurology ED consultations and the potential for a type II error. Potentially, neurologic consultation and advanced imaging studies may improve accurate diagnosis and treatment decisions by emergency physicians. This requires further definitive study.

Physicians practicing in the ED come from a variety of medical backgrounds. Specialty board certification of the physicians practicing in ED in Nueces County closely resembles national estimates. Of the physicians practicing in ED in Nueces County, 46% are American Board of Emergency Medicine certified compared with 50% nationally.15 A growing number of physicians who practice in ED have received residency training in emergency medicine and received board certification in this discipline. Increasingly, emergency medicine training programs require stroke education. This will likely improve acute stroke care.

Multivariable analysis showed that the presence of motor or language deficits decreased the odds of a false-negative diagnosis. This finding, along with past studies that have shown a low percentage of neurology consultations for patients with nonmotor symptoms,16 suggests that nonneurologists fail either to recognize other stroke symptoms or to appreciate the significance of the symptoms when language or motor deficits are not present. The OR of a false-negative diagnosis were significantly lower for ICH and SAH subtypes. This finding is not unexpected as hemorrhage is easily diagnosed by a positive finding on a noncontrast head CT.

Several attributes of this study lend credibility to the results and support its generalizability. The BASIC Project is a population-based study designed to identify all suspect strokes in the community, regardless of the mode of health care delivery or the diagnosis assigned by the physician involved in the case. To ensure the capture of all suspect stroke/TIA cases, the study employed a combination of active and passive surveillance methods that has demonstrated high sensitivity in identifying potential cerebrovascular events.5 Because the study is population based and highly inclusive, it does not have the bias inherent in studies of clinic and referral populations. The attributes of the community studied by the BASIC Project also contribute to its generalizability. Most research on stroke has focused on populations in and around major metropolitan centers that are well served by academic medical centers. In contrast, the BASIC Project studies an urban population with health care provided by nonacademic community medical centers. Data from this “real world” setting are important when considering health policy decision making.

Our study has several limitations. In their review of source documents, validators had access to source documents that included the diagnosis given by the ED physician. This may have introduced bias that contributed to the high sensitivity calculated for ED physicians. The sensitivity may also be biased in the other direction if patients with unusual stroke syndromes were misdiagnosed by the physician practicing in the ED and discharged home without sequelae and without screening terms used by BASIC to detect suspected strokes. This is exceptionally unlikely owing to the inclusiveness of the screening terms. An additional limitation of the study is that it does not evaluate the diagnostic sensitivity of ED physicians relative to general neurologists practicing in the same setting. Further, we used the NIH Stroke Scale score as a standardized neurologic assessment of severity of neurologic injury. Although this scale was validated for retrospective chart abstraction,10 the validation contained stroke patients only and not nonstroke patients as we included. These study limitations detract somewhat from the conclusion that it may be appropriate for ED physicians to independently diagnose acute stroke and lead to a recommendation for further study to evaluate the independent diagnostic skills of ED physicians and general neurologists.

Despite the limitations of the study, the finding that ED physicians exhibit high sensitivity in the diagnosis of stroke lends support to the notion that ED physicians are good at screening patients with acute stroke. The proportion of false-positive diagnoses by physicians practicing in the ED argues for a systems approach with neurology support so that proper decisions regarding acute stroke therapy can be made.