Do published criteria improve clinical diagnostic accuracy in multiple system atrophy?

Subjects and methods.

Among 806 patients with parkinsonism whose brains were donated to the Queen Square Brain Bank for Neurological Disorders between 1984 and 2000, we identified 59 consecutive cases with a clinical diagnosis of MSA when last assessed prior to death. Clinicopathologic details of up to 34 of these 59 cases have been reported previously.9-12⇓⇓⇓ Clinical data abstracted from the patients’ medical records included autonomic, cerebellar, pyramidal, and parkinsonian symptoms and signs. These enabled individual patients to be classified by both the modified Quinn criteria13 and Consensus criteria.14 The Consensus criteria use the terms “criteria” and “features” in a specific and precise way. To avoid confusion, we will use the terms “criteria” and “features” in quotes when referring to their usage according to the Consensus statement. Response to levodopa was graded on a four-point scale, and initial as well as last recorded response was noted. The occurrence of symptoms and signs was determined at both the first and last neurologic visit.

In this article, we refer to pathologically confirmed cases as true MSA and to clinically misdiagnosed cases as false positive cases. At first visit, 11 of the 51 true MSA cases and one of the eight false positive MSA cases were diagnosed as MSA, all by consultant neurologists. None of the other 47 patients later clinically diagnosed with MSA were initially diagnosed as such by either neurologists (n = 23) or non-neurologists (n = 24). The two other common diagnoses at the first clinic visit were PD (n = 12) and parkinsonism (n = 12). Among the eight false positive cases, seven had been seen by consultant neurologists at first clinic visit; their diagnoses at this time were PD (n = 4), MSA (n = 1), postprandial hypotension (n = 1), and unknown (n = 1). The eighth false positive case had been initially diagnosed as parkinsonism by a non-neurologist. By the time of the last clinic visit, all cases had been seen by consultant neurologists.

Clinical and pathologic data from patient subgroups were compared using Student t-test. The proportion of cases with each clinical feature was compared between true MSA and false positive cases using the χ² test for proportions for a two-by-two contingency table. To assess the validity of each clinical feature as well as the diagnostic criteria, we chose sensitivity and positive predictive value (PPV) as our outcome measures. We report the results for both categories of diagnosis (probable or possible) for both criteria. We do not report the specificity as it is statistically imprecise owing to the small number of false positive cases.

Results.

The 59 MSA cases comprised 37 men and 22 women. The mean age at onset was 54.3 years (range 34 to 74), the mean survival from first symptom was 7.8 years (range 2 to 17), and mean age at death was 62.1 years (range 39 to 77) for the whole group. Fifty-one cases were confirmed pathologically, resulting in a PPV of 86%. The remaining 8 cases (14%) had a false positive clinical diagnosis of MSA. Lewy body PD with variable degrees of cortical Lewy body involvement was established in six of these eight cases; one case was found to have PSP and one had cerebrovascular disease.

False positive cases tended to be older at onset (58.3 ± 9.1 vs 53.6 ± 9.7 years) and had significantly longer survival (10.4 ± 4.0 vs 7.5 ± 3.3 years) as well as older age at death (68.6 ± 6.7 vs 61.1 ± 8.8 years) than true MSA cases. Table 1 shows clinical details of the eight false positive cases, according to the two sets of criteria. Four cases were Quinn probable and Consensus possible, and three were probable and one possible on both sets of criteria, suggesting lower potential for false positive diagnosis with the Consensus criteria.

Table 2 compares the sensitivity and PPV of the Quinn and Consensus criteria at both first and last visit. The mean duration between these visits was 66 ± 46 months; i.e., 5.5 years. The sensitivity and PPV of the clinical diagnosis made in the patients are also presented for comparison. Details of the levodopa response at first visit were lacking in 16 cases. To avoid a loss in statistical power, the patients’ first ever levodopa response record was used; this was recorded on average 15 (range 3 to 48) months after the first visit. Sensitivity with the Quinn criteria was higher than that for Consensus criteria at first visit and vice versa for PPV. Quinn possible and probable categories and Consensus possible category had higher sensitivities than the initial clinical diagnosis.

At the last clinic visit, sensitivities were over 90% for the Quinn possible and probable and Consensus possible categories, but only 63% for the Consensus probable category. There was little difference between the PPV, and the Quinn probable and Consensus probable categories had only slightly higher PPV than the last clinical diagnosis.

The upper half of table 3 shows the sensitivity and PPV of key clinical manifestations at first clinic visit. The highest sensitivity (78%) was observed for both the Consensus “criterion” and “feature” for parkinsonism. The lower half of table 3 shows sensitivity and PPV of the same key clinical manifestations at last clinic visit. The sensitivity, as might be expected, had increased for all these clinical features as the disease had progressed. By this time, autonomic and cerebellar manifestations presented equal or higher, and parkinsonian manifestations fractionally lower, PPV than the last clinical diagnosis.

Discussion.

This is the first study that has empirically attempted to validate two existing sets of diagnostic criteria for MSA in a large neuropathologically examined series of cases. The mean age at onset of our true MSA cases was 53.6 years and the mean survival from first symptom was 7.5 years. These results are consistent with previous reports and indicate that our case series is comparable to other studies.5,9,11,12,15⇓⇓⇓⇓

A probable diagnosis on either set of criteria was associated with lower sensitivity and higher PPV because a higher diagnostic threshold is required for a probable diagnosis. Thus some true MSA cases will not be classified as probable cases as they fail to exhibit all the required clinical signs. By the last visit, there was little difference in either sensitivity or PPV between the Quinn possible and probable categories. However, Consensus probable category had considerably lower sensitivities than the possible category. The possible category requires a greater number of dysfunction items as options, whereas the probable category requires fewer, but more obligatory, items. This same pattern was generally observed for Consensus “criteria,” which are stricter than mere “features,” although surprisingly this was not the case for parkinsonism, where “criterion” and “feature” performed almost identically.

At first visit, sensitivity with Quinn possible (63%) and probable (37%) and Consensus possible (28%) categories was higher than the initial clinical diagnosis (22%). However, by the last visit there was little difference in either sensitivity or PPV (see table 2). These data show that, using Quinn probable or Consensus possible criteria, one can make a definitive clinical diagnosis in a quarter to a third of MSA cases, even very early in the natural history of the disease.

Parkinsonism was the most frequent feature of true MSA, followed by autonomic and cerebellar and pyramidal signs (see table 3). This may be biased as all cases were ascertained from a PD brain bank, and therefore parkinsonian cases are likely to be overrepresented.

Overall, 14% of the cases were clinically misdiagnosed, most commonly owing to confusion with PD or PD with cortical involvement. All false positive cases fulfilled either possible or probable criteria for MSA. All had parkinsonism and all but one were reported to have a poor levodopa response. The second most common feature among them was autonomic symptomatology. Five of the six cases that were pathologically diagnosed as PD or PD with cortical involvement showed both “autonomic and urinary dysfunction” and “levodopa poorly responsive parkinsonism,” a combination that presumably had led to their misdiagnosis as MSA. Wenning et al.7 previously reported that 23% of Queen Square Brain Bank cases with PD showed a poor initial levodopa response compared with 58% poor initial responders within the MSA group, and that autonomic failure had developed in 84% of the MSA cases but in only 26% of the PD cases. Conversely, when looking at patients diagnosed with PD from 1996 to 1998 in the same brain bank, Hughes et al.16 found that the condition most often mistaken for PD was MSA. Distinguishing these problem cases purely on the basis of clinical examination may be difficult and laboratory-based diagnostic procedures such as sphincter EMG,17 further autonomic function tests like the clonidine-growth hormone stimulation test,18 or neuroimaging19-21⇓⇓ may be helpful in this subgroup of patients, although this awaits empirical confirmation. In addition, some of the clinically misdiagnosed cases in the current series were reported to have cerebellar features such as lower limb ataxia, although this is difficult to identify with certainty in severely disabled patients with postural instability of whatever cause. Thus, motor features of parkinsonism may partly overlap with the incoordination of cerebellar ataxia, leading one to an erroneous diagnosis of MSA rather than PD. One clinically diagnosed MSA case with brainstem and cortical Lewy bodies (Case 7) satisfied National Institute of Neurological Disorders and Stroke–Society for Progressive Supranuclear Palsy, Inc. criteria for possible PSP22 as well.

Establishing accepted diagnostic criteria is necessary to enable researchers to standardize case definition and enable greater comparability between studies. Both sets of existing criteria were developed on the basis of expert opinion rather than a robust empirical evidence-based or clinicopathologic correlation. Our data indicate that further development and refinement are needed.

There is no one set of ideal criteria as these will vary depending on the clinical or research objectives and the nature of therapeutic intervention. A clinical trial that aims to recruit true MSA cases for a proposed intervention that is costly and relatively toxic will need to sacrifice sensitivity for a high PPV. In this case the use of the Consensus probable category seems the optimum choice. Conversely, a pragmatic trial of a cheap, safe, and potentially effective therapy that requires initiation early in the natural history would favor high sensitivity at the expense of PPV. In this scenario, the Quinn possible category performs best, because even at first visit it will lose relatively few true MSA cases, but the number of cases needed to achieve adequate power may need to be increased to correct for misdiagnosis.

Published diagnostic criteria are often not used for making a diagnosis in routine practice. This is probably because in a classic case, clinicians have a high a priori probability (or conviction) that they are correct, and hence there is apparently little added value in applying the criteria. This is true in our series, and also in a subset of the cases seen by movement disorder specialists,12 in both of which the overall PPV was high at 86%. However, early in the natural history, or with difficult gray cases, the use of evidence-based criteria might improve clinical performance if clinicians are trained in the use of criteria that are sufficiently user-friendly to be adopted readily within the context of normal clinical care. For example, the Consensus criteria are more complex and time consuming to apply than the Quinn criteria, although this may be facilitated through a computer-administered algorithm. One should not overinterpret our data for the purpose of clinical diagnosis. This is because our sample consisted of clinically diagnosed cases of MSA rather than gray cases. As such, by the last visit we observed a ceiling effect so that both actual clinical diagnosis and diagnostic criteria all had very high PPV. We are currently extending this work to a sample of gray cases, which present a greater diagnostic challenge, to test whether existing criteria can outperform clinical opinion.

In conclusion, this study demonstrates high clinical diagnostic accuracy for MSA in a series of postmortem confirmed cases. The clinical misdiagnosis of MSA was predominantly due to confusion with PD, and may be ascribed to both “autonomic and urinary dysfunction” and “levodopa poorly responsive parkinsonism” occurring in PD or PD with dementia cases in this study. Although at first visit, the application of current diagnostic criteria for MSA may improve accuracy of diagnosis, at last visit they are no more accurate than neurologists’ final diagnosis.

Appendix 1

⇓

Appendix 2

Consensus criteria (adopted from Gilman et al.14) Table 1 Clinical domains, features, and criteria used in the diagnosis of multiple system atrophy (MSA) (a feature [A] is a characteristic of the disease and a criterion [B] is a defining feature or composite of features required for diagnosis)

• I. Autonomic and urinary dysfunction

• A. Autonomic and urinary features

• 1. Orthostatic hypotension (by 20 mm Hg systolic and 10 mm Hg diastolic)

• 2. Urinary incontinence or incomplete bladder emptying

• B. Criterion for autonomic failure or urinary dysfunction in MSA

• Orthostatic fall in blood pressure (by 30 mm Hg systolic or 15 mm Hg diastolic) or urinary incontinence (persistent, involuntary partial or total bladder emptying, accompanied by erectile dysfunction in men) or both

• II. Parkinsonism

• A. Parkinsonian features

• 1. Bradykinesia (slowness of voluntary movement with progressive reduction in speed and amplitude during repetitive actions)

• 2. Rigidity

• 3. Postural instability (not caused by primary visual, vestibular, cerebellar, or proprioceptive dysfunction)

• 4. Tremor (postural, resting or both)

• B. Criterion for parkinsonism in MSA

• Bradykinesia plus at least one of items 2 to 4

• III. Cerebellar dysfunction

• A. Cerebellar features

• 1. Gait ataxia (wide-based stance with steps of irregular length and direction)

• 2. Ataxic dysarthria

• 3. Limb ataxia

• 4. Sustained gaze-evoked nystagmus

• B. Criterion for cerebellar dysfunction in MSA

• Gait ataxia plus at least one of items 2 to 4

• IV. Corticospinal tract dysfunction

• A. Corticospinal tract features

• 1. Extensor plantar responses with hyperreflexia

• B. Corticospinal tract dysfunction in MSA: no corticospinal tract features are used in defining the diagnosis of MSA

Table 2 Diagnostic category of MSA (the features and criteria for each clinical domain are shown in table 1)

• I. Possible MSA: one criterion plus two features from separate other domains. When the criterion is parkinsonism, a poor levodopa response qualifies as one feature (hence only one additional feature is required).

• II. Probable MSA: criterion for autonomic failure/urinary dysfunction plus poorly levodopa responsive parkinsonism or cerebellar dysfunction.

• III. Definite MSA: pathologically confirmed by the presence of high-density glial cytoplasmic inclusions in association with a combination of degenerative changes in the nigrostriatal and olivopontocerebellar pathways.

Table 3 Exclusion criteria for the diagnosis of MSA

• I. History

• Symptomatic onset under 30 years of age

• Family history of a similar disorder

• Systemic disease or other identifiable causes for features listed in table 1

• Hallucinations unrelated to medication

• II. Physical examination

• DSM criteria for dementia

• Prominent slowing of vertical saccades or vertical supranuclear gaze palsy

• Evidence of focal cortical dysfunction such as aphasia, alien limb syndrome, and parietal dysfunction

• III. Laboratory investigation

• Metabolic, molecular, genetic, and imaging evidence of an alternative cause of features listed in table 1

*In practice, MSA is most frequently confused with PD or progressive supranuclear palsy (PSP). Mild limitation of upward gaze alone is nonspecific, whereas a prominent (50%) limitation of upward gaze or any limitation of downward gaze suggests PSP. Before the onset of vertical gaze limitation, clinically obvious slowing of voluntary saccades is usually easily detectable in PSP and assists in the early differentiation of these two disorders.