Accuracy of four clinical diagnostic criteria for the diagnosis of neurodegenerative dementias

Diagnosis of dementia.

The diagnosis of dementia was based on a progressive deterioration of the cognitive function, in the absence of reversible causes of dementia, and history of normal intellectual function before the onset of cognitive abnormalities. Patients had impairment in at least two cognitive domains, which did not necessarily include memory.38

Statistical analysis.

Generalized κ statistics were used to examine the inter-rater reliability among the four clinicians. Strength of the agreement was designed as poor (κ < 0), slight (κ = 0 to 0.20), fair (κ = 0.21 to 0.40), moderate (κ = 0.41 to 0.60), substantial (κ = 0.61 to 0.80), and near-perfect to perfect (κ = 0.81 to 1.0). The pooled test determined the significance between κ values. Chi-square and one-way ANOVA with post hoc comparisons were used to analyze contingency tables.

Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were ascertained by comparing the clinicians’ diagnoses with the pathologic diagnoses. Sensitivity refers to the probability of having a clinical diagnosis of a disease given the neuropathologic presence of the disease. PPV refers to the probability of having the disease given the clinical diagnosis of having the disease. Specificity refers to the probability of not having a clinical diagnosis of a disease given the neuropathologic absence of the disease. NPV refers to the probability of not having the disease given the clinical diagnosis of not having it.39 These indices were calculated twice. First, we classified a patient if there was any mention of the condition in the diagnosis (i.e., probable, possible). Second, we applied more stringent criteria and considered a patient to have a condition only if the clinician used a “probable” diagnosis.

Alzheimer’s disease.

The presence of another potential cause of dementia in a patient who otherwise meets all NINCDS-ADRDA clinical criteria for probable AD produced the greatest diagnostic disagreement. In particular, the presence of CVD produced mixed diagnostic results. In some cases, clinicians felt that AD and CVD contributed independently to the dementia syndrome and classified patients as having possible AD with CVD; in other cases the cerebrovascular factor was minimized, and the patients were classified as having probable AD (Patients 5, 8, 9, and 20). Similarly, they classified a patient with laboratory findings indicating thyroid disease as having probable AD (Patient 7). This use of multiple diagnoses represents a lack of strict adherence to the NINCDS-ADRDA clinical criteria and has been reported in other reliability studies.20 Furthermore, because we did not include mildly demented AD patients, or nondemented individuals with mild cognitive impairment, the NINCDS-ADRDA criteria were not tested in cases where there is a possibility of AD.

There was a high sensitivity for AD (95%) but with a relatively low specificity (79%) and PPV (61%), reflecting a tendency to overdiagnose. Indeed, the specificity (82%) and PPV (61%) remained low, even when we used more stringent criteria for probable AD. The clinicians classified as having AD patients with other cortical (e.g., FTD) (Patients 30, 31, and 34) and subcortical dementias who manifested a cortical neuropsychological dysfunction (Patient 38). Patients with dementia syndromes may appear more similar as they progress, which makes it difficult to differentiate them with only the MMSE or MDRS. To the extent that it is possible to differentiate between dementia syndromes with predominant cortical versus subcortical pathology,42 additional, more sensitive neuropsychological measures might have improved specificity.

Progressive supranuclear palsy.

There was a high level of inter-rater agreement for the NINDS-SPSP criteria for PSP, similar to other tested criteria.29 The most frequent source of disagreement was between probable and possible PSP. In contrast to the way that these modifiers were used in AD diagnosis, probable and possible PSP indicated the degree of certainty of the diagnosis. However, clinicians inappropriately used the term “possible PSP” to indicate that a second disease process was contributing to the patient’s cognitive and neurologic symptomatology, such as CVD (Patient 11) or vitamin B₁₂ deficiency (Patient 13), just as they would for AD. The NINDS-SPSP criteria do not mention how these criteria should be applied in cases with concomitant systemic or CNS conditions that can affect the clinical presentation of PSP, but do specify that the presence of CVD is an exclusionary criterion. The clinicians found a way to acknowledge that the presence of CVD and vitamin B₁₂ deficiency may have affected the cognitive presentation by using the term “possible PSP,” following the current interpretation of the NINCDS-ADRDA criteria for possible AD (when AD coexists with other disease processes). However, in the future it may be better to indicate the association of PSP with other conditions that may affect its presentation and course, as it is commonly seen in clinical practice, by using the term “associated with.”

Despite this, the NINDS-SPSP criteria have an optimal specificity (98.5%) and PPV (96%), even for the category probable PSP alone (specificity and PPV for probable PSP were 100%), with a relatively good sensitivity (50 to 75%). Underdiagnosis is the main concern when using these criteria, and this may be inevitable in some cases. For example, the presence of a dementia syndrome, mild parkinsonism, and normal extraocular movements led clinicians to diagnose these patients as having possible PSP or PD with dementia (Patient 14). Furthermore, the NINDS-SPSP criteria were more accurate than previous ones. Litvan et al.29 examined the accuracy of four clinical criteria for PSP,9-11 and their sensitivity ranged from 13 to 55%.

Dementia with Lewy bodies.

The reliability for the diagnosis of DLB was poor, consistent with Litvan et al.,39 who found only a fair (κ = 0.38) level of reliability for DLB. By contrast, McKeith et al.43 reported a moderate to near-perfect agreement (κ = 0.50 to 0.87) for DLB. It has been noted that sample composition could have accounted, in part, for the differences between these studies, in that McKeith et al. examined patients with AD, DLB, and vascular dementia, whereas Litvan et al. examined patients with multiple cortical and subcortical pathologies (e.g., Pick’s disease, DLB, PSP, MSA, CBD, postencephalic parkinsonism, Creutzfeldt-Jakob disease, Whipple’s disease), but included several AD cases, all with extrapyramidal signs. Furthermore, although Litvan et al. did not use standardized clinical criteria for DLB, our clinicians used the Consensus Guidelines for the Clinical Diagnosis of DLB. However, we did not include patients with “pure” DLB, which may have allowed us only to test reliability of the most difficult cases where DLB is associated with AD pathology.

The low mean sensitivity (34%) and PPV (55%) for the diagnosis of DLB with a high mean specificity (94%) indicated that our clinicians were better at identifying non-DLB cases than DLB cases. The sensitivity for DLB was low compared with previous studies. Indeed, Mega et al.44 found a sensitivity of 79% and a specificity of 100% for DLB, and McKeith et al.43 found a sensitivity of 90% and a specificity of 97%.

The diagnostic boundary between AD and DLB is difficult to identify. In some cases, clinicians used the term “probable AD” and minimized the presence of DLB symptomatology, whereas in other cases, they used the term “possible DLB” to indicate that both AD and DLB were present (Patients 20, 21, 23, and 24). Interestingly, none of the raters used the term “probable DLB.” This could be attributed to the tendency of the clinicians to overdiagnose AD or to the difficulty determining when the core signs and symptoms of DLB are different from those observed in AD.

Although DLB is described as including alterations in the sensorium, this can range from normal function to frank fluctuations in alertness and to delirium. Differentiating episodes of mild fluctuations of consciousness from diurnal hypersomnia (frequently seen in cortical and subcortical dementias) is difficult. However, when overt alteration of alertness occurred with episodes of delirium, all clinicians to agreed with the presence of DLB (Patient 22). Although hallucinations are more frequent in DLB than in AD,43,44 and are a reliable predictor for DLB,39 they can occur in other cortical and subcortical dementias.45 Indeed, the PSP and FTD patients used here did not experience hallucinations, and no statistical differences were observed between the proportion of patients with DLB with hallucinations (37.5%), AD (20%), and other disease processes (20%) (p = 0.18).

Extrapyramidal signs may be helpful in differentiating DLB from AD or FTD in early stages, but this is more difficult in advanced cases (Patients 21 and 26). In addition, they cannot discriminate DLB from other dementias associated with parkinsonism. Our findings suggested that DLB is still frequently misdiagnosed and that the Consensus Guidelines for the Clinical Diagnosis of DLB are suboptimal.39,46 Alternatively, one could argue that the difficulty diagnosing DLB is not because of weaknesses in the criteria, but because LB are part of the neuropathologic spectrum of AD and sometimes can exist without clinical manifestations.

Frontotemporal lobe dementia.

The inter-rater agreement for FTD was substantial. The most frequent source of disagreement was between FTD and patients with a frontal lobe syndrome in the context of AD. In addition, the presence of frontal lobe symptoms associated with parkinsonism was another source of disagreement among raters. The Lund and Manchester criteria for FTD do not provide a degree of certainty (e.g., probable, possible), but they provide core diagnostic features that are commonly observed in FTD.25 Because it is difficult to differentiate clinically frontal-temporal lobe degeneration from Pick’s disease,47 these criteria do not attempt to classify these entities independently. Nevertheless, their neuropathologic classification includes frontal-temporal lobe degeneration, Pick’s disease, and frontal lobe degeneration associated with motor neuron disease. In our study, none of the clinicians attempted to classify patients as having Pick’s disease. These results are different from those of Litvan et al.,40 who found a moderate (κ = 0.54) level of agreement in identifying a frontal lobe type dementia in patients with pathologically diagnosed Pick’s disease. However, they did not provide their raters with a set of diagnostic criteria.

The high sensitivity and specificity for the diagnosis of FTD found in this study indicated that the use of the Lund and Manchester criteria can improve diagnostic accuracy. By contrast, Mendez et al.48 found that only 25% of patients with Pick’s disease were correctly diagnosed antemortem, and Litvan et al.40 found a mean sensitivity of 43% for Pick’s disease. However, despite the high degree of accuracy observed in this study, in some cases clinicians diagnosed FTD as AD, and in one case as Creutzfeldt-Jakob disease or normal pressure hydrocephalus (Patient 33). This latter patient had a rapidly progressive motor syndrome (e.g., frequent falls, severe gait problems, dysarthria, urinary incontinence) and a gradual deterioration of his cognitive abilities, with apathy, depressed mood, and social withdrawal.

The presence of early behavioral abnormalities is the hallmark of FTD,38,49 which usually helps clinicians to differentiate FTD from other cortical dementias. However, the behavioral abnormalities of FTD can be heterogeneous and sometimes difficult to differentiate from AD. In this study, clinicians were able to classify as having FTD not only those patients who initially presented with significant disruptive social behavior (e.g., disinhibition, social inappropriateness, stereotype/perseverative behavior) but also those who presented marked apathy, mood lability, or depression.38,49 The availability of neuropsychological measures that indicated a frontal-temporal lobe compromise (e.g., verbal fluency and attention/abstract thinking deficits with a relative preservation of visuospatial and visuoconstructional abilities) in conjunction with the psychiatric information was an important factor that influenced the high accuracy.

We found that the use of multiple diagnostic criteria can increase the level of inter-rater agreement as well as diagnostic accuracy. Nevertheless, these criteria have their limitations, and they sometimes failed to capture cases in a “gray zone.” Although diagnostic criteria for AD have been available for research and clinical practice for the past 15 years,19 clinicians still tend to overdiagnose this disease. By contrast, DLB was underdiagnosed. Consequently, it is necessary to better operationalize both diagnostic criteria. The Lund and Manchester criteria for FTD and the NINDS-SPSP criteria for PSP helped to diagnose these relatively rare disorders.