Abstract
Background: Brief measures that accurately discriminate normal cognitive aging from very mild dementia are lacking. Cognitive tests often are insensitive to very mild dementia. Informant-based measures may be more sensitive in detecting early dementia.
Objective: To identify informant-reported clinical variables that differentiate cognitively normal individuals from those with very mild dementia.
Methods: A 55-item battery of informant queries regarding an individual's cognitive status was derived from a semistructured interview and a consensus panel of dementia experts. The battery was evaluated with informants for 189 consecutive participants of a longitudinal study of memory and aging and compared with an independently obtained Clinical Dementia Rating (CDR) score for the participant. Multiple regression and receiver operator characteristic curves assessed subsets of the items to discriminate between CDR 0 (no dementia) and CDR 0.5 (very mild dementia).
Results: The final version (AD8) querying memory, orientation, judgment, and function was administered to an additional sample of 112 CDR 0 and 68 CDR 0.5 participants. Using a cut-off of two items endorsed, the area under the curve was 0.834, suggesting good to excellent discrimination, sensitivity was 74%, and specificity was 86% (prevalence of 0.38 for very mild dementia). Inclusion of 56 additional individuals with mild to severe dementia (increasing dementia prevalence to 0.53) increased sensitivity to 85%.
Conclusions: The AD8 is a brief, sensitive measure that reliably differentiates between nondemented and demented individuals. Use of the AD8 in conjunction with a brief assessment of the participant could improve diagnostic accuracy in general practice.
Alzheimer disease (AD) and other dementias are both underrecognized and underdiagnosed in the community.1–3 This may be due in part to the lack of brief measures that can discriminate normal aging from very mild dementia. Tests such as the Mini-Mental State Examination4 (MMSE) have a ceiling effect that makes them insensitive to early signs of dementia,5 especially in highly educated individuals, and they may not be culturally sensitive.6,7 Other scales such as the Cognitive Abilities Screening Instrument 8 and the General Practitioner Assessment of Cognition9 have less cultural bias but require extensive training to administer and generally are too lengthy for use in general practice. Brief measures such as the Short Blessed Test10 (SBT) and the Memory Impairment Screen11 are heavily weighted toward memory deficits and may not detect nonamnestic dementias. The Clock-Drawing Task or Cube Copying are similarly limited to a single cognitive domain and may not be useful in detecting mild cases of dementia.12–14 The Rowland Universal Dementia Assessment Scale has been applied in research samples and tests six domains but takes about 10 minutes to complete.15
Published criteria for AD diagnosis such as those developed by the Work Group of the National Institute of Neurological and Communication Disorders and Stroke/Alzheimer's Disease and Related Disorders Association (NINCDS/ADRDA)16 require standard assessment of patients. Comparison of individual performance on cognitive test measures with normative values, however, may not detect declines that occur in very mild dementia, particularly in high-functioning individuals.7,17 Further, brief objective testing may falsely identify as demented people with life-long poor cognitive functioning. Informant-based assessments, on the other hand, may reveal early cognitive change because of a longitudinal perspective, have face validity, and are established in studies characterizing AD and in AD clinical trials.18–21 Global rating systems such as the Clinical Dementia Rating18 (CDR) and the Informant Questionnaire on Cognitive Decline in the Elderly19 (IQCODE) incorporate information from a collateral source to assess change in the patient's cognitive ability to conduct their accustomed activities. These systems do not require a baseline assessment for comparison and minimize issues such as practice effects and educational and sociocultural influences that confound interpretation of performance-based assessments.20,21 Current informant-based assessments, however, are lengthy and require interpretation by an experienced clinician; therefore, they are difficult to use in community practice. Given the brief period available to primary care physicians in a standard office visit, there will likely be some acceptable trade-off to the clinician, sacrificing specificity and sensitivity to detect dementia to keep the clinical tool brief. Given that requirement, our goal was to develop a dementia screening instrument that could be completed in a just in a few minutes with maximally balanced sensitivity and specificity.
We developed a brief dementia assessment that includes both performance-based assessment of the patient and a brief informant interview. We identified clinical variables that we believed would distinguish individuals with very mild dementia from those without dementia based on a review of the literature and our experience with the semistructured interview of the informant in the research protocol used to derive the CDR18 (we also use information from the participant to derive the CDR). We identified 55 questions asked of the informant about the cognitive function of the participant that were thought to assist in dementia detection and when combined with a brief objective assessment of the participant might serve as a diagnostic screening tool. The results were compared with an independently derived CDR rating of the participant. Data from these participants were then used to develop an eight-item informant interview (the AD8) to discriminate between CDR 0 (nondemented) and CDR 0.5 (very mildly impaired conditions). The psychometric and discriminative properties of the AD8 were then tested prospectively in a second sample. Evaluation of the AD8 in combination with the brief participant assessment is ongoing; here we describe the development and validation of the AD8 and its correlation with other dementia screening measures.
Methods.
Clinical data.
All participants were volunteers who enrolled in a longitudinal study (initiated in 1979 and studying to date 2,049 individuals) of healthy aging and dementia. Participants are recruited via word of mouth, public service announcements, and referrals from physicians in the St. Louis area. Both nondemented and demented participants underwent identical annual assessments. There are currently 549 active participants. The Washington University Human Studies Committee approved all procedures.
Experienced clinicians (neurologists, psychiatrists, geriatricians, and masters-prepared geriatric nurse specialists) conducted independent semistructured interviews with the participant and a knowledgeable collateral source (usually the spouse or close family member)22 to capture features suggestive of a dementing disorder. The diagnostic criteria for dementia of the Alzheimer type used in this study (impairment in memory and at least one other cognitive domain and interference with daily activities) are comparable with the Diagnostic and Statistical Manual of Mental Disorders IV definition23 and of the “probable AD” category in the NINCDS/ADRDA criteria.16 Wherever possible, published criteria were used for other dementing disorders, including dementia with Lewy bodies (DLB)24 and vascular dementia (VaD).25
The CDR was used to determine the presence or absence of dementia and to stage its severity.18 Using all information from the clinical assessment protocol but without reference to the individual's psychometric performance, the CDR rates cognitive function in each of six categories (memory, orientation, judgment and problem solving, and performance in community affairs, home and hobbies, and personal care). The global CDR is derived from the individual ratings in each of the six categories, where CDR 0 indicates no dementia. CDR 0.5 may represent very mild dementia or in some cases with minimal impairment or uncertain or questionable dementia. CDR 1 corresponds to mild, CDR 2 to moderate, and CDR 3 to severe dementia.18 The sum of the individual category ratings (sum of boxes [SB]) provides a quantitative expansion of the CDR and ranges from 0 (no impairment) to 18 (maximum impairment).26 In our sample, the CDR 0.5 rating equates with very mild dementia27 and is the threshold to distinguish nondemented (CDR 0) from demented (CDR ≥0.5) status. In other samples CDR 0.5 has been used as the threshold for the diagnosis of mild cognitive impairment.28 In both cases, the CDR is useful to detect the change in cognitive abilities from a prior level of function and also to assess interference with accustomed activities. The interrater reliability of the CDR has been established,29 and its validity has been established by correlation with neuropathologic features observed at autopsy.26 Therefore, the CDR was used as the gold standard for recognition of cognitive impairment in this study.
Instrument development.
The AD8 was developed using a combination of expert clinical judgment and statistical modeling. A list of 55 questionnaire items was developed based on an extensive review of the literature, our experience with semistructured informant interviews, the consensus opinion of four dementia experts (a neurologist, a psychologist, and two advanced practice nurses), and the results of a previously collected telephone survey of community-dwelling older adults (unpublished data). Three response options were offered with instructions: 1) yes, a change; 2, no, no change; or 3), N/A, don't know. Between June 3, 2002, and February 21, 2003, the 55-item questionnaire was administered to 290 consecutive collateral sources (CSs) of study participants at their annual assessment. None of the 55 items was contained within the structured part of the interview. An open-ended part of the interview allows clinicians to ask nonstandard questions that probe changes in memory, orientation, and other cognitive domains specific to that participant. To maximize its ability to discriminate between normal cognitive aging and very mild dementia, the AD8 was developed using data from the CS responses of participants who were age 55 or older and who received a CDR score of 0 (n = 86) or 0.5 (n = 103) at their assessment. There were no significant differences between the CDR groups with regard to age, gender, race, education, or the relationship between the CS and the participant.
Because the AD8 was conceptualized as a brief instrument, the RSQUARE selection method (SAS version 8 for Sun OS, Cary, NC) was used to reduce the number of items. This method finds subsets of independent variables that best predict a dependent variable using multiple linear regression. After specifying the subset size, the procedure computes all possible regressions and their R2 value so the models' predictive ability can be compared. The procedure was carried out within the CDR 0.5 sample (n = 103) using the CDR SB as the dependent variable. All 55 items were used as predictor variables, and subset size was set to 10. Thus, regressions were calculated for all possible 10-item combinations of the 55 predictor variables.
Ten models with the largest R2 values were examined. Nine models contained negative regression coefficients indicating the statistical phenomenon of suppression. On a practical level, a negative coefficient means that the variable would have to be subtracted from the total, whereas the other variables would be added together. Because the R2 values of the 10 models were almost identical (range = 0.7407 to 0.7458), the model without negative coefficients was chosen (R2 = 0.744).
Further analysis of this 10-item model led to several revisions. Two items from this model were rarely endorsed: wears seasonally inappropriate clothing (3.9%) and trouble dressing (1.9%); therefore, an index score was also computed using only eight items. Data from the CDR 0 and CDR 0.5 groups were used to generate receiver operator characteristic (ROC) curves to compare the 8- and 10-item indexes. These analyses demonstrated that the predictive ability of the 8-item scale (area under the curve [AUC] = 0.852) was equal to that of the 10-item scale (AUC = 0.853).
The eight-item scale was then distributed to each member of the research team for review. Based on comments from the research team, two other items (difficulty voting and confuses family relationships) that were endorsed by <10% of the sample were deleted; these are behaviors that are more common in later stages of dementia. Based on a clinical rather than statistical perspective, two additional items (consistent problems with memory or thinking and reduced interest in hobbies and activities) were added to enhance the face validity of the scale to ensure that the tool included questions pertaining to memory and activities of daily living. The ability of the new eight-item scale to predict membership in the CDR 0.5 group vs the CDR 0 group was compared with that of the original eight-item index. Inspection of the ROC curves and the area under those curves indicated that the revised eight-item scale (the AD8) yielded the best predictive ability (AUC for final AD8 = 0.870).
The final eight-item version of the AD8 (see figure E-1 on the Neurology Web site at www.neurology.org) queries the CS about the participant's cognitive abilities in the areas of memory (consistent problems with memory, repetition, remembering appointments), temporal orientation, judgment (making decisions, handling finances), and function (reduced interest in activities, use of appliances). The total AD8 score is generated by summing the number of items responded to with “yes, a change”; thus, possible AD8 scores range from 0 to 8. As described earlier, the AUC was 0.870 in the developmental sample, suggesting good to excellent discrimination between CDR 0 and CDR 0.5 groups, and was comparable to the full 55-item instrument (AUC = 0.895).
The AD8 was then administered for a 6-month period to a new sample from our longitudinal study at their annual assessment. The AD8 was administered to the CS prior to the clinician's interview with the CS and assessment of the participant. The clinician was not told the answers given by the CS. At the end of the assessment, the clinician generated an independent CDR rating for the participant according to our standard procedures.
Statistical analysis.
Descriptive statistics were used to summarize sample characteristics as well as AD8 scores at each CDR level. To demonstrate how the AD8 scores compare with the independently generated CDR rating and brief office-based objective measures, correlations between AD8, CDR, MMSE, and SBT scores were determined using Pearson product-moment correlation coefficients. We hypothesized that higher AD8 scores would represent more severe stages of dementia and correlate with higher CDR stages and worsening performance on the MMSE and SBT. ROC curves and AUC were generated to reflect graphically and quantitatively the ability of the AD8 to discriminate between participants with CDR 0 and CDR 0.5 and between those with CDR 0 and CDR ≥0.5. Classification tables were constructed, and the sensitivity, specificity, and positive and negative predictive values of the AD8 were calculated. All statistical analyses were conducted using SAS.
Results.
Sample characteristics.
The AD8 was administered to 236 participants between October 6, 2003, and April 2, 2004. Characterization of CDR stages and diagnoses of the sample are shown in table 1. The participant's mean age at time of assessment was 78.1 ± 9.2 years (range 55 to 102 years), and 53% were women. The collateral sources were spouses (52%), children (29%), friends (10%), and other sources such as social workers, case managers, and health aides (9%). The participant's cognitive status ranged from nondemented (47% CDR 0) through all stages of dementia (very mild, CDR 0.5 = 29%; mild, CDR 1 = 18%; moderate, CDR 2 = 5%; and severe, CDR 3 = 1%). Clinical diagnoses from the standard assessment included nondemented elders (47% of sample), uncertain or questionable dementia (12%), and demented individuals (41%). Dementia diagnoses were largely made up of dementia of the Alzheimer type; however, other dementia diagnoses included mixed dementia, VaD, and DLB. The AD8 was administered to the CS by the center's secretarial staff, taking on average under 3 minutes to complete.
Table 1 Diagnosis and dementia severity of sample (n = 236)
AD8 scores compared with CDR stages.
Total AD8 scores were compared by CDR stages determined independently by experienced clinicians (table 2). Those individuals who were rated as CDR 0 had a mean AD8 score of 0.6 (range 0 to 5) compared with individuals who were rated as having at least very mild dementia (CDR ≥0.5) who had a mean score of ≥2.91 (range 0 to 8).
Table 2 Total AD8 score by Clinical Dementia Rating (CDR) stage
Within the CDR 0 group, six individuals had AD8 scores of >4. There was no difference between diagnoses or the type of CS to distinguish these CDR 0 individuals from those with AD8 scores of <4. Several characteristics, however, did distinguish the two groups. First, the CSs of CDR 0 individuals with high AD8 scores were more likely (5/6 or 83%) to endorse the AD8 question dealing with “consistent problems with thinking and memory” than the CSs of CDR 0 individuals with low AD 8 scores (5/101 or 5%). Second, the six CDR 0 individuals with AD8 score of >4 had a lower mean MMSE score than CDR 0 individuals with AD8 scores of <4 (26.3 vs 28.8; p = 0.0005). It is possible that these individuals had early cognitive change although insufficient to warrant a dementia diagnosis.
Within the CDR 0.5 group, there were six individuals with total AD8 scores of >6. There was no differences in caregiver type or questions endorsed, but several features distinguished these individuals from the rest of the CDR 0.5 group. Those participants with AD8 scores of >6 had lower MMSE scores (24.5 vs 26.8; p = 0.048), higher SBT scores (19.3 vs 10.8; p < 0.0001), and higher CDR SB scores (2.9 vs 1.5; p = 0.004). These individuals might be further along in the course of disease. There is the possibility that mood disorders might play a role.
The CDR 0.5 group with AD8 scores of >6 also were more likely to have a history of a mood disorder (4/6, 67%) compared with the rest of the CDR 0.5 group (17/62, 27%). Longitudinal follow-up of these individuals will be helpful to answer these questions.
Discriminative ability of the AD8.
ROC curves were generated to measure the effectiveness of the AD8 in classifying CDR 0 (nondemented) vs CDR 0.5 (very mild dementia) participants. The AUC (figure 1) is 83%, suggesting good to excellent ability to discriminate between CDR 0 and CDR 0.5 groups. Using a cut-off score of ≥2 on the AD8 to predict dementia yielded the most desirable combination of sensitivity (74%) and specificity (86%) (table 3). With a prevalence of 38%, the positive predictive value (the probability that someone with an AD8 score of ≥2 has dementia) was 76%, whereas the negative predictive value (the probability that someone with an AD8 score of <2 is nondemented) was 84%.
Figure 1. Receiver operator characteristic (ROC) curve for CDR 0 vs CDR 0.5: ROC curve comparing the AD8 scores of nondemented (CDR 0) participants with the AD8 scores of participants with very mild dementia (CDR 0.5). The area under the ROC curve is 0.834, suggesting good to excellent ability of the AD8 to discriminate between groups.
Table 3 Classification using AD8 score of >2 for CDR 0 and 0.5 groups and for all participants with CDR of >0.5
When predicting membership in the demented group (CDR ≥ 0.5) vs status as a nondemented participant (CDR 0), the ROC area under the curve (figure 2) was 90%, suggesting excellent discriminative ability. Sensitivity increased to 85%, with specificity remaining at 86% (see table 3). The positive predictive value increased (87%) and negative predictive value (84%) remained essentially the same with the increased prevalence rate (53%) and inclusion of more severely demented individuals.
Figure 2. Receiver operator characteristic (ROC) curve for CDR 0 vs CDR ≥ 0.5: ROC curve comparing the AD8 scores of nondemented (CDR 0) participants with the AD8 scores of participants with at least very mild dementia (CDR ≥ 0.5). The area under the ROC curve is 0.904, suggesting excellent ability of the AD8 to discriminate between groups.
Correlation of AD8 with other measures of cognitive status.
Strong correlations were demonstrated between the AD8 and the CDR staging (r = 0.74, p < 0.0001) and between AD8 and SBT scores (r = 0.58, p < 0.0001). An inverse correlation was found between the AD8 and MMSE scores (r = −0.64, p < 0.0001).
Discussion.
The AD8 is a brief informant-based measure that reliably differentiates nondemented from demented individuals and is sensitive to the earliest signs of cognitive change as reported by an informant. The AD8 is highly correlated with our gold standard, the CDR, as well as performance on brief objective measures such as the MMSE and SBT. The AD8 took the CS <3 minutes to complete and was administered by clerical staff.
The goals of any screening test are to separate people with a high probability of having the disease from those with a low probability and to presumptively identify unrecognized disease.30 Diagnostic confirmation is generally required. An effective screening test should be inexpensive, and its characteristics should include reliability, sensitivity, specificity, social acceptability, safety, and brevity.26 We have presented data indicating that the AD8 meets all of these requirements and discriminates cognitively healthy older adults from individuals at the very earliest symptomatic stage of dementing illnesses. The AD8 is currently being used as part of a brief dementia detection instrument (in conjunction with a patient assessment) to evaluate patients in community settings. If the instrument were used to select research participants, a different cut-off score might be chosen depending on the costs of identifying individuals (sensitivity) vs the costs of screening and excluding them (specificity).
A number of brief screening measures are already in use, but most are based on patient performance and are therefore unable to detect or quantify change from previous levels of function. Some performance-based measures are also insensitive to subtle changes in high-functioning individuals who may score well within the normal range through the early stages of dementia. These same measures also may prevent detection of dementia in individuals with poorer lifelong abilities. Further, many cognitive tests are culturally insensitive and may underestimate the abilities of African American and other minority groups.31
Informant-based assessments provide an opportunity for the clinician to assess change from the patient's prior level of function and determine interference with the patient's accustomed functioning in daily tasks. The use of an informant permits the use of patients as their own control while eliminating the need for baseline assessments. The time required to complete the informant interview to derive the CDR, however, is greater than an average office visit permits. Other informant-based interviews such as the IQCODE32,33 are also too lengthy for general use. The Observation List for Early Signs of Dementia34 is a brief 12-item measure; however, it is performed by the physician rather than caregivers or family members. The physician does not have as much opportunity to observe the patient as does a family member.
A potential drawback of the AD8 is that knowledgeable informants may not be readily available. The validity of using informant-based assessments has been addressed by several authors.35–37 In general, informants who live with the patient are able to give more accurate reports than informants who do not. Spouses are generally more accurate than other informant relationship types. The patient's educational level, social status, or neuropsychiatric symptoms do not appear to affect informant accuracy.34 In the study, we report data from several different informant relationships, ranging from spouse to close friends, social workers, or paid caregivers.
The sample for our study was not population based; as with any volunteer sample, selection biases limit generalization of the results. Our sample was largely Caucasian, so it is unknown if these results generalize to other ethnicities. Because the AD8 requires only comment on observable change in the patient's cognitive abilities, however, it is less apt to be biased by gender, education, or ethnicity. Our convenience sample includes community volunteers rather than referrals from memory disorder clinics, and the participants' gender and education attributes reflect those of the similarly aged population in the St. Louis metropolitan area. The sample is well characterized, enabling comparison of the AD8 with the longer semistructured interview used to derive the CDR as a gold standard.
We are now testing the psychometric properties of the AD8 in a clinic population alone and in combination with brief objective measures of the participant's cognitive function. The use of the AD8 in conjunction with a brief cognitive assessment could improve diagnostic accuracy in general practice and may be applicable for dementia screening in clinical trials, community surveys, and epidemiologic studies.
Footnotes
-
Additional material related to this article can be found on the Neurology Web site. Go to www.neurology.org and scroll down the Table of Contents for the August 23 issue to find the title link for this article.
Dr. Powlishta's current address is the Department of Psychology, Saint Louis University, St. Louis, MO.
Supported by grants from the National Institute on Aging (K08 AG20764, P01 AG03991, and P50 AG05681), the Longer Life Foundation, the American Federation for Aging Research, and the Alan A. and Edith L. Wolff Charitable Trust. Dr. Galvin is a recipient of the Paul Beeson Physician Faculty-Scholar in Aging Research Award.
Disclosure: The authors report no conflicts of interest.
Received December 13, 2004. Accepted in final form May 4, 2005.
References
Letters: Rapid online correspondence
REQUIREMENTS
You must ensure that your Disclosures have been updated within the previous six months. Please go to our Submission Site to add or update your Disclosure information.
Your co-authors must send a completed Publishing Agreement Form to Neurology Staff (not necessary for the lead/corresponding author as the form below will suffice) before you upload your comment.
If you are responding to a comment that was written about an article you originally authored:
You (and co-authors) do not need to fill out forms or check disclosures as author forms are still valid
and apply to letter.
Submission specifications:
- Submissions must be < 200 words with < 5 references. Reference 1 must be the article on which you are commenting.
- Submissions should not have more than 5 authors. (Exception: original author replies can include all original authors of the article)
- Submit only on articles published within 6 months of issue date.
- Do not be redundant. Read any comments already posted on the article prior to submission.
- Submitted comments are subject to editing and editor review prior to posting.
