Abstract
Objective: To develop a reliable and valid clinical scale measuring the severity of ataxia.
Methods: The authors devised the Scale for the Assessment and Rating of Ataxia (SARA) and tested it in two trials of 167 and 119 patients with spinocerebellar ataxia.
Results: The mean time to administer SARA in patients was 14.2 ± 7.5 minutes (range 5 to 40). Interrater reliability was high, with an intraclass coefficient (ICC) of 0.98. Test-retest reliability was high with an ICC of 0.90. Internal consistency was high as indicated by Cronbach's α of 0.94. Factorial analysis revealed that the rating results were determined by a single factor. SARA ratings showed a linear relation to global assessments using a visual analogue scale, suggesting linearity of the scale (p < 0.0001, r2 = 0.98). SARA score increased with the disease stage (p < 0.001) and was closely correlated with the Barthel Index (r = −0.80, p < 0.001) and part IV (functional assessment) of the Unified Huntington's Disease Rating Scale (UHDRS-IV) (r = −0.89, p < 0.0001), whereas it had only a weak correlation with disease duration (r = 0.34, p < 0.0002)
Conclusions: The Scale for the Assessment and Rating of Ataxia is a reliable and valid measure of ataxia, making it an appropriate primary outcome measure for clinical trials.
With continuous progress in the understanding of the molecular pathogenesis of many ataxia disorders, novel therapies are on the horizon that will need to be evaluated in clinical trials. An essential prerequisite for such trials is the availability of validated neurologic assessment methods to measure the severity of ataxia. Although a number of ataxia rating scales have been proposed,1–4 there have been only few attempts to validate these scales. A widely used ataxia rating scale is the International Cooperative Ataxia Rating Scale (ICARS). Although interrater reliability of this scale was shown to be high,5 more detailed evaluations revealed problems concerning practicability and subscale structure that questioned the usefulness of ICARS for future interventional trials.6 Recently, a new scale, Friedreich's Ataxia Rating Scale, was devised and validated.4 However, this scale is not readily applicable to cerebellar ataxias because it was designed as a disease-specific scale for the evaluation of Friedreich ataxia and thus also considers other symptoms than ataxia.
We therefore devised a new scale named Scale for the Assessment and Rating of Ataxia (SARA) that is based on a semiquantitative assessment of cerebellar ataxia on an impairment level. We report here the results of a large two-step validation trial performed in patients with spinocerebellar ataxia (SCA).
Methods.
Description of the scale.
SARA was devised during a meeting held in February 2004 in Bonn, Germany, by a group of European neurologists. Test items were selected according to specificity for ataxia and the possibility to standardize testing and rating procedures. Moreover, assessment had to be close to standard neurologic examination and not require technical equipment. In its initial form, SARA was composed of nine items including one item related to oculomotor function. Based on the results of Trial 1, SARA was modified and the oculomotor item was omitted. In its final form, SARA has eight items that yield a total score of 0 (no ataxia) to 40 (most severe ataxia); 1: gait (score 0 to 8), 2: stance (score 0 to 6), 3: sitting (score 0 to 4), 4: speech disturbance (score 0 to 6), 5: finger chase (score 0 to 4), 6: nose-finger test (score 0 to 4), 7: fast alternating hand movements (score 0 to 4), 8: heel-shin slide (score 0 to 4). Limb kinetic functions (items 5 to 8) are rated independently for both sides, and the arithmetic mean of both sides is included in the SARA total score (see appendix E-1 on the Neurology Web site at www.neurology.org).
Development of the scale.
The study population consisted of 167 patients with SCA (SCA1: 23 patients, SCA2: 56 patients, SCA3: 23 patients, SCA6: 18 patients, SCA7: seven patients, SCA14: two patients, SCA17: one patient, unknown mutation: 29 patients) and eight controls in Trial 1, and 119 patients with SCA (SCA1: 15 patients, SCA2: 28 patients, SCA3: 26 patients, SCA6: 19 patients, SCA7: eight patients, SCA14: one patient, SCA17: three patients, SCA23: one patient, unknown mutation: 18 patients) and 110 controls in Trial 2. Clinical characteristics are given in table 1. Inclusion criteria for the patients with SCA were 1) progressive ataxia and 2) positive genetic testing for any SCA mutation or linkage to any SCA locus or clinical evidence of autosomal dominant inheritance. The majority of patients (Trial 1: n = 97, Trial 2: n = 95) were rated independently by two investigators. In Trial 2, a number of patients were rated twice with an interval of 1 to 34 days by as many as three investigators, resulting in a set of 15 intrarater assessments. All investigators were neurologists or senior residents in neurology training. There was a maximum of three investigators at each center. Investigators did not receive formal training, but were asked to closely follow the instructions that are part of the SARA scale (see appendix E-1).
Table 1 Clinical characteristics of the study population
In Trial 1, SARA was compared with ataxia disease stages, ICARS, and the Barthel Index and in Trial 2 with ataxia disease stages, Barthel Index, and the part IV (functional assessment) of the Unified Huntington's Disease Rating Scale (UHDRS-IV).3,7–9 Disease stages were defined as follows: Stage 0: no gait difficulties; Stage 1: disease onset, as defined by onset of gait difficulties; Stage 2: loss of independent gait, as defined by permanent use of a walking aid or reliance on a supporting arm; Stage 3: confinement to wheelchair, as defined by permanent use of a wheelchair.7 The ICARS results have been reported elsewhere.6
Thirty assessments of Trial 2 were video recorded. Eleven videos that were representative of the entire range of clinical severity were selected and randomly ordered on a video disk. From these videos, SARA rating was done by two examiners who were not involved in the clinical assessments. In addition, the severity of ataxia of the 11 cases presented on video was globally assessed by 14 examiners on a visual analogue scale (VAS) ranging from 0 (no ataxia) to 100 (most severe ataxia).
The study was approved by the ethics committees of the participating centers. Informed and written consent was obtained from all study participants.
Statistical analyses.
Data are given as mean ± SD and range. Interrater and test-retest reliability was expressed using intraclass coefficients (ICCs), internal consistency using Cronbach's α coefficient. Calculation of ICCs was based on a model with only random effects. Coefficients exceeding 0.80 are considered acceptable for scales used to make group comparisons. In order to analyze internal validity, a principal component analysis was performed using varimax rotation.
In order to test the linearity of SARA, a regression analysis of the video ratings with the VAS assessments as a reference was performed. To make SARA and the VAS values directly comparable, SARA values were multiplied with a factor of 2.5 to yield a range from 0 to 100. As SARA will be used to measure differences between two assessments, i.e., before and after treatment, we also verified the linearity of differences between two measures. To this end, all differences between patient ratings of a given examiner were computed. For both analyses, a linear model was tested. For the SARA scores, a model with no intercept was used, and for the differences, a model with an intercept was used.
To test for correlation between SARA scores and Barthel Index, UHDRS-IV, and disease duration, Pearson's correlation test was used. The relation between SARA scores and disease stages was analyzed with analysis of variance. All tests were two sided; p < 0.05 was considered significant. Statistical analyses were performed using the SAS 8.2 statistical package (SAS Institute, Cary, NC).
Results.
Analysis of Trial 1 showed high interrater reliability of SARA total score (ICC = 0.97) and of the items related to gait, stance, sitting, fast alternating hand movements, and heel-shin slide. In contrast, the items related to finger chase, nose-finger test, speech, and oculomotor function were less successful, with ICC <0.80 (range 0.70 to 0.79). Internal consistency was high, with Cronbach's α of 0.90; it increased when the oculomotor item was omitted. A factorial analysis showed that the results of all items, with the exception of the oculomotor item, were determined by a single factor. Based on these results, instructions for the items with interrater reliability <0.80 were modified, and the oculomotor item was omitted.
Analysis of Trial 2 determined the metric properties of SARA in its final form (see appendix E-1). In the patient group, the mean SARA score was 15.9 ± 8.5 (range 1.5 to 40). Ceiling effects were negligible, with one patient (0.84%) receiving the maximal possible score. Skewness of the distribution was 0.74. The mean SARA score of the control group was 0.4 ± 1.1 (range 0 to 7.5), and 79% of controls had a score of 0. Positive ratings in controls were mainly obtained in items 5 and 8 left. Mean time to administer SARA was 14.2 ± 7.5 minutes (range 5 to 40) in patients and 7.2 ± 2.6 minutes (range 3 to 13) in controls.
Based on the results obtained in 95 patients assessed by two examiners, interrater reliability was very high, with an ICC of 0.98. All single items had good interrater reliability, with ICC >0.80 with exception of item 6 left (ICC = 0.76) and item 8 left (ICC = 0.74). Based on a limited set of 15 intrarater assessments, test-retest reliability was high, with an ICC of 0.90. Internal consistency was high, as indicated by Cronbach's α of 0.94; it did not increase when any item was deleted (table 2).
Table 2 Reliability and internal consistency of SARA
Factorial analysis revealed that the rating results in all items were determined by a single factor with an eigenvalue of 6.4, which explained 80% of the variance. All other factors had eigenvalues far less than 1.
Linearity of SARA was tested by a regression analysis of video ratings of 11 patients with global assessments as a reference. The SARA ratings and the differences between measures fitted a linear model (SARA: p < 0.0001, r2 = 0.98; differences: p < 0.0001, r2 = 0.72) (figure 1).
Figure 1. Regression analysis of Scale for the Assessment and Rating of Ataxia (SARA) video ratings of 11 representative patients with global assessments as a reference. SARA rating was done by two examiners who were not involved in the clinical assessments. SARA values were multiplied with a factor of 2.5 to yield a range from 0 to 100. Severity of ataxia was globally assessed by 14 examiners on a visual analogue scale (VAS) ranging from 0 (no ataxia) to 100 (most severe ataxia). (A) SARA ratings (p < 0.0001, r2 = 0.98). (B) Differences between SARA ratings of a given examiner (p < 0.0001, r 2 = 0.72).
The SARA score increased with the disease stage (p < 0.001) (figure 2A). The SARA score was closely correlated with the Barthel Index (r = −0.80, p < 0.0001) and UHDRS-IV (r = −0.89, p < 0.0001), while it had only a weak correlation with disease duration (r = 0.34, p < 0.0002) (figure 2, B through D). Further analysis showed that the correlation between SARA score and disease duration was better, when only SCA-1 (r = 0.74, p < 0.003) or SCA3 patients (r = 0.59, p < 0.002) were considered. This was not found for SCA2 (r = 0.08, p > 0.68) and SCA6 (r = 0.26, p > 0.28).
Figure 2. Validation of Scale for the Assessment and Rating of Ataxia (SARA) with external measures. (A) SARA scores at different ataxia disease stages. Data are given as box plots. (B) Correlation of the SARA score with Barthel Index (r = −0.80, p < 0.0001). (C) Correlation with UHDRS-IV (r = −0.89, p < 0.0001). (D) Correlation with disease duration (r = 0.34, p < 0.0002).
Discussion.
SARA, the new ataxia scale reported here, includes eight items reflecting neurologic manifestations of cerebellar ataxia. SARA only rates ataxia-related symptoms and does not consider nonataxia symptoms that often occur in patients with SCA. Therefore, it is possible that disease severity in certain diseases with extracerebellar features might not be faithfully reflected in the SARA score. However, consideration of nonataxia symptoms would have precluded the development of a scale that meets basic biometric requirements. To account for nonataxia symptoms, we have devised an Inventory of Non-Ataxia Symptoms (INAS) that provides structured and semiquantitative information on nonataxia symptoms. The items selected for SARA follow the steps of a standard neurologic examination. Oculomotor functions, which were part of the initial SARA version, are not considered in the final SARA version because the first validation trial indicated two underlying constructs, one globally reflecting cerebellar ataxia and the second reflecting solely oculomotor dysfunction.
SARA underwent a rigorous validation procedure involving two clinical trials in large groups of patients with SCA and controls. The number of patients with SCA included in these trials is far greater than the number included in any previous clinical study of patients with SCA. Clinical presentation of most patients with SCA is characterized by prominent cerebellar ataxia. They therefore represent an appropriate study population for the validation of an ataxia scale. To demonstrate that SARA is also appropriate for other groups of ataxia patients, we are currently conducting further clinical studies.
There was a clear separation of the SARA ratings in patients and controls. Nevertheless, about 20% of controls had positive ratings in at least one item. The items producing the majority of positive ratings were related to kinetic functions of the nondominant side. These observations show that SARA is not an ideal clinical instrument to detect disease onset. Elimination of these items would increase the discriminatory power of SARA but reduce its sensitivity.
SARA was easy to administer and required less than 15 minutes per patient. This time is considerably shorter than the time required to complete ICARS.6 SARA satisfied a number of accepted criteria of reliability. Interrater reliability was high, both for the total score and for most of the items. At the single-item level, SARA performed better than ICARS.6 Although based on a small number of observations, our data suggest acceptable test-retest reliability of the total SARA score. However, meaningful analysis of the test-retest reliability of each item was not possible. Because the calculation of ICCs was based on a model with only random effects, the results are independent of the investigators and can be generalized to investigators with the same level of training as the investigators of this study. The internal consistency of SARA was high, as indicated by Cronbach's α of 0.94.
Most importantly, SARA results were determined by a single factor, which indicates that SARA measures a common underlying construct, cerebellar ataxia. In this respect, SARA favorably contrasts with ICARS, the results of which were shown to be influenced by four different factors that did not coincide with the ICARS subscales.6
Linearity of a scale and particularly of the differences between ratings is an important prerequisite for the use of a scale in clinical trials that include patients with a wide spectrum of clinical severity. To demonstrate linearity, we performed a regression analysis of the SARA ratings of two examiners with global assessments of 14 examiners as a reference. Our analysis showed that SARA ratings and the differences between ratings fitted a linear model. A limitation of this analysis comes from the fact that these ratings were done on videos.
To further demonstrate the validity of SARA, we used external measures. SARA scores increased with ataxia disease stages, as defined in a previous study.7 In addition, SARA scores decreased with the Barthel Index, which measures activities of daily living and is routinely used to evaluate stroke patients.8 SARA also decreased with UHDRS-IV, which provides a functional assessment of patients with Huntington disease (HD).9 Because patients with ataxia and patients with HD face qualitatively similar functional limitations in daily life, it appeared appropriate to use this scale as an external validation criterion. These observations are consistent with the hypothesis that SARA is a valid method to measure the severity of ataxia. Based on the clinical knowledge that SCAs are progressive, but that the progression rate is variable depending on the genotype, repeat length, and unknown genetic and nongenetic factors,7,10 only a moderate correlation between the SARA score and disease duration was expected. This was indeed found in the present study. Separate analysis of the four most common genotypes showed that the correlation between SARA score and disease duration was better, when only patients with SCA1 or SCA3 were considered. This was not found for SCA2 and SCA6. However, the small number of patients in each subgroup did not allow us to draw any final conclusions.
Acknowledgment
The authors thank the probands for participating in this study. The work of M.E.E.C. Glazowski (Warsaw) in videotaping of assessments is gratefully acknowledged. They also thank all other clinical investigators who contributed to data acquisition: S. Bonato (Brussels); M. Abele, M. Minnerop, B. Viebahn, and I. Vogt (Bonn); S. Romano (Milan); W.F. Abdo and H.J. Schelhaas (Nijmegen); C. Charles (Paris); and J. Zaremba and E. Zdzienicka (Warsaw).
Footnotes
-
Additional material related to this article can be found on the Neurology Web site. Go to www.neurology.org and scroll down the Table of Contents for the June 13 issue to find the title link for this article.
See also page 1711
Supported by grant EUROSCA/LSHM-CT-2004-503304 from the European Union, grant GeneMove/01 GM 0503 from the German Ministry of Education and Research, and grant 3 PO5B 019 24 from the Polish Ministry of Scientific Research and Information Technology.
Disclosure: The authors report no conflicts of interest.
Received October 10, 2005. Accepted in final form March 6, 2006.
Letters: Rapid online correspondence
REQUIREMENTS
You must ensure that your Disclosures have been updated within the previous six months. Please go to our Submission Site to add or update your Disclosure information.
Your co-authors must send a completed Publishing Agreement Form to Neurology Staff (not necessary for the lead/corresponding author as the form below will suffice) before you upload your comment.
If you are responding to a comment that was written about an article you originally authored:
You (and co-authors) do not need to fill out forms or check disclosures as author forms are still valid
and apply to letter.
Submission specifications:
- Submissions must be < 200 words with < 5 references. Reference 1 must be the article on which you are commenting.
- Submissions should not have more than 5 authors. (Exception: original author replies can include all original authors of the article)
- Submit only on articles published within 6 months of issue date.
- Do not be redundant. Read any comments already posted on the article prior to submission.
- Submitted comments are subject to editing and editor review prior to posting.
You May Also be Interested in
Hastening the Diagnosis of Amyotrophic Lateral Sclerosis
Dr. Brian Callaghan and Dr. Kellen Quigg