Measuring Friedreich ataxia: Interrater reliability of a neurologic rating scale

There have been few attempts at validating neurologic assessment methods to measure disease severity in ataxias, with the exception of an interrater reliability study of the International Cooperative Ataxia Rating Scale (ICARS).¹ There are many problems in the development of such scales. Ataxias are rare disorders with variable onset age and progression rate. The phenotype can be multifaceted with many non-cerebellar signs in some patients, even within the same genotype. Clinical signs evolve in a way that can be difficult to measure: hyperreflexia may give way to areflexia or nystagmus may disappear with progression of disease. Thus “better” performance in these measures may not mean less severe disease. Some quantifiable abnormalities, such as oculomotor abnormalities, may have minimal impact on patient performance. We describe a rating scale for Friedreich ataxia (FA) and report on the interrater reliability of the instrument.

Methods.

Results.

The mean scores and SDs for each subset of the scale for the 14 patients are shown in table E-1 (on the Neurology Web site at www.neurology.org). The patients ranged from those with nearly normal ambulation to those that were severely affected.

Excellent interrater reliability as shown by high ICC was demonstrated for the following scores: disease stage, ADL, upper limb coordination, lower limb coordination, upright stability/gait, total neurologic examination, PATA rate, pegboard and gait times (table E-2). Bulbar and peripheral nerve scores were less reliable among the raters. ANOVA values indicating some rater bias were present for: disease stage, ADL, upper limb coordination, peripheral nerve scores, and total neurologic examination. Factor loadings were high for most of the measures with more scatter for bulbar, peripheral nerve, and lower limb scores.

The Promax rotation analysis (table E-3) indicates that just two factors explain 87% of the variance of the data; thus stage of disease, lower limb scores, peripheral nerve scores, and upright stability scores are highly correlated implying these measure a common underlying construct; similarly bulbar, upper limb, PATA, and pegboard scores significantly correlated among each other. Additionally, there were substantial correlations between the neurologic examination and most of the subscores with ADL and mobility measures (table).

To address the issue of fatigue and learning, the 1st and the 7th examinations for the patients tested over the course of the day were compared. The examiners involved were different for each occasion. Paired t tests of these comparisons revealed no significant trends over the course of the day. The time to complete one examination typically did not exceed 30 minutes.

Discussion.

Previous scales used for quantifying ataxic diseases have been for small-scale studies and had subjective estimates of clinical abnormalities.^5,6,7 An early scale to undergo reliability testing was not specifically designed for ataxia and most of the ataxia measures did not meet reliability criteria.⁸ Recently ICARS has been shown to have good interrater reliability.¹

The scale reported here included neurologic signs reflecting specific neural substrates that are affected in FA. The scale was found to have high ICC values for most of its components, indicating acceptable interrater reliability. We have not yet tested its intrarater reliability. The scale also incorporated a functional stage to denote overall mobility and an ADL score to examine functional status, not addressed in previous ataxia instruments. Components of the neurologic examination correlated well with ADL and mobility scores, demonstrating construct validity of the scale as a measure of FA, though further studies are needed to demonstrate validity and sensitivity to change.

Timed activities tested had high interrater reliability and less rater bias, as indicated by nonsignificant ANOVA p values, and may be especially useful to detect mild to moderate changes in disease severity. The correlations of individual timed measures with ADLs and mobility were lower, suggesting that a composite of both timed and examination measures may be needed to accurately reflect the global dysfunction in FA.

The correlations found using Promax rotation analysis appear to support the global nature of the scale, with one set of measures reflecting lower body dysfunction and the other, upper body dysfunction. Because the factor analysis of the subscales indicated possibly two underlying constructs at work, we did not compute Cronbach’s α.

Larger scale studies should provide evidence for the validity of the scale.