Abstract
Objective: To perform a psychometric evaluation of the inflammatory neuropathy cause and treatment (INCAT) sensory sumscore (ISS) in sensory–motor immune-mediated polyneuropathies. This new sensory scale was evaluated to strive for uniformity in assessing sensory deficit in these disorders.
Methods: The ISS comprises vibration and pinprick sense plus a two-point discrimination value and ranges from 0 (normal sensation) to 20 (maximum sensory deficit). Before its clinical use, a panel of expert neurologists concluded that the ISS has face and content validity. The construct validity of the ISS was investigated by correlation and regression studies with additional scales (Nine-Hole Peg Test, 10-Meter Walking Test, a disability sumscore). All scales were applied in 113 patients with a stable neurologic condition (83 patients who experienced Guillain–Barré syndrome [GBS] in the past, 22 with chronic inflammatory demyelinating polyneuropathy [CIDP], 8 patients with a monoclonal gammopathy associated polyneuropathy), and 10 patients with recently diagnosed GBS or CIDP with changing clinical conditions. Reliability of the ISS was evaluated in the stable patients. Its responsiveness was investigated in the patients examined longitudinally.
Results: A moderate to good validity was obtained for the ISS (stable group: r = 0.38 to 0.56, p ≤ 0.006; longitudinal group: R = 0.60 to 0.82, p ≤ 0.007, except for the association with the 10-Meter Walking Test [p = 0.08]). Acceptable internal consistency, and inter- and intraobserver reliability were demonstrated for the ISS (α = 0.68 to 0.87; R = 0.85 to 0.89, p < 0.0001). Standardized response mean scores for the ISS were high (≥0.8), indicating good responsiveness.
Conclusions: All psychometric requirements are provided for the the inflammatory neuropathy cause and treatment sensory sumscore. The use of this scale is therefore suggested for bedside evaluation of sensory deficit in the individual patient with a sensory–motor immune-mediated polyneuropathy as well as in clinical trials.
The increased emphasis on accuracy in clinical neurologic studies has intensified the need for the use of psychometric well-evaluated outcome measures to quantify relevant deficits. In immune-mediated polyneuropathy studies, including patients with Guillain–Barré syndrome (GBS) or chronic inflammatory demyelinating polyradiculoneuropathy (CIDP), weakness has been assessed primarily using various valid and reliable motor scales that are based on the Medical Research Council grading system.1-4 Conversely, assessment of sensory deficit has been performed less in these patients, although sensory abnormalities may contribute to disability.5 It was argued that sensory deficit tends to be less obvious clinically and more prone to subjective interpretation compared with motor deficit.6 These suggestions may therefore explain the difficulties when assessing sensory deficit and the use of various sensory scales in immune-mediated polyneuropathy studies thus far, most of them not fulfilling all psychometric requirements, such as being valid, reliable, and responsive to clinical changes over time.5,7-24
Prompted by these observations, we constructed a new sensory scale—the inflammatory neuropathy cause and treatment (INCAT) sensory sumscore (ISS)—to strive for uniformity in assessing, during bedside examinations, various sensory qualities representing different types of nerve fibers. This scale was constructed after a systematic literature review of all sensory methods applied in sensory–motor immune-mediated polyneuropathy clinical studies from January 1988 to January 1999. The ISS was created on behalf of the INCAT group, a collaborating force of European neurologists with a special interest in immune-mediated polyneuropathies. These neurologists formed a panel that contributed to the formation and extensive evaluation of the ISS. Moreover, the psychometric requirements (validity, reliability, and responsiveness) for the ISS were examined in patients with GBS, CIDP, or a monoclonal gammopathy of undetermined significance related polyneuropathy (MGUSP).25 These disorders are suggested to represent parts of a continuum regarding their neuromuscular dysfunction pattern.26 The construct validity of the ISS was calculated by correlation and regression studies with the measured values of the Nine-Hole Peg Test (a dexterity test), the 10-Meter Walking Test, and an overall disability sumscore (DSS) in all patients.27-29
Methods.
Patients.
A total of 113 patients (83 GBS, 22 CIDP, 8 MGUSP) with a stable clinical condition were recruited from the Rotterdam immune-mediated polyneuropathy databank and the Dutch GBS study group (stable group). The selected patients still had residual symptoms or signs due to their illness, representing a broad range of disability. Nine CIDP patients required interval treatment ranging from weeks to months, with IV immunoglobulins (IVIg). With this therapy their clinical condition has been stable for more than 6 months. Additionally, 10 patients with recently diagnosed GBS (n = 4) or CIDP (n = 6) with changing clinical conditions were enrolled to investigate the responsiveness of the ISS (longitudinal group). All GBS and CIDP patients met the international criteria for their illness.30,31 The diagnosis of MGUSP was established after excluding all possible causes for the gammopathy and polyneuropathy.32
Literature review.
A systematic MEDLINE search was performed from January 1988 to January 1999, reviewing all methods (apart from traditional neurologic examination) evaluating the sensory system in clinical studies including sensory–motor immune-mediated polyneuropathies. We investigated whether these scales have been evaluated formally in terms of being valid, reliable, and responsive before their use as a neurologic outcome measure.25 Reports published in English that included 10 patients or more were identified using the following key words: GBS, CIDP, acquired/idiopathic (poly)(radiculo)neuropathy, polyneuritis, gammopathy, dysimmune, paraprotein(a)emia, and monoclonal gammopathy of undetermined significance polyneuropathy.
Assessment tools/scales.
The conceptual framework of the ISS was created using a judgmental approach based on the literature review and the consensus of an expert panel consisting of 13 senior neurologists (all INCAT members) with a special interest in immune-mediated neurologic disorders.33 The ISS ranges from 0 (normal sensation) to 20 (most severe sensory deficit) and is composed of the summation of the following sensation qualities: pinprick arm grade (range, 0 to 4) and vibration arm grade (range, 0 to 4), and pinprick leg grade (range, 0 to 4) and vibration leg grade (range, 0 to 4), and two-point discrimination grade (range, 0 to 4). Pinprick was tested using the sharp end of an esthesiometer. Patients were asked to indicate whether they experienced the pinprick as normal or abnormal. Paresthesia, dysesthesia, or hyperesthesia were scored as abnormal. We sought a normal reference point (e.g., the face) if a patient was experiencing problems indicating whether the pinprick was normal or not. Vibration sense was tested using the validated graduated Rydel–Seiffer tuning fork, and the measures obtained were compared with the reported normative threshold values.34 The sites of examination with corresponding grades were defined as follows: normal (grade 0) or disturbed (grade 1) pinprick or vibration sense at the dorsum distal interphalangeal joint of the index finger or hallux, and abnormal sense at the ulnar styloid process or medial malleolus (grade 2), at the medial humerus epicondyle or patella (grade 3), and at the acromioclavicular joint or anterior superior iliac spine (grade 4). Pinprick and vibration sense examination took place distal to proximal, and only the highest extension of dysfunction of the most affected arm and leg was recorded separately for both qualities. If, for example, the vibration sense was scored as abnormal at the index finger at both sides, but as normal at the styloid process, a more proximal examination was not performed. This patient would have a vibration grade score of 1 for the arms. For the two-point discrimination quality, a hand-held esthesiometer was used where the exact measurable distance in millimeters could be read on the instrument. This instrument was assessed in a “static” manner at the ventral side, distal phalanx of the index finger, and the corresponding grades were chosen arbitrarily (grade 0, ≤4 mm; grade 1, 5 to 9 mm; grade 2, 10 to 14 mm; grade 3, 15 to 19 mm; grade 4, ≥20 mm; table 1).
Inflammatory neuropathy cause and treatment sensory sumscore (ISS)*
The DSS is composed of an arm and leg disability scale with a total score ranging from 0 (no signs of disability) to 12 (most severe disability score).29 The DSS comprises a good functional description of the arms and legs in a checklist form suitable for interviewing patients. The Nine-Hole Peg Test and the 10-Meter Walking Test were also applied to all patients to measure focal disability.6,27,28
Test procedures.
General aspects.
All participants gave informed consent before the start of the study. All measures were obtained in a quiet, temperature-controlled room (approximately 20 °C) at our outpatient clinic. Sensory modalities were examined in random order with the patients lying in a supine position. The graduated tuning fork was used as described previously.34 Briefly, this tuning fork was applied as perpendicular as possible, resting on its own weight with the arms of the fork swinging maximally. The participants indicated the moment when they no longer perceived the decreasing vibration stimulus. The readings of three repeated tests were averaged and considered the vibration value for that site of examination. The time to complete the ISS was recorded at each assessment (in seconds).
All patients received training in assessing the Nine-Hole Peg Test before the start of the study to exclude any training effect. This test was performed under the prescribed standard conditions, in alternating order for both hands.6,27 Patients were also requested to walk 10 m in a straight line at their preferential speed, using whatever aid needed.6,28 Three measures were completed for each of these tests and the corresponding time was recorded at each assessment (in seconds). For each test separately, the mean time of completion was calculated by averaging the three obtained measures.
Validity and reliability.
The INCAT expert panel was asked at various occasions to review the ISS and comment on its general structure and components, and to indicate whether the scale measures what it is supposed to measure, thus providing face and content validity for the ISS. Construct validity of the ISS was investigated by correlation and regression studies with additional scales (Nine-Hole Peg Test, 10-Meter Walking Test, and a DSS).
For the assessment of reliability and construct validity of the ISS in the stable group of 113 patients, 2 senior neurologists and 6 experienced residents in neurology formed 28 different couples. Preceding the study, all investigators received instructions in assessing the outcome measures. Twenty-seven (variable) couples investigated 68 patients (2 to 3 patients per couple). The remaining 45 stable patients were investigated by the “experienced” couple. The latter couple was formed to examine the effect of training (and thus a possible increase in reliability) when using the ISS often.
The patients were examined at two different occasions at our outpatient clinic. During the first visit the two members of an appointed pair acquired their scores independently and consecutively (usually within 2 hours; interobserver measures). Within 2 to 4 weeks, the patient returned for a second visit and only one investigator of the earlier assigned pair reexamined the patient (intraobserver values) without having access to the previous results. The assessment sequence at entry and the examination at the second visit were distributed equally among the members of an assigned couple. Eventually, each member of a couple examined approximately the same number of patients. The Nine-Hole Peg Test, 10-Meter Walking Test, and the DSS were assessed only once by the first examiner in each patient.
Responsiveness.
Ten patients were examined longitudinally by the same clinician (I.S.J.M.), and all scales were assessed at study entry and 8 to 13 times in each patient during follow-up. There was a standard follow-up schedule (week 0, 2, 4, 8, 12, 16, 21, 26, 32, 40, and 52) with additional clinical investigations if necessary. During each visit, the patients were requested to judge whether their clinical condition deteriorated (grade 1), remained stable (grade 2), or improved (grade 3) when compared with the last visit (clinical judgment scores). The study took place between March 1997 and January 1999.
Statistics.
Validity, internal consistency, and interobserver and intraobserver reliability.
In the stable group, correlation between the ISS and additional scales was analyzed using Spearman’s rank correlation coefficient. Also, random effects linear regression analyses of the ISS on additional scales were performed in the longitudinal group, taking into account the correlation of the data caused by the longitudinal structure. The latter was achieved using the program “xtreg” in STATA 5.0 for Windows 95 (Stata Statistical Software, release 5.0, 1997; Stata Corporation, College Station, TX), which is based on a cross-sectional time series regression model as described by Dwyer and Feinleib.35A logarithmic transformation was applied to the variables (ISS, Nine-Hole Peg Test, 10-Meter Walking Test, DSS) before the regression analyses.
Cronbach’s alpha coefficient was estimated for the ISS in both (stable and longitudinally followed) patients groups.36 The inter- and intrarater reliability for the ISS values in the stable group of patients was quantified by estimation of the intraclass correlation coefficient using a one-way random effects analysis of variance (ANOVA) model for the two investigator (experienced and variable) groups.
Responsiveness.
The association between the serially obtained ISS values and the clinical judgment scores in these patients was estimated using xtreg.35 Responsiveness was also investigated by calculating the standardized response mean (SRM) scores for the ISS at various, arbitrarily chosen occasions during follow-up (weeks 2, 4, 12, 26, and 52).37 SRM is equal to the mean change in score divided by the SD of the change in score.37 An SRM value between 0.5 and 0.8 is considered moderate responsiveness, and ≥ 0.8 as high responsiveness.37,38 Median ISS values at 12, 26, 40, and 52 weeks of follow-up were compared with the median value at entry (Wilcoxon’s signed rank test). All analyses were performed using Stata 5.0 for Windows 95. A value of p ≤ 0.05 was considered significant.
Results.
General aspects, and face and content validity of the ISS.
The ISS was evaluated thoroughly by the expert panel of 13 neurologists and was modified according to their recommendations. Eventually, all members of this panel concluded that the ISS has face and content validity. Subsequently, this scale was assessed in the selected patients. All eight examiners who investigated the patients concluded that the ISS was administered easily. A median time of 270 seconds (range, 84 to 751 seconds) was needed to complete the ISS.
The stable group of patients (54 women, 59 men; median age, 56 years; range, 14 to 84 years) had a median duration of symptoms till onset of the study of 5.1 years. In these patients, the ISS had a median value of 3 at all three assessments (range at first assessment, 0 to 15; range at second and third assessments, 0 to 18). The median time required to complete the Nine-Hole Peg Test in these patients was 23.1 seconds (range, 14.6 to 143.6 seconds) for the right hand and 23.8 seconds (range, 15.8 to 192.2 seconds) for the left hand. A median time of 8.3 seconds (range, 5.4 to 32 seconds) was needed to walk 10 m. The DSS ranged from 0 to 11 (median value, 4).
Construct validity, internal consistency, and interobserver and intraobserver reliability of the ISS.
The internal consistency value for the ISS in the stable group of patients was 0.68, 0.73, and 0.71 at the first, second = and third assessments. An internal consistency value of 0.87 was obtained for the ISS in the patients examined longitudinally. The ISS correlated moderately with the additional scales in the stable group, thus demonstrating its validity (Spearman’s rank correlation coefficient; r = 0.38 to 0.56, p ≤ 0.006; table 2). In the patients followed longitudinally, a significant correlation was also demonstrated between the ISS and additional scales except with the 10-Meter Walking Test (figure 1). The inter- and intraobserver reliability values for the ISS were as follows: R = 0.85 to 0.89 (ANOVA, p < 0.0001 for all associations; see table 2).
Validity and reliability analyses of the INCAT sensory sumscore (ISS) in a stable group of patients with immune-mediated polyneuropathies (n , 113)
Figure 1. Ten patients with immune-mediated polyneuropathies were followed longitudinally. A total of 109 visits were completed. Random effects linear regression analyses were performed of the inflammatory neuropathy cause and treatment sensory sumscore (ISS; range, 0 [normal sensation] to 20 [most severe sensory deficit]) on additional scales using “xtreg”35 (see Statistics). The disability sumscore ranges from 0 (no signs of disability) to 12 (most severe disability).
Responsiveness of the ISS.
Four women and six men (median age, 40.5 years; range, 15 to 70 years) were examined longitudinally. A total of 109 visits were completed in these patients. The follow-up period ranged from 52 to 58 weeks. At entry, seven patients (three GBS patients, four CIDP patients) were unable to walk independently. All patients experienced general loss of strength and sensory disturbances. Except for one GBS patient, who only experienced mild symptoms, all patients received initial treatment with IVIg (0.4 g/kg body weight/day for five consecutive days). During follow-up, all patients showed good physical and functional improvement compared with entry. The GBS patients did not show any deterioration during follow-up. After initial improvement, all six CIDP patients showed some clinical deterioration. Remarkably, four of these patients demonstrated at various moments an increase in sensory symptoms and signs with relatively stable muscle strength conditions (figure 2). Consequently, the intervals of IVIg therapy were shortened to regain the earlier achieved clinical and functional improvement.
Figure 2. Two patients with chronic inflammatory demyelinating polyneuropathy treated with intermittent IV immunoglobulins (IVIg) were selected to illustrate the prominent changes of the inflammatory neuropathy cause and treatment sensory sumscore (ISS; range, 0 [normal sensation] to 20 [most severe sensory deficit]) compared with muscle strength (Medical Research Council sumscore [MRCS]: range, 0 [total paralysis] to 60 [maximum strength]).3 The triangles (▵) indicate infusion with IVIg. The first treatment (week 0) consisted of five consecutive days of IVIg 0.4 g/kg body weight/day. Subsequent treatments (triangles) indicate only 1 to 2 days of IVIg therapy with 0.4 g/kg body weight/day.
These longitudinally followed patients graded their clinical condition 31 times as “deteriorating,” 21 times as “stable,” and 57 times as “improving.” There was a general reduction in ISS values (compatible with improvement) during follow-up. These values were associated significantly with the clinical judgment scores by these patients (p < 0.0001; figure 3). The median ISS values during follow-up were lower (2, 0, 3, and 1 at the weeks 12, 26, 40, and 52) compared with the median value of 6.5 at entry (Wilcoxon’s signed rank test, p = 0.05 to 0.006). The calculated SRM scores for the ISS were good (SRM values: 0.8, 1.0, 1.6, 2.2, and 1.1 at weeks 2, 4, 12, 26, and 52).
Figure 3. Ten patients with sensory-motor immune-mediated polyneuropathies were examined longitudinally. For each patient, differences between all two consecutive inflammatory neuropathy cause and treatment sensory sumscore (ISS) values were calculated (ISS value at visit-i − ISS value at visit-(i-1) , ISS changes). These differences were associated with the corresponding clinical judgment scores using “xtreg”35 (see Statistics). The circles in the figure represent the measured ISS changes for all patients. A reduction in ISS changes (compatible with less sensory disturbances over time) was associated highly with the clinical judgment scores by these patients (random effects linear regression analyses, p < 0.0001).
Discussion.
Instruments for measuring outcome must be appropriate to the patient group being studied, must not be time-consuming, and must be easy to administer. Also, fulfillment of the psychometric demands (being valid, reliable, and responsive to changes over time) is essential in the evaluation of such an instrument.25 In the current study, the ISS demonstrated itself to be easily applicable and only required a median time of 4.5 minutes for the evaluation of various sensory modalities in patients with sensory–motor immune-mediated polyneuropathies. Because its validity, reliability, and responsiveness were also demonstrated in these conditions, the ISS fulfills all psychometric requirements.25 Furthermore, the reproducibility of the ISS turns out to be independent of the frequent use of this scale, because the reliability values did not differ between the variable and experienced investigative couples.
Neurologic examination has been employed traditionally to assess different sensory qualities in a wide range of disorders. During the past few decades, several attempts were made in sensory–motor immune-mediated polyneuropathy studies to quantify the results of sensory examination to reach a single score that describes a clinically detected neurologic deficit. Various sensory grading systems were applied using arbitrarily defined scoring systems.5,7-24 Unfortunately, except for the sensory subset of the neurologic disability score (NDS), none of these grading systems has been submitted to a comprehensive psychometric evaluation before their general use. Regarding the NDS sensory subset, its validity and reliability were demonstrated in diabetes patients with signs of a polyneuropathy.39-41 Good internal consistency was obtained for this scale in patients with hereditary and other polyneuropathies.24 Despite these observations, the NDS has some limitations. First, the NDS does not provide information regarding the more proximal extension of sensory disturbances because sensory qualities are only assessed at the index finger and hallux. Second, despite its general use, the NDS has not been investigated systematically in terms of its statistic and heuristic responsiveness.42 The latter is not surprising because, of all the psychometric requirements, responsiveness has been the least studied in evaluation of outcome measures in general.42
Clinicians and researchers need measures that discriminate between irrelevant changes (normal, minor fluctuations in the activity of an illness; “noise”) and clinically meaningful changes on which a treatment policy can be based (“signal”). For this purpose, the usefulness of an outcome instrument depends not only on its simplicity, applicability, validity, and reliability, but also on its ability to detect these meaningful changes—an ability often addressed as “responsiveness.”42,43 A statistic and heuristic approach to examining responsiveness of a measure has been proposed.42 Statistical responsiveness captures the ability of an instrument to measure any change, irrespective of its relevance. Heuristic techniques are based on comparing changes as assessed by an outcome measure with an external indicator; for example, the clinical judgment scores by the patient in the current study.42,43 We examined these two approaches in the responsiveness evaluation of the ISS. In the patients examined longitudinally, improvement of sensory disturbances (corresponding with a decrease in ISS value) was related significantly to an external criterion—the patients’ clinical judgment scores. Additionally, the SRM scores for the ISS, a statistical measure that assesses responsiveness, were equal to or more than the proposed value of 0.8, thus representing good responsiveness.37,38,42 The ISS also helped to determine whether clinical changes over time were predominantly sensory related. As demonstrated in figure 2, substantial ISS changes were noted during follow-up whereas strength remained relatively stable. This is important, because sensory disturbances may contribute to disability.5
With respect to the aims of the current study, some methodological issues should be addressed. First, the arbitrarily chosen normal values for two-point discrimination (grade, 0; ≤4 mm) were based primarily on the experiences of the expert panel and were not evaluated formally in terms of a possible age-related change in healthy individuals. Second, the obtained SRM scores for the ISS only demonstrated intragroup responsiveness. It is not clear whether substantial discriminatory responsiveness scores will be obtained for the ISS when evaluating various groups of patients (e.g., in a trial setting comparing a placebo group versus a treated group).44 Third, a significant correlation was demonstrated between the ISS and the disability scale. The use of the ISS may therefore be suggested as an indirect indicator of general recovery. However, this should be done with some caution, because we did not determine what portion of disability is due to sensory disturbances compared with, for example, muscle strength changes. Future studies are required to determine which impairment quality will have the strongest impact on disability in patients with sensory–motor immune-mediated polyneuropathies. Despite these limitations, the ISS was demonstrated to be a valuable instrument for assessing sensory deficit in patients with sensory–motor immune-mediated polyneuropathies.
Appendix
The members of the INCAT group are: J. Aubry, PhD; N. Baumann, PhD; P. Bouché, MD, PhD; G. Comi, MD, PhD; M.C. Dalakas, MD, PhD; H.P. Hartung, MD, PhD; R.A.C. Hughes, MD, PhD; I. Illa, MD, PhD; M.R.J. Knapp, PhD; J.-M. Léger, MD, PhD; R. Nemni, MD, PhD; E. Nobile-Orazio, MD, PhD; K.V. Toyka, MD, PhD; P. van de Bergh, MD, PhD; F.G.A. van der Meché, MD, PhD; P.A. van Doorn, MD, PhD; H.J. Willison, MD, PhD; and B. Younes-Chennoufi, PhD.
Acknowledgments
Acknowledgment
This study was part of the Biomed project (no. BMH4-CT96 0324). The authors thank Johnny Samijn, Rinske van Koningsveld, Ron Meijer, Wibe Moll, and Frank Opstelten for their participation in the reliability studies.
Footnotes
-
For a list of the members of the INCAT Group, please see the Appendix on page 948.
- Received August 13, 1999.
- Accepted October 1, 1999.
References
Letters: Rapid online correspondence
REQUIREMENTS
You must ensure that your Disclosures have been updated within the previous six months. Please go to our Submission Site to add or update your Disclosure information.
Your co-authors must send a completed Publishing Agreement Form to Neurology Staff (not necessary for the lead/corresponding author as the form below will suffice) before you upload your comment.
If you are responding to a comment that was written about an article you originally authored:
You (and co-authors) do not need to fill out forms or check disclosures as author forms are still valid
and apply to letter.
Submission specifications:
- Submissions must be < 200 words with < 5 references. Reference 1 must be the article on which you are commenting.
- Submissions should not have more than 5 authors. (Exception: original author replies can include all original authors of the article)
- Submit only on articles published within 6 months of issue date.
- Do not be redundant. Read any comments already posted on the article prior to submission.
- Submitted comments are subject to editing and editor review prior to posting.
