Pupillary diameter assessment: Need for a graded scale

Methods.

Four third- and fourth-year neurology residents consecutively evaluated the pupil size of 100 subjects (60% men; median age, 41.5 years) in randomized order. Subjects gave informed consent. Raters assessed pupil diameter twice with a 20-minute difference between trials. Room conditions including light were constant. Raters were unfamiliar with the subjects they examined. Raters 1 and 2 assessed pupil size using a graded scale; Raters 3 and 4 assessed pupil size subjectively. The scale, adapted from the French catheter scale (Cordis, Miami, FL), is graduated in 0.3-mm increments (from 1 to 11.3 mm) and has perforated circles rather than filled circles of different sizes. Raters positioned the scale on the lateral aspect of the patient’s eye, moving it to locate the circle on the scale corresponding to the pupil’s size. Raters completed a form recording each pupil size and whether there was anisocoria.

Interrater (e.g., assessing Rater 1 versus Rater 2) and intrarater (assessing the same rater’s ratings over Trials 1 and 2) reliability were measured with the intraclass correlation coefficient (ICC).5 Kappa measures were used to evaluate differences in raters’ assessment of anisocoria. To determine if there were differences between the raters’ objective and subjective assessments, a two-factor ANOVA—using as factors raters,1-4 pupil side (right [RP]/left [LP]), and trial (I/II)—was performed.

Results.

Using an objective scale, the raters reported larger median pupil sizes. At the first trial, the interrater reliability for the whole group was fair (ICC, 0.40 LP) to moderate (0.43 RP). Raters using the scale achieved substantial agreement, whereas those using their own impression had moderate agreement (table). There were significant differences between Raters 3 and 4, and between them and those using the scale. No differences were found between Raters 1 and 2.

At the second trial, the whole group achieved similar interrater reliability (0.47). Likewise, interrater agreement for raters who used the scale was better than for those using their impression (see the table).

Intrarater reliability was substantial to optimal (see the table). A two-factor ANOVA showed differences in pupil measurements between raters (p < 0.0001) but not between trials. Differences were between Raters 3 and 4 (p < 0.0002), and between them and Raters 1 and 2 (p < 0.0001), but not between Raters 1 and 2. There was no interaction between raters and trials.

Anisocoria.

Interrater reliability of the four raters was slight (0.18). Interrater reliability between raters using objective and subjective measures was also slight. There were no significant differences between trials. Raters using the scale reported anisocoria more often (23 versus 9 of 100 subjects).

Discussion.

Our study demonstrates that physicians using a scale achieved consistently better interrater reliability (substantial) than those using their own impression (moderate). This finding is particularly relevant when patients are assessed by different physicians in potentially evolving emergency situations. However, the method of assessment did not influence the intrarater reliability.

In the current study, each neurologist independently assessed a large number of subjects randomly assigned to assure sufficient statistical power. Our study design eliminated methodologic bias related to order of evaluation and lack of blindness, which limited the interpretation of previous studies.3,4

Our study, similar to previous ones,6,7 showed a low interrater reliability of neurologists in assessing anisocoria. This finding should be carefully interpreted because kappa is a measure that considers agreement beyond chance. The study group consisted of 100 normal subjects, most without anisocoria, and when the number of events is low, the chance of agreement increases. In fact, raters’ percentage of agreement (91%) and agreement by chance were high (87%). It could be argued that the lack of an a priori definition of anisocoria contributes to a low interrater agreement.

Based on our findings, a graded scale should be used when more than one physician assesses pupillary diameter. However, when the same physician evaluates the pupils, the subjective method may still be reliable, if one excludes potential emergency evolving neurologic situations. Whether an a priori working definition of anisocoria will improve interrater reliability needs to be determined.