Statistical approaches to assessing the effects of neutralizing antibodies

In this presentation, we discussed this question in the context of the pivotal trial of interferon-β (IFNβ-1b) in patients with relapsing-remitting multiple sclerosis (RRMS).1–3⇓⇓ In this trial, 372 patients were randomized to placebo, low-dose, or high-dose treatment arms. Patients were followed for at least 3 years, with some patients undergoing blind follow-up evaluation for as long as 5 years. Serum samples were obtained quarterly, usually 12 or 36 hours after the most recent injection, and two assays were subsequently used for antibody measurement. The first, a viral cytopathic effect reduction assay (CPE), was carried out only on the serum samples collected for the first 3 years. Later, a myxovirus A (MxA) assay was developed and used on all serum samples available to the end of the blind follow-up period of up to 5 years.

Objectives of the Statistical Analysis.

We first discussed the key issue of appropriate statistical approaches to assess the effects of neutralizing antibodies (NAbs) on treatment efficacy. To address this, one first must decide what question is to be answered by the statistical analysis. Two possibilities are:

1. How do the responses of the NAb+ subgroup and the NAb− subgroup compare?

2. Is the switch from NAb− to NAb+ status associated with deterioration of responses in individual patients?

If the objective is to answer the first question, then cross-sectional analyses that compare the responses of NAb+ and NAb− patients at fixed times (e.g., after 2 years on study) suffice. But if the second question is to be answered, then analyses that compare the responses of patients during periods when they are NAb− and periods when they are NAb+ are required. Such longitudinal analyses require a data set with responses of patients recorded repeatedly during the study period.

Cross-sectional comparisons of the relapse rates (and other responses) for the NAb+ and NAb− subgroups of the low-dose and high-dose treatment arms in the pivotal trial of IFNβ-1b in patients with RRMS have been reported.4 In this report, NAb status was based on the CPE assay, with the time of switching from NAb− to NAb+ status taken as the time of the first positive (at least 20 NU/mL) serum titer for patients having two consecutive positive titers at some point during follow-up evaluation. With this definition, 48 of the 125 patients (38%) on the low-dose arm and 43 of the 124 patients (35%) on the high-dose arm switched from NAb− to NAb+ status. Although more than 60% of these patients subsequently had at least one negative serum titer, a “once NAb+, always NAb+” convention was adopted for the analysis: a patient who switched from NAb− to NAb+ status was considered to remain NAb+ to the end of their follow-up period, irrespective of his or her later serum titers.

For each treatment arm, the relapse rates of NAb+ and NAb− patients were compared within successive 6-month periods (table 2 in reference 4). In these comparisons, the NAb+ subgroup included all patients who became NAb+ before the end of the period. For the high-dose treatment arm, the relapse rates of the NAb+ patients were significantly (p < 0.05) higher than those of the NAb− patients in the 19- to 24-month and the 25- to 30-month periods. Further, in the third year of follow-up evaluation, the relapse rates of these NAb+ patients were comparable with those of the patients receiving placebo. In contrast, for the low-dose treatment arm, the relapse rates of the NAb+ patients were generally lower than those of the NAb− patients in the first 2 years of follow-up evaluation, and the difference was statistically significant in the 19- to 24-month period.

These cross-sectional comparisons are complicated by the changing composition of the NAb+ and NAb− subgroups over time: as patients switch from NAb− to NAb+ status, they switch membership from the NAb− subgroup to the NAb+ subgroup. A different approach, used in most other reports on the effects of NAbs, focuses on the “eventually NAb+” subgroup of patients: those patients who achieve NAb+ status at some point during the trial. The remaining patients make up the “never NAb+” subgroup. Comparisons of the relapse rates of the eventually NAb+ and the never NAb+ subgroups on the low-dose and high-dose treatment arms of the pivotal trial of IFNβ-1b in patients with RRMS led to results that are qualitatively similar to those reported.4

Limitations of Cross-Sectional Analyses.

Some interpret these results for the high-dose treatment arm in the pivotal trial of IFNβ-1b in patients with RRMS (and the results of similar cross-sectional comparisons from other studies) as convincing evidence that the presence of NAbs decreases the efficacy of treatments. Setting aside the issue of how the contrary result observed in the low-dose treatment arm of the pivotal trial of IFNβ-1b in patients with RRMS might be explained, we discussed two serious limitations of such cross-sectional comparisons of the eventually NAb+ and never NAb+ subgroups:

1. A lack of statistical power of such subgroup comparisons because of the reduced sample sizes involved, and

2. The cause of any difference detected by such comparisons is unclear because these subgroups may differ on baseline covariates (possibly unmeasured) predictive of NAb development and reduced efficacy.

It is easy to evaluate the power of such subgroup comparisons for a two-armed clinical trial with n patients randomly assigned to each arm. Suppose the trial was designed based on an endpoint analysis of the means of a continuous primary outcome, with type I error of 5% and specified power. Suppose further that the endpoint responses of the eventually NAb+ patients have the statistical behavior postulated for the patients receiving placebo when the trial was designed and the never NAb+ patients behave as was postulated for the treatment patients. The power of the comparison of the endpoint values of the eventually NAb+ and never NAb+ subgroups can then be expressed in terms of the power specified for the clinical trial and the proportion of patients on the treatment arm who belong to the eventually NAb+ subgroup. The resulting power is illustrated in the figure for various values of these inputs. For example, if the power specified for the clinical trial was 80% and the proportion of eventually NAb+ patients on the treatment arm is 40% (slightly more than in the pivotal trial of IFNβ-1b in patients with RRMS), then the power of this comparison of the eventually NAb+ and never NAb+ subgroups is only 28%.

• Download figure
• Open in new tab
• Download powerpoint

Figure. Power of cross-sectional comparisons of the eventurally NAb+ and never NAb+ subgroups in a two-armed clinical trial.

The second limitation is more fundamental. Because no randomization or other device of experimental design has been used to ensure that the eventually NAb+ and never NAb+ subgroups are comparable in all other respects, it is not possible to conclude that an observed difference is because the eventually NAb+ patients developed NAbs whereas the never NAb+ patients did not. The difference could be the result of some other feature, possibly unmeasured, that differs systematically between these two subgroups. Such cross-sectional comparisons correspond to an observational study of the eventually NAb+ and never NAb+ subgroups and are susceptible to the influence of such confounding factors.

Longitudinal Data Analyses.

These limitations indicate a clear need for other statistical approaches to the analysis of the data. We then described an alternative approach based on longitudinal data analysis methods where the influence of NAbs in patients is assessed by comparing their responses during periods when they are NAb+ with periods when they are NAb−. Such an approach makes more complete use of the available longitudinal data and is intended to more clearly identify the impact of NAb status on the therapeutic efficacy of treatments because it directly addresses the question of whether the switch from NAb− to NAb+ status is associated with deterioration of responses in patients.

Any longitudinal data analysis requires repeated observations of the responses of patients over time, and the more frequently, the better. In the MS context, the approach is ideally suited for relapse data because effectively daily observations are available. However, it is less well suited for responses that are observed only infrequently in patients, such as annual MRI or “time to event” responses.

Finally, we described the results obtained using this approach when we analyzed the relapse rates, Expanded Disability Status Scale scores, and T2 lesion burdens for patients who changed from NAb− to NAb+ status during the pivotal trial of IFNβ-1b in patients with RRMS. The generalized estimating equations approach to longitudinal data analysis5,6⇓ was used to account for the correlation in the repeated responses over time in patients. Separate analyses were carried out for NAb status as identified by the CPE and MxA assays. The analyses adjust for possible time trends in the responses during the course of the trial and were carried out separately for each of the treatment arms. All the analyses were limited to the eventually NAb+ subgroups of patients.

In addition to assessing the effect of the switch from NAb− to NAb+ status, for the relapse data, we also presented the results of analyses examining therelative importance of high and low titers. The results of all of these analyses will be reported elsewhere (Petkau et al., submitted for publication, 2003), so details are not described here. The same longitudinal data analysis approach has been used to address this issue for data from the European trial of IFNβ-1b in patients with secondary progressive MS.7