MRI as an outcome in multiple sclerosis clinical trials
Maria PiaSormani, Biostatistics Unit, Department of Health Sciences, University of Genoa, Genoa, Italym.filippi@hsr.it
Massimo Filippi (Neuroimaging Research Unit, Department of Neurology, San Raffaele Scientific Institute, Milan, Italy) and Nicola De Stefano (Quantitative Neuroimaging Lab, Dept. of Neurological & Behavioral Sciences, University of Siena, Siena, Italy)
Submitted May 26, 2009
We read with interest the retrospective study of Daumer et al. who analyzed the placebo arms of several randomized clinical trials (RCTs) to establish the relationship of T2-weighted and gadolinium-enhanced T1- weighted MRI lesions with clinical endpoints. [1]
The results showed no correlation between clinical and MRI markers and were interpreted as an indication that MRI adds little, if anything, to clinical measures. If confirmed, this should lead us to reconsider the use of MRI measures as surrogates of clinical endpoints in MS clinical trials. However, we believe that the study has important methodological flaws, which limit the validity of the results and the consequent conclusions.
The concept of clinical surrogacy should always be related to a treatment effect. Indeed, the Prentice criteria have been developed to validate surrogate markers in the context of individual RCTs or meta-analysis of several complete RCTs. The clinical/MRI correlation performed by Daumer et al. using only placebo arms of different trials does not meet an established validation procedure and might be misleading. In contrast, a previous analysis of complete data sets of RCTs revealed that a clinical/MRI correlation does exist providing preliminary evidence of the validity of MRI surrogates. [2-3] Another recent study confirmed this by demonstrating that the reduction in MRI lesions across different treatment trials correlated with the effect of reducing relapses. [4]
Using the same database, it was shown that the main source of variability of MRI lesion counts in placebo arms of different RCTs is the MRI center where data were analyzed. [5] Therefore, reporting the correlation of MRI lesions and clinical endpoints using individual data from different trials necessarily dilutes the correlation. In a highly variable disease such as MS, the low individual level of correlation between MRI markers and clinical variables is likely due to the high variability of both measures rather than to the lack of a biologically meaningful relationship.
The well documented capability of the brain to reorganize after tissue injury makes this relationship even more difficult to prove with the authors’ simplified approach. In addition, the known limitations of any retrospective analysis based on data from old and heterogeneously evaluated trials should prompt a more cautious interpretation of the results.
References
1. Daumer M, Neuhaus A, Morrissey S, Hintzen R, Ebers GC. MRI as an outcome in multiple sclerosis clinical trials. Neurology 2008 (e-pub ahead of print).
2. Sormani MP, Bruzzi P, Comi G, Filippi M. MRI metrics as surrogate markers for clinical relapse rate in relapsing-remitting MS patients. Neurology 2002;58:417-421.
3. Sormani MP, Bruzzi P, Beckmann K, et al. MRI metrics as surrogate endpoints for EDSS progression in SPMS patients treated with IFN beta-1b. Neurology 2003;60:1462-1466.
4. Sormani MP, Bonzano L, Roccatagliata L, Cutter GR, Mancardi GL, Bruzzi P. Magnetic resonance imaging as a potential surrogate for relapses in multiple sclerosis: a meta-analytic approach. Ann Neurol 2009;65:268-275.
5. Schach S, Scholz M, Wolinsky JS, Kappos L. Pooled historical MRI data as a basis for research in multiple sclerosis. A statistical evaluation. Mult Scler 2007;13:509-516.
*The MAGNIMS Steering Commettee Chairs: F Barkhof (Amsterdam, The Netherlands) and X Montalban (Barcelona, Spain) Members: F Fazekas (Graz, Austria), J Frederiksen (Copenhagen, Denmark), L Kappos (Basel, Switzerland), DH Miller (London, United Kingdom), J Palace (Oxford, United Kingdom), CH Polman (Amsterdam, The Netherlands), MA Rocca (Milan, Italy), M Rovaris (Milan, Italy), A Rovira (Barcelona, Spain), T Yousry (London, United Kingdom)
Disclosures: Dr. M. Filippi has received honoraria for lectures and travel expenses, and consulting fees as an investigator in previous and current treatment trials from Teva, Merck-Serono, Bayer-Schering, Biogen-Dompè, and Genmab. Dr. Filippi reports 100% effort of his clinical practice on MRI procedures and has received fees from Teva, Merck-Serono, Bayer-Schering, Biogen-Dompè, and Genmab for consultation regarding performing MRIs. Dr M. Pia Sormani received honoraria for lectures and travel expenses, and consulting fees as Teva, Merck-Serono, Biogen-Idec, Actelion. Dr. N. De Stefano has received honoraria for lectures and travel expenses, and consulting fees as an investigator in previous and current treatment trials from Teva, Merck-Serono, Bayer-Schering and Biogen-Dompè.
The results showed no correlation between clinical and MRI markers and were interpreted as an indication that MRI adds little, if anything, to clinical measures. If confirmed, this should lead us to reconsider the use of MRI measures as surrogates of clinical endpoints in MS clinical trials. However, we believe that the study has important methodological flaws, which limit the validity of the results and the consequent conclusions.
The concept of clinical surrogacy should always be related to a treatment effect. Indeed, the Prentice criteria have been developed to validate surrogate markers in the context of individual RCTs or meta-analysis of several complete RCTs. The clinical/MRI correlation performed by Daumer et al. using only placebo arms of different trials does not meet an established validation procedure and might be misleading. In contrast, a previous analysis of complete data sets of RCTs revealed that a clinical/MRI correlation does exist providing preliminary evidence of the validity of MRI surrogates. [2-3] Another recent study confirmed this by demonstrating that the reduction in MRI lesions across different treatment trials correlated with the effect of reducing relapses. [4]
Using the same database, it was shown that the main source of variability of MRI lesion counts in placebo arms of different RCTs is the MRI center where data were analyzed. [5] Therefore, reporting the correlation of MRI lesions and clinical endpoints using individual data from different trials necessarily dilutes the correlation. In a highly variable disease such as MS, the low individual level of correlation between MRI markers and clinical variables is likely due to the high variability of both measures rather than to the lack of a biologically meaningful relationship.
The well documented capability of the brain to reorganize after tissue injury makes this relationship even more difficult to prove with the authors’ simplified approach. In addition, the known limitations of any retrospective analysis based on data from old and heterogeneously evaluated trials should prompt a more cautious interpretation of the results.
References
1. Daumer M, Neuhaus A, Morrissey S, Hintzen R, Ebers GC. MRI as an outcome in multiple sclerosis clinical trials. Neurology 2008 (e-pub ahead of print).
2. Sormani MP, Bruzzi P, Comi G, Filippi M. MRI metrics as surrogate markers for clinical relapse rate in relapsing-remitting MS patients. Neurology 2002;58:417-421.
3. Sormani MP, Bruzzi P, Beckmann K, et al. MRI metrics as surrogate endpoints for EDSS progression in SPMS patients treated with IFN beta-1b. Neurology 2003;60:1462-1466.
4. Sormani MP, Bonzano L, Roccatagliata L, Cutter GR, Mancardi GL, Bruzzi P. Magnetic resonance imaging as a potential surrogate for relapses in multiple sclerosis: a meta-analytic approach. Ann Neurol 2009;65:268-275.
5. Schach S, Scholz M, Wolinsky JS, Kappos L. Pooled historical MRI data as a basis for research in multiple sclerosis. A statistical evaluation. Mult Scler 2007;13:509-516.
*The MAGNIMS Steering Commettee Chairs: F Barkhof (Amsterdam, The Netherlands) and X Montalban (Barcelona, Spain) Members: F Fazekas (Graz, Austria), J Frederiksen (Copenhagen, Denmark), L Kappos (Basel, Switzerland), DH Miller (London, United Kingdom), J Palace (Oxford, United Kingdom), CH Polman (Amsterdam, The Netherlands), MA Rocca (Milan, Italy), M Rovaris (Milan, Italy), A Rovira (Barcelona, Spain), T Yousry (London, United Kingdom)
Disclosures: Dr. M. Filippi has received honoraria for lectures and travel expenses, and consulting fees as an investigator in previous and current treatment trials from Teva, Merck-Serono, Bayer-Schering, Biogen-Dompè, and Genmab. Dr. Filippi reports 100% effort of his clinical practice on MRI procedures and has received fees from Teva, Merck-Serono, Bayer-Schering, Biogen-Dompè, and Genmab for consultation regarding performing MRIs. Dr M. Pia Sormani received honoraria for lectures and travel expenses, and consulting fees as Teva, Merck-Serono, Biogen-Idec, Actelion. Dr. N. De Stefano has received honoraria for lectures and travel expenses, and consulting fees as an investigator in previous and current treatment trials from Teva, Merck-Serono, Bayer-Schering and Biogen-Dompè.