The utility of MRI in suspected MS: Report of the Therapeutics and Technology Assessment Subc.
B.M. J.Uitdehaag, VU Medical Centre, Amsterdam, bmj.uitdehaag@vumc.nl
J.J.G. Geuerts, F. Barkhof, C.H. Polman
Submitted April 20, 2004
We read with interest the Report of the Therapeutics and
Technology Assessment Subcommittee of the AAN.[1] The Subcommittee
concludes that in
patients with a typical clinically isolated syndrome the finding of even a
few (three, perhaps even one) white matter lesions on a T2-weighted MRI
scan is a more sensitive predictor of the subsequent development of
clinically definite MS (CDMS) than the fulfillment of more stringent
criteria as recommended by the International Panel on the diagnosis of
MS. [2] However, it is also suggested that this increased sensitivity
can be achieved without sacrificing specificity, a statement that we
consider incorrect.
In the studies they reviewed, several biases can be
identified [3] including those due to patient selection like referral
bias, patient filtering
bias and spectrum bias. Although these biases do not necessarily affect
the
internal validity of a study, they limit the clinical applicability which
is crucial for the purpose of the recommendations.
Another bias is induced by the large number of patients lost to follow up.
Since the gold standard is the diagnosis of CDMS verified during follow
up, loss to follow up is a source of verification bias. Since sensitivity
and specificity estimates are not valid when verification bias is present,
it has been advocated to perform a sensitivity analysis in these
situations. [4]
Even more bias occurs when patients are excluded because of
another
diagnosis made during follow up. The Report correctly states that in our
Centre, 10 patients from our original cohort were excluded because they
were subsequently diagnosed as having a disease other than MS. [5] We
present further data on those patients. These patients were referred to
our Centre when MRI was not yet widely available in the Netherlands and
clinicians specifically ordered MRI in less typical cases. The diagnoses
subsequently made in these 10 patients from a cohort of 59 (16.9%) were:
ischemic vascular disease in three, neoplasms in two, Lyme disease in two,
acute
disseminated encephalomyelitis in one, cervical stenosis in one, and
ocular
disease in one patient. Of these patients, eight had at least one
cerebral white matter lesion, two at least three cerebral white matter
lesions,
and one (subsequently diagnosed as Lyme disease) five white matter lesions
including three in the periventricular region. By contrast, none of these
patients
fulfilled the criteria for dissemination in space as recommended by the
International Panel. [2]
We agree with the Subcommittee that alternative diagnoses
should
be excluded, but this can be difficult at first presentation (table 2) of
the Report which lists diagnostic alternatives,
including ‘unidentified bright objects’. Specificity will therefore
always be an issue with respect to MRI criteria for MS, especially when
early initiation of treatment is considered. It is important to realize
that increasing sensitivity will always be accompanied by decreasing
specificity. Future research on this topic is urgently needed. The
additional data we present here strongly suggest that the conclusions of
the AAN Report should be interpreted with great caution.
References
1. Frohman EM, Goodin DS, Calabresi PA, et al. The utility
of MRI in suspected MS. Neurology 2003;61:602-611.
2. McDonald WI, Compson A., Edan G, et al. Recommended
diagnostic criteria for multiple sclerosis: Guidelines from the
international panel on the diagnosis of multiple sclerosis. Ann Neurol
2001;50:121-127.
3. Kelly S, Berry E, Roderick P, et al. The identification
of bias in studies of the diagnostic performance of imaging modalities. Br
J Radiol 1997;70:1028-1024.
4. Kosinski A, Barnhart H. A global sensitivity analysis of
performance of a medical diagnostic test when verification bias is
present. Statist Med 2003;22:2711-2721.
5. Barkhof F, Filippi M, Miller DH, et al. Comparison of MR
imaging criteria at first presentation to predict conversion to clinically
definite multiple sclerosis. Brain 1997;120:2059-2069.
We read with interest the Report of the Therapeutics and Technology Assessment Subcommittee of the AAN.[1] The Subcommittee concludes that in patients with a typical clinically isolated syndrome the finding of even a few (three, perhaps even one) white matter lesions on a T2-weighted MRI scan is a more sensitive predictor of the subsequent development of clinically definite MS (CDMS) than the fulfillment of more stringent criteria as recommended by the International Panel on the diagnosis of MS. [2] However, it is also suggested that this increased sensitivity can be achieved without sacrificing specificity, a statement that we consider incorrect.
In the studies they reviewed, several biases can be identified [3] including those due to patient selection like referral bias, patient filtering bias and spectrum bias. Although these biases do not necessarily affect the internal validity of a study, they limit the clinical applicability which is crucial for the purpose of the recommendations. Another bias is induced by the large number of patients lost to follow up. Since the gold standard is the diagnosis of CDMS verified during follow up, loss to follow up is a source of verification bias. Since sensitivity and specificity estimates are not valid when verification bias is present, it has been advocated to perform a sensitivity analysis in these situations. [4]
Even more bias occurs when patients are excluded because of another diagnosis made during follow up. The Report correctly states that in our Centre, 10 patients from our original cohort were excluded because they were subsequently diagnosed as having a disease other than MS. [5] We present further data on those patients. These patients were referred to our Centre when MRI was not yet widely available in the Netherlands and clinicians specifically ordered MRI in less typical cases. The diagnoses subsequently made in these 10 patients from a cohort of 59 (16.9%) were: ischemic vascular disease in three, neoplasms in two, Lyme disease in two, acute disseminated encephalomyelitis in one, cervical stenosis in one, and ocular disease in one patient. Of these patients, eight had at least one cerebral white matter lesion, two at least three cerebral white matter lesions, and one (subsequently diagnosed as Lyme disease) five white matter lesions including three in the periventricular region. By contrast, none of these patients fulfilled the criteria for dissemination in space as recommended by the International Panel. [2]
We agree with the Subcommittee that alternative diagnoses should be excluded, but this can be difficult at first presentation (table 2) of the Report which lists diagnostic alternatives, including ‘unidentified bright objects’. Specificity will therefore always be an issue with respect to MRI criteria for MS, especially when early initiation of treatment is considered. It is important to realize that increasing sensitivity will always be accompanied by decreasing specificity. Future research on this topic is urgently needed. The additional data we present here strongly suggest that the conclusions of the AAN Report should be interpreted with great caution.
References
1. Frohman EM, Goodin DS, Calabresi PA, et al. The utility of MRI in suspected MS. Neurology 2003;61:602-611.
2. McDonald WI, Compson A., Edan G, et al. Recommended diagnostic criteria for multiple sclerosis: Guidelines from the international panel on the diagnosis of multiple sclerosis. Ann Neurol 2001;50:121-127.
3. Kelly S, Berry E, Roderick P, et al. The identification of bias in studies of the diagnostic performance of imaging modalities. Br J Radiol 1997;70:1028-1024.
4. Kosinski A, Barnhart H. A global sensitivity analysis of performance of a medical diagnostic test when verification bias is present. Statist Med 2003;22:2711-2721.
5. Barkhof F, Filippi M, Miller DH, et al. Comparison of MR imaging criteria at first presentation to predict conversion to clinically definite multiple sclerosis. Brain 1997;120:2059-2069.