Abstract
Objective: To compare contrast-enhanced MR angiography (CEMRA) with intra-arterial digital subtraction angiography (DSA) for evaluating carotid stenosis.
Methods: A total of 167 consecutive symptomatic patients, scheduled for DSA following screening duplex ultrasound (DUS), were prospectively recruited to have CEMRA. Three independent readers reported on each examination in a blinded and random manner. Agreement was assessed using the Bland-Altman method. Diagnostic and potential clinical impact of CEMRA was evaluated, singly and in combination with DUS.
Results: CEMRA tended to overestimate stenosis by a mean bias ranging from 2.4 to 3.8%. A significant part of the disagreement between CEMRA and DSA was directly caused by interobserver variability. For detection of severe stenosis, CEMRA alone had a sensitivity of 93.0% and specificity of 80.6%, with a diagnostic misclassification rate of 15.0% (n = 30). More importantly, clinical decision-making would, however, have been potentially altered only in 6.0% of cases (n = 12). The combination of concordant DUS and CEMRA reduced diagnostic misclassification rate to 10.1% (n = 19) at the expense of 47 (24.9%) discordant cases needing to proceed to DSA. An intermediate approach of selective DUS review resulted in a marginally worse diagnostic misclassification rate of 11.6% (n = 22) but with only 6.8% of discordant cases (n = 13).
Conclusions: DSA remains the gold standard for carotid imaging. The clinical misclassification rate with CEMRA, however, is acceptably low to support its safe use instead of DSA. The appropriateness of combination strategies depends on institutional choice and cost-effectiveness issues.
Several pivotal studies have clearly demonstrated the benefits of carotid endarterectomy in recently symptomatic patients with severe internal carotid artery (ICA) stenosis.1,2⇓ Intra-arterial digital subtraction angiography (DSA), the gold standard method used in these trials, is still perceived as the most accurate method for assessing the degree of ICA stenosis. DSA is, however, a relatively expensive procedure and carries a small but significant risk of causing a stroke or death, which could reduce the overall benefits of surgery.3 These concerns have generated substantial interest in the use of alternative noninvasive imaging modalities.
Whereas duplex ultrasound (DUS) is very accurate in experienced hands, its limitations, such as high interobserver variability, interhospital variations, and artifacts due to plaque calcifications, have also been well described.4 Its low false negative rate, however, makes it an ideal screening tool prior to a confirmatory test such as DSA, MR angiography (MRA), or CT angiography (CTA). Despite an abundance of literature, there is no consensus on whether noninvasive techniques can adequately and safely replace DSA and this is reflected in the wide variety of practices existing today.5 Thus, while recent surveys suggest that there is a definite trend toward noninvasive modalities, with the combination of screening DUS and MRA emerging as the preferred strategy, up to 50% of centers still consider DSA to be necessary.6,7⇓ Part of the problem may reside in the poor methodologic criteria used by many studies evaluating noninvasive modalities, with the consequence that clinicians are often confused by a wide range of reported diagnostic performance that ranges from disturbingly poor to perfect.8,9⇓
Contrast-enhanced MR angiography (CEMRA) has recently emerged as a significant technical improvement over traditional time-of-flight (TOF) MRA techniques. The use of a paramagnetic agent to improve vascular contrast results in higher quality images, which are less susceptible to motion or dephasing artifacts.10 This technical superiority increases the clinician’s confidence and has generated widespread enthusiasm for the modality but it remains to be proven whether this translates into better diagnostic performance.11 Current evidence supporting the use of CEMRA is largely promising, although solely based on a number of relatively small series, none of which were adequately powered to detect differences with clinically useful precision.12-17⇓⇓⇓⇓⇓ We sought to comprehensively evaluate the diagnostic performance and potential therapeutic impact of CEMRA, with DSA as the reference standard.
Methods.
Participants.
Between August 2000 and January 2003, 167 consecutive symptomatic patients suspected of having ICA stenosis, and who were scheduled for DSA, were prospectively recruited at a single academic institution in the United Kingdom. Patients underwent CEMRA within a maximum of 3 weeks of their DSA examination. All patients had experienced symptoms of ICA disease (transient ischemic attacks or strokes) and were considered suitable candidates for carotid endarterectomy. Other inclusion criteria were age 40 to 90 years and screening DUS showing at least 50% stenosis in one or both arteries. We excluded patients with contraindications for MRI such as severe claustrophobia or metallic implants. Our study was approved by the local ethics research committee and all patients gave informed consent. We adhered to the Standards for Reporting of Diagnostic Accuracy (STARD) criteria for design and presentation of diagnostic studies.18
Diagnostic tests.
DSA, the reference standard, was performed by one of three attending neuroradiologists, on a digital angiographic unit (Angioskop; Siemens, Erlangen). After femoral puncture, each common carotid artery was selectively catheterized in turn and injected with a total of 100 to 150 mL of contrast agent (Niopam 300; Merck, Alton, Hampshire). Images of each bifurcation were acquired on four projections (anteroposterior, lateral and bilateral 45° obliques) with a 33-cm field of view, 1024 × 1024 matrix, and resolution of 0.32 × 0.32 mm, and recorded on hard copy films.
DUS was usually performed by four experienced technologists (with two having 4 and 23 years of experience and the other two with 6 years) at our own institution with a combination of B-mode and pulse-wave Doppler with a high-frequency linear array-probe (5 to 10 MHz) (Powervision 7000, Toshiba America Medical Systems, Tustin, CA). Estimates of the degree of stenosis were made using spectral identification and velocity criteria in the common carotid artery and ICA, with peak systolic velocities of 120 cm/second as the determinant for 50% stenosis or greater and 200 cm/second as the determinant for severe (70 to 99%) stenosis.19
CEMRA was performed on a 1.5-T machine (Signa CV/i, GE Medical Systems, Milwaukee, WI) with the use of a transmit-receive neurovascular coil (Medrad, Indianola, PA). CEMRA was performed in the coronal plane with a three-dimensional fast spoiled gradient echo acquisition (Fractional echo time = 1.6 msec, repetition time = 5.3 msec, flip angle = 44°, bandwidth = 62.5 kHz, field of view = 23 × 16 cm, slice thickness = 0.8 mm, matrix = 256 × 256, number of excitations = 0.5). An elliptic-centric phase-encode ordering scheme was used.20 Typical imaging time was 36 seconds during which patients were asked to hold their breath. A power injector (Spectris, Medrad) was used for the IV injection of a 28 mL dose of a gadolinium-based contrast agent (Magnevist, Schering, Berlin) at a flow rate of 2 mL/second, which was followed by 60 mL of saline solution. Zero-filling interpolation was used resulting in a final interpolated voxel size of 0.45 × 0.45 × 0.40 mm. Imaging delay was determined by performing a test-bolus with 2 mL of contrast agent.
Each CEMRA data set was transferred to a dedicated workstation (Advantage 4.0, GE Medical Systems) for processing using a maximum intensity projection (MIP) algorithm. After removal of the vertebral arteries, each carotid bifurcation was recorded onto film as four projections identical to the ones used for DSA.
Image analysis.
Three experienced neuroradiologists, blinded to clinical history and results of other diagnostic tests, independently reviewed each MR and conventional angiogram. Patients’ identifiers on films were masked. The DSA images were arranged in a randomized order and reported over a period of 4 months. The CEMRA images were then subsequently reported over the following 4 months, after re-randomization of the order.
The percentage of ICA stenosis was determined, using a vernier caliper calibrated to 0.01 mm (UKAS calibration, Northants, UK), according to the North American Symptomatic Carotid Endarterectomy Trial (NASCET) method.2 Lumen diameters at the site of maximal stenosis and at the normal distal internal carotid artery (beyond the bulb and where the vessel walls become parallel) were measured on the projection that demonstrated the tightest stenosis. When an ICA demonstrated a near occlusion with collapse of the distal lumen, the degree of stenosis was assigned as 95%.21 For CEMRA, the presence of a signal void or flow gap was assumed to represent a severe stenosis (70 to 99%) based on published evidence.22 The gold standard measurement of stenosis, used as reference standard for all comparisons between CEMRA and DSA, was defined as the average of the independent stenosis measurements by the three readers on DSA. No analysis of CEMRA source data was performed in the context of this study.
Image quality of DSA and CEMRA were evaluated for overall quality including vascular signal intensity, venous suppression, and presence of artifacts. Evaluation criteria for overall quality were 1, excellent; 2, more than adequate for diagnosis; 3, adequate for diagnosis; and 4, inadequate for diagnosis. Both DSA and CEMRA images were also evaluated for the presence of plaque ulceration. A plaque was defined as ulcerated if it fulfilled the radiographic criteria of an ulcer niche, seen in profile as a crater penetrating into the plaque and double density on an en face view.23 The presence of these findings was defined as agreement of at least two readers.
Statistical methods.
The patient group was prescreened by DUS and thus has a high pretest prevalence of moderate to severe carotid stenosis. Agreement between CEMRA and DSA was assessed using the Bland-Altman approach, where clinically acceptable limits of agreement were defined as within ±6% difference in the stenosis measurements between the two modalities.24 These limits were arbitrarily chosen because typical errors for different observers to read DSA itself have been shown to be in the order of ±5% and it is therefore unrealistic to expect two different modalities to agree better than this.25 Assuming no systematic bias and the SD of the difference in stenosis measurements between CEMRA and DSA to be 2.5%, we calculated that a sample size of 196 arteries will have 90% power, at a 5% significance level, to reject the null hypothesis that the two modalities agree poorly (i.e., that either or both calculated limits of agreement fall outside the ±6% limits of clinically acceptable agreement).
The diagnostic accuracy of CEMRA for detection of severe (70 to 99%) stenosis was determined by calculating sensitivity, specificity, and predictive values. Assuming the expected sensitivity of CEMRA for detection of severe stenosis (estimated prevalence of 40% among the study population) to be 85%, we calculated that a sample size of 80 arteries (200 arteries × 0.40) with severe disease would give a one-sided 95% CI for the sensitivity, using the large sample normal approximation, that would extend 6.6% from the calculated/observed sensitivity. For an expected specificity of 85%, a sample size of 120 arteries that would not be recommended for surgery would yield a one-sided 95% CI for the specificity that would extend 5.4% from the calculated/observed specificity.
Diagnostic misclassification rate was defined at the percentage of arteries in which CEMRA and DSA disagreed regarding classification into the severe category. The decision of whether to perform carotid endarterectomy was made in the clinical setting solely on the basis of DSA. All cases of diagnostic misclassification were then individually reviewed by the first author (J.M.U.-K.-I.) from our prospectively maintained database to assess how the use of CEMRA instead of DSA would have affected clinical outcome, i.e., whether the decision to proceed to surgery would have been altered. This process is described in further detail in Results. The percentage of cases resulting in altered outcome was defined as the clinical misclassification rate.
Interobserver agreement for CEMRA and DSA was evaluated by using both the Bland-Altman approach and Cohen kappa and weighted kappa statistics for classification into mild (0 to 49%), moderate (50 to 69%), or severe (70 to 99%) stenosis and occlusion.
Finally, DUS alone and two combination strategies involving DUS and CEMRA results were also assessed in terms of diagnostic accuracy and misclassification rates.
Results.
Participants.
The baseline characteristics and relevant medical history of the 167 patients recruited in the study are listed in table 1.
Table 1. Baseline characteristics of the study population
CEMRA and DSA examinations were performed on the same day in 138 patients (82%) and within 3 weeks in the remainder. We adopted an intention-to-image approach and images of poor quality were not excluded. Twenty-five patients were, however, excluded from the final analysis. Sixteen patients failed to complete the CEMRA examination because of the following reasons: claustrophobia (n = 4), metallic contraindications (n = 3), equipment or other technical failure (n = 5), and patient withdrawal from study (n = 4). Three patients did not complete the DSA examination because of the following reasons: patient withdrawal (n = 1) and unsuccessful catheterization of the common carotid artery (n = 2). Finally, six patients were excluded because they had been recruited despite not satisfying the inclusion criteria of having screening DUS performed and subsequently were found to have normal carotid arteries on both CEMRA and DSA. This is summarized in the STARD patient flow diagram (figure 1).
Figure 1. Standards for Reporting of Diagnostic Accuracy flow diagram. Arteries are only included in the diagram if they satisfy the inclusion criteria of 50% stenosis or greater. DUS = duplex ultrasound; CEMRA = contrast-enhanced MR angiography; DSA = digital subtraction angiography.
After exclusion of these patients, 142 patients, who completed both CEMRA and DSA, remained, yielding 200 abnormal arteries to meet our sample size requirements. An abnormal artery was defined as having at least 50% stenosis on either screening DUS or DSA. Normal or near-normal arteries were not included in the analysis because they do not adequately reflect a prescreened population, which is the main focus of this study.26 Overall image quality of CEMRA was as follows: 100 excellent, 72 more than adequate for diagnosis, and 28 adequate for diagnosis. Image quality in 10 of the adequate for diagnosis studies was probably related to the patient’s difficulty with breath-holding. Overall image quality of DSA was as follows: 140 excellent, 54 more than adequate for diagnosis, and 6 adequate for diagnosis. Four of these latter six cases were clearly due to motion artifacts in frail elderly patients.
CEMRA.
Figure 2 shows Bland-Altman plots of agreement for CEMRA against the gold standard, the average DSA stenosis measurement for the three independent readers. CEMRA tended to overestimate stenosis by a mean bias ranging from 2.4 to 3.8%. Limits of agreement, which reflect the range in which 95% of differences between the two modalities are expected to lie, were wide, ranging from ±21.0% to ±23.8% about the mean bias, suggesting that significant disagreement may occur between the two methods. Interobserver agreement plots for DSA are shown in online figure E-1 and for CEMRA in online figure E-2 (see www.neurology.org). Again, substantial differences between different readers occurred, with limits of agreement about the mean bias ranging from ±12.6% to ±14.9% for DSA itself and from ±13.9% to ±17.1% for CEMRA.
Figure 2. Bland-Altman plots of agreement (difference versus mean) for contrast-enhanced MR angiography (CEMRA) versus average digital subtraction angiography (DSA). Middle horizontal line represents the mean of the differences in stenosis while dotted lines represent the upper and lower limits of agreement, within which 95% of the differences between the two methods are expected to lie. The values for mean differences and the limits of agreement, together with their 95% CI, are shown on the right.
Table 2 summarizes the categorical agreement between CEMRA and DSA as well as interobserver variability for each modality in terms of classification into mild, moderate, and severe ICA stenosis and complete occlusion. Agreement between CEMRA and DSA was moderate (kappa = 0.61, 0.61, and 0.62). Interobserver agreement for DSA (kappa = 0.82, 0.78, and 0.78) and CEMRA (kappa = 0.82, 0.80, and 0.83) was very good.
Table 2. Categorical agreement between CEMRA and DSA and different observers for each modality
CEMRA had a sensitivity of 93.0% and specificity of 80.6% for identifying severe ICA stenosis (table 3). CEMRA was able to reliably distinguish between cases of near-occlusion and complete occlusion (figure 3). For detection of complete occlusions (n = 35), CEMRA had a sensitivity of 97% and specificity of 99%. CEMRA correctly identified eight out of nine cases of DSA-defined near-occlusion, missing only one case with very sluggish flow, which was depicted as a complete occlusion. Moreover, one case of complete occlusion on DSA was clearly depicted as a near-occlusion by CEMRA. The CEMRA examination had taken place 3 hours before DSA and the most likely reason was that the artery had occluded in the interval between CEMRA and DSA.13
Table 3. Diagnostic accuracy and misclassification rates of CEMRA, DUS, and their combination strategies in recognizing severe stenosis with DSA as reference standard
Figure 3. Digital subtraction angiography (left) and contrast-enhanced MR angiography (right) depicting a near-occlusion of the internal carotid artery (arrows) with collapse of the distal lumen.
Twenty-eight cases of signal voids, all of which affected only very short segments, were depicted on CEMRA. In 23 cases, this represented a severe stenosis (70 to 99%) on DSA. The remaining five cases of signal voids were seen in the 60 to 69% group on DSA. The mean DSA stenosis of a signal void was 81.5% (SD 9.1%). For detection of DSA-defined plaque ulcerations (n = 35), CEMRA had a sensitivity of 66% and specificity of 98% (figure 4).
Figure 4. Digital subtraction angiography (left) and contrast-enhanced MR angiography (right) depicting a severe stenosis of the internal carotid artery with a large ulcer crater (arrows) into the atheromatous plaque.
Table 4 shows the comparison between CEMRA and DSA for all grades of stenosis. The overall diagnostic misclassification rate was 15% (n = 30 cases). These 30 cases included 24 cases overestimated as severe by CEMRA, 4 cases of DSA-defined severe stenosis underestimated by CEMRA, 1 case of complete occlusion on DSA shown as a near-occlusion by CEMRA, and finally 1 case of near-occlusion on DSA shown as a complete occlusion by CEMRA. The decision of whether to perform carotid endarterectomy was made in the clinical setting solely on the basis of DSA. Each case of misclassification was reviewed individually with regards to actual outcome to assess whether the use of CEMRA instead of DSA would have altered management. Thus, among the 24 cases of overestimation by CEMRA, 17 actually proceeded to carotid endarterectomy on the basis of DSA showing moderate stenosis (50 to 69%) and ulceration (n = 7). Thus, in these 17 patients, clinical decision-making would not have been altered if CEMRA were used instead of DSA. The patient who was shown to have a near-occlusion on DSA was treated medically, on the basis of recent evidence.21 CEMRA showed a total occlusion, which would, therefore, not have altered management. Overall, out of the 30 patients misdiagnosed, 12 remaining patients might have potentially been inappropriately denied surgery (n = 4) or inappropriately referred for surgery (n = 8) if CEMRA were used instead of DSA, resulting in a clinical misclassification rate of 6.0% (see table 3).
Table 4. Categorized stenosis measurements of internal carotid stenosis: Average CEMRA vs average DSA
DUS.
Screening DUS results were available in 189 arteries with the remaining 11 arteries being unassessable due to heavy plaque calcification. The sensitivity and specificity of screening DUS for detection of DSA-defined severe stenosis was 88.1% and 60.7% in our series. With regards to the severe category, 56 cases (29.6%) were diagnostically misclassified by DUS, with 43 cases of overestimation. Review of each individual case of diagnostic misclassification, according to clinical outcome of patients and the same principles as outlined above for CEMRA, showed that clinical decision-making would have been potentially altered in 32 cases (clinical misclassification rate = 16.9%).
Combination DUS and CEMRA strategies.
In 75.2% of these arteries (142/189), DUS and CEMRA were in agreement with regards to the presence of severe stenosis. In this subgroup, the combination of concordant CEMRA and DUS results yielded a sensitivity of 96.6% and a specificity of 79.8% for detection of DSA-defined severe stenosis. The overall diagnostic misclassification rate was reduced to 10.1% (n = 19), with 17 cases of overestimation. This translated into potentially altered clinical outcome in 3.7% (n = 7) of cases. In 47 cases (24.8%), however, discordant CEMRA and DUS results would have necessitated proceeding to DSA.
A strategy of selective DUS review was also evaluated (see table 3). With this strategy, DUS results are only taken into consideration if CEMRA shows a severe stenosis. Thus, DUS results are not reviewed for final decision-making if CEMRA depicts a less than severe stenosis. While this strategy had a slightly higher diagnostic (11.6%) and clinical (4.8%) misclassification rate compared to the concordant strategy described above, substantially fewer cases (n = 13, 6.8%) would have needed to proceed to DSA (see table 3).
Discussion.
Over the past few years, CEMRA has emerged as an important technical improvement over traditional TOF MRA and is gaining rapid acceptance among clinicians because it produces anatomically accurate images of the carotid bifurcation, which are less susceptible to dephasing and motion artifacts.11 Most of the current evidence is, however, based on studies with relatively small sample sizes and methodologic weaknesses such as retrospective designs and selection bias.1-17⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓ At the time of writing, only two published CEMRA studies had sample sizes of more than 50 patients, and none were adequately powered.12,15⇓ Moreover, the adequacy of CEMRA alone to safely replace DSA has recently been questioned. A systematic review on noninvasive modalities surprisingly found no significant differences between the diagnostic performance of CEMRA compared to traditional MRA.27 Only four studies of CEMRA were, however, included in this review. Moreover, a recent series has highlighted disturbingly high misclassification rates of 24% with CEMRA alone and recommended its use in combination with DUS.28 The retrospective design of that study may, however, have been subject to selection bias as not all patients who had CEMRA proceeded to DSA. Our study is the largest prospective series to date comparing CEMRA and DSA and is the first to be designed according to prespecified sample size calculations.
We chose the Bland-Altman method because it enables assessment of agreement between CEMRA and DSA in the context of potential interobserver errors. Validating noninvasive techniques against DSA is complex because DSA is not a true gold standard, being itself subject to significant interobserver variability. For instance, Young et al. showed that Bland-Altman limits of agreement between different observers for DSA were as wide as ±23%, resulting in clinically significant disagreements.25 In our study, for each individual reader, CEMRA systematically overestimated the degree of stenosis by a mean bias ranging from 2.4 to 3.8%. Moreover, the limits of agreement ranged from ±21.0 to ±23.8% around these mean biases, suggesting that in a prescreened population, wide disagreements can be found between CEMRA and DSA. However, even with experienced readers, there was also significant disagreement between different observers for each modality. Thus, limits of agreement about the mean bias were as wide as ±14.9% for DSA and ±17.1% for CEMRA. This clearly demonstrates that, while there are real inherent differences between the two modalities, a significant part of the observed disagreement is also directly accounted for by interobserver errors. The inherent differences between the two techniques probably relate to the difference in spatial resolution. Despite using a state-of-the-art CEMRA technique with elliptic-centric k-space acquisition, the true in-plane spatial resolution achieved (0.90 × 0.90 mm) is still two to three times less than that of DSA (0.32 × 0.32 mm), introducing further measurement variability.
We found a sensitivity of 93.0% for CEMRA in identifying DSA-defined severe stenosis, which is consistent with reported values in the literature.17-28⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓ The main diagnostic parameter of concern for CEMRA, however, remains specificity, with ranges from 62% to 100% reported by different studies.17-28⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓ This is mainly due to overestimation of stenosis by CEMRA, with the consequence that a significant number of patients may be inappropriately referred for revascularization. Most previous CEMRA studies have included contralateral normal or near-normal arteries in their analysis despite clearly using CEMRA as a confirmatory test after screening DUS.17-28⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓ These arteries have a spectrum of findings that are more easily correctly classified by both CEMRA and DSA. As demonstrated by Kallmes et al., including these arteries leads to a higher inflated specificity and does not reflect a prescreened population, which will have a higher prevalence of arteries with moderate to severe disease.26 Yet, in clinical practice, because it is inexpensive and portable, DUS is likely to remain the first line investigation for carotid disease. CEMRA is thus primarily used as a confirmatory test, as evidenced by a recent survey which found that not a single institution used CEMRA as first line investigation.7 We did not include normal or near-normal arteries in our analysis and therefore argue that our calculated specificity of 80.9% genuinely reflects the population in which CEMRA is normally used in clinical practice.
CEMRA and DSA disagreed about the presence of a severe stenosis in 30 cases, yielding a diagnostic misclassification rate of 15%. Most (n = 24) of these cases were due to the tendency of CEMRA to overestimate stenosis. This diagnostic rate, however, assumes a hard-cut threshold of 70% stenosis as requiring revascularization. Such absolute thresholds are necessary for statistical analysis but are not ideal. In clinical practice, there is a continuum of benefits and selected patients with moderate stenoses also frequently proceed to surgery. According to these criteria, a stenosis measured at 69% on DSA and 71% on CEMRA would have thus been considered a misclassification, when it is most probable that such small differences are not clinically significant, especially given the magnitude of potential interobserver variability. The more relevant question, as recognized previously by several authors, is therefore what implications CEMRA would have for clinical decisions for individual patients.12,29⇓ Cosottini et al. examined the use of CEMRA in revascularization decision-making and found low misclassification rates of 3.1%.12 Similarly, we reviewed each individual case of misclassification and followed actual clinical outcome, to define how use of CEMRA instead of DSA would have potentially affected clinical decision-making. We found that in only 12 cases, patients would have been potentially inappropriately referred for or denied surgery, resulting in a clinical misclassification rate of 6%. However, not all of these 12 patients will have a bad outcome caused by the wrong decision. Wardlaw et al. found a 7% misclassification rate for MRA and calculated that 14% of misdiagnosed patients would need to die and 29% have a major stroke for the inaccuracies caused by this misclassification to outweigh the risks associated with DSA.30 We agree with their argument that this is improbable and therefore consider that our 6% clinical misclassification rate is acceptably low to support the safe replacement of DSA by CEMRA alone.
In addition, the design of our study specifically addresses another important issue, which has not been previously answered about CEMRA. We used identical four-view projections to report on both DSA and CEMRA. While conventional DSA is a two-dimensional technique, with two to four projections usually acquired, CEMRA is a three-dimensional technique with all projection directions possible. Reporting on CEMRA as a four-view study undeniably does not take full advantage of its three-dimensional nature and we certainly do not advocate reporting on CEMRA in this way in clinical practice. In the context of a method comparison study between CEMRA and DSA, this is, however, essential. There is substantial evidence in the literature that, in the presence of noncircular lumens, a limited number of projections may fail to demonstrate the tightest stenosis.31 All previous CEMRA studies have failed to specify or compared a different number of projections for CEMRA and DSA.17-28⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓ There have been recent studies suggesting that the overestimation with traditional TOF MRA may be directly accounted for by differences in projections used.32 Our study clearly shows that, irrespective of the projections used, the tendency of CEMRA to overestimate stenosis occurred and may be inherent to the technique.
There is a growing trend for studies to support the use of combination strategies of concordant DUS and MRA results.17,28,29,33,34⇓⇓⇓⇓ This certainly is an attractive proposition and there is no doubt that this reduces misclassification rates. Combination DUS and MRA strategies are, however, not ideal, because both tests tend to overestimate stenosis, with both having only moderate specificities and positive predictive values. The major issue with combination strategies therefore remains the number of cases in which disparate DUS and CEMRA results would mean that patients would need to proceed to DSA. In our study, we found that concordant DUS and CEMRA reduced the diagnostic misclassification rate from 15% (n = 30) to 10.6% (n = 19). A similar improvement occurred in clinical misclassification rates from 6.0% (n = 12) to 3.7% (n = 7). Discordant DUS and CEMRA results, however, meant that 47 cases (25%) would have needed DSA to confirm the degree of stenosis. In the literature, a wide range of discordant rates, as poor as 52%, have been reported.17,28,29,33,34⇓⇓⇓⇓ The improvements in misclassification rates therefore need to be closely balanced against issues of cost-effectiveness, notwithstanding the fact that a significant proportion of patients may be exposed to the risks of DSA. In this study, we propose an intermediate strategy of selective review of DUS results, only if CEMRA were to show a severe stenosis. This takes advantage of the fact that CEMRA has a very high negative predictive value (95%). Thus, if CEMRA were to demonstrate a surgically nonrelevant stenosis, we suggest that review of DUS is not necessary. This intermediate approach might be a more practical solution, with a substantial reduction in the number of cases requiring DSA (n = 13, 6.8%) with only marginally worse diagnostic (11.6%, n = 22) and clinical (4.8%, n = 9) misclassification rates.
The results of the diagnostic accuracy of DUS alone may seem low compared to other studies, with a specificity of 60.7% only.4,33⇓ Two factors may have contributed to this. First, verification bias exists as the decision to perform DSA was directly dependent on results of DUS. The sensitivity may be lower but the specificity higher after adjustment of this bias.34 Secondly, verification bias would have been accentuated by the fact that we did not include normal or near-normal arteries in the analysis. Our reported diagnostic accuracy therefore reflects a prescreened population, which is not the population in which DUS is used in clinical practice. Although these biases affect our reported results for DUS alone (which is not the main focus of our study), they do not influence our reported results for combination DUS and CEMRA strategies. Moreover, our calculated values of diagnostic accuracy and misclassification rates (see tables 3 and 4⇑) are derived from the use of the average of the CEMRA readings, thereby reflecting a comparison with DSA under ideal and best circumstances. In clinical practice, only one of a team of clinicians would usually report on each examination. Results based on single reader measurements are thus presented in online tables E-1 and E-2 (access www.neurology.org). For each artery, one measurement is selected at random from the three available measurements from the three readers. These results are not significantly different from those based on average CEMRA measurements and therefore do not affect our overall conclusions.
We did not review CEMRA source images in this study to maintain a like-for-like comparison between the two modalities. Whether source image review does affect the diagnostic accuracy of CEMRA remains to be adequately demonstrated but there is some evidence that it may slightly improve specificity.13 Our own experience, however, suggests that it is currently practically difficult to make accurate measurements on axial source images with standard existing software and we agree with Huston et al. that an improved display or presentation technique for MRA data that harvests the advantages of both source and MIP data are necessary.13
In our study, seven patients were unable to undergo CEMRA because of MR contraindications such as claustrophobia and metal. In practice, these patients would need to proceed to DSA or may be suitable for CTA. However, similar to MRA, the reported diagnostic accuracy of CTA is confusingly wide and the modality has yet to be validated in adequately powered and methodologically robust studies.14,16⇓ In addition, difficulty in categorizing stenosis due to plaque calcification as well as exposure to radiation may constitute limitations of CTA. A final limitation of our study is that we did not examine the ability of CEMRA to detect tandem stenoses but concentrated on the most important issue, the degree of ICA stenosis, instead. This is an issue that has not been adequately tested in the literature and there is, moreover, disagreement among authors about the relevance of these tandem lesions.11 Their low prevalence (1 in 50 cases) means that, to rigorously and prospectively address this issue, independent and blinded assessment of all vascular segments from aortic arch to circle of Willis, in terms of potential diagnostic information, would have been necessary.11
Acknowledgments
Supported by a grant from the National Health Service Research and Development Programme. R.A.T. was funded by the UK Stroke Association.
The authors thank Dr. Peter Martin, Mike Gaunt, Kevin Varty, Claire Sims, Ilse Joubert, Christina Pittock, Anna Crawley, Robin Holloway, Kate Austen, Tessa Lewis, Michelle Varley, Jennifer Golledge, Karen Blake, and Carole Turner from Addenbrooke’s Hospital; Professor Gary Ford and Dr. Anil Gholkar from Newcastle University; and Professor Joanna Wardlaw from Edinburgh University for their support and expertise.
Footnotes
-
Additional material related to this article can be found on the Neurology Web site. Go to www.neurology.org and scroll down the Table of Contents for the April 27 issue to find the title link for this article.
-
See also page 1246
- Received July 24, 2003.
- Accepted February 10, 2004.
References
Disputes & Debates: Rapid online correspondence
NOTE: All authors' disclosures must be entered and current in our database before comments can be posted. Enter and update disclosures at http://submit.neurology.org. Exception: replies to comments concerning an article you originally authored do not require updated disclosures.
- Stay timely. Submit only on articles published within the last 8 weeks.
- Do not be redundant. Read any comments already posted on the article prior to submission.
- 200 words maximum.
- 5 references maximum. Reference 1 must be the article on which you are commenting.
- 5 authors maximum. Exception: replies can include all original authors of the article.
- Submitted comments are subject to editing and editor review prior to posting.