Gary S.Gronseth, AAN Guidelines, 1080 Montreal Avenue, St. Paul, MN 55116[email protected]
Jacqueline A French
Submitted January 16, 2009
We appreciate the opportunity to respond to Dr. Pincus’s comments. Several of his points require clarification.
Not all studies rated Class I are equal. Within the Class I designation there are stronger Class I studies and weaker Class I studies. A Class I study that through bad luck resulted in substantial differences of known confounders in treatment groups should be—as implied by Dr. Pincus— considered to provide weaker evidence than a Class I study that did not result in substantial differences in known confounders between groups. This would be true even if statistical adjustment could be made for the confounding differences. However, whether relatively weak or strong, evidence from studies earning a Class I designation should be considered to have a lower risk of bias than evidence from studies rated Class II, III, or IV.
Even in the most well-designed and well-executed study there is a risk of bias from unknown confounders. This is true whether or not statistical adjustments for known confounders are made. This is why even the best Class I study can never “prove” a treatment effect. The results of a randomized control trial are never definitive.
Dr. Pincus correctly points out that an ITT analysis biases studies to show equivalence of the compared treatments. This is the reason that the evidence classification system developed by the AAN requires additional criteria for claims of therapeutic equivalence. These additional criteria are stated in the therapeutic classification of evidence scheme. [1] To avoid the equivalence bias introduced by an ITT analysis, the criteria specify that an observed case analysis is preferred.
Disclosure: The authors report the following conflicts of interest: Dr. French holds financial interests in Jazz, Eisai, Valeant, Marinus, Pfizer, and UCB. She has received research funding from the Epilepsy Therapy Development Project, FACES, UCB, Eisai, Johnson and Johnson, and Merck. Dr. French estimates that 30% of her time is spent in outpatient epilepsy practice. Dr. Gronseth has received speaker honoraria from Pfizer, GlaxoSmithKline, and Boehringer Ingelheim and served on the IDMC Committee of Ortho-McNeil. Dr. Gronseth estimates that <_2 of="of" his="his" time="time" is="is" spent="spent" on="on" emg="emg" and="and" eeg.="eeg."/>
We appreciate the opportunity to respond to Dr. Pincus’s comments. Several of his points require clarification.
Not all studies rated Class I are equal. Within the Class I designation there are stronger Class I studies and weaker Class I studies. A Class I study that through bad luck resulted in substantial differences of known confounders in treatment groups should be—as implied by Dr. Pincus— considered to provide weaker evidence than a Class I study that did not result in substantial differences in known confounders between groups. This would be true even if statistical adjustment could be made for the confounding differences. However, whether relatively weak or strong, evidence from studies earning a Class I designation should be considered to have a lower risk of bias than evidence from studies rated Class II, III, or IV.
Even in the most well-designed and well-executed study there is a risk of bias from unknown confounders. This is true whether or not statistical adjustments for known confounders are made. This is why even the best Class I study can never “prove” a treatment effect. The results of a randomized control trial are never definitive.
Dr. Pincus correctly points out that an ITT analysis biases studies to show equivalence of the compared treatments. This is the reason that the evidence classification system developed by the AAN requires additional criteria for claims of therapeutic equivalence. These additional criteria are stated in the therapeutic classification of evidence scheme. [1] To avoid the equivalence bias introduced by an ITT analysis, the criteria specify that an observed case analysis is preferred.
Disclosure: The authors report the following conflicts of interest: Dr. French holds financial interests in Jazz, Eisai, Valeant, Marinus, Pfizer, and UCB. She has received research funding from the Epilepsy Therapy Development Project, FACES, UCB, Eisai, Johnson and Johnson, and Merck. Dr. French estimates that 30% of her time is spent in outpatient epilepsy practice. Dr. Gronseth has received speaker honoraria from Pfizer, GlaxoSmithKline, and Boehringer Ingelheim and served on the IDMC Committee of Ortho-McNeil. Dr. Gronseth estimates that <_2 of="of" his="his" time="time" is="is" spent="spent" on="on" emg="emg" and="and" eeg.="eeg."/>