I refer to Section 5.2 of the review of study which looks at the Nissen study:
N&W used the Peto method to compute odds ratios and confidence intervals. Trials with zeros in both arms are excluded from the analysis when using this approach as well as other approaches, such as the exact test used by this reviewer. In cases where only a few studies are excluded (as for MI where 4 studies
were excluded), the impact is minimal but when about half the trials are excluded (as is the case for the CV mortality endpoint in both N&W’s database and in the GSK database) there may be a greater impact on the results.(my emphasis...that's called withholding evidence.)
The latter point is illustrated with the database provided by GSK. This reviewer performed several analyses of the mortality data (both CV and all-cause; overall event rates less than 0.3%) and the results clearly show that the analytical approach can change non-significant results when including all the data (p>0.3) to borderline significant results when just considering those studies with at least one death (third
row of Table 3.1.2). The results for analyses using a continuity correction of 0.5 in each cell of those trials with zeros in either one arm or both arms are particularly striking with odds ratios close to one. "
This reviewer thinks that these results demonstrate the problems with any meta-analytic technique when data is extremely sparse and suggest that performing additional analyses may be warranted under these circumstances."
For the table and full discussion please go to:
http://www.fda.gov/ohrms/dockets/ac/07/briefing/2007-4308b1-00-index.htm
N&W used the Peto method to compute odds ratios and confidence intervals. Trials with zeros in both arms are excluded from the analysis when using this approach as well as other approaches, such as the exact test used by this reviewer. In cases where only a few studies are excluded (as for MI where 4 studies
were excluded), the impact is minimal but when about half the trials are excluded (as is the case for the CV mortality endpoint in both N&W’s database and in the GSK database) there may be a greater impact on the results.(my emphasis...that's called withholding evidence.)
The latter point is illustrated with the database provided by GSK. This reviewer performed several analyses of the mortality data (both CV and all-cause; overall event rates less than 0.3%) and the results clearly show that the analytical approach can change non-significant results when including all the data (p>0.3) to borderline significant results when just considering those studies with at least one death (third
row of Table 3.1.2). The results for analyses using a continuity correction of 0.5 in each cell of those trials with zeros in either one arm or both arms are particularly striking with odds ratios close to one. "
This reviewer thinks that these results demonstrate the problems with any meta-analytic technique when data is extremely sparse and suggest that performing additional analyses may be warranted under these circumstances."
For the table and full discussion please go to:
http://www.fda.gov/ohrms/dockets/ac/07/briefing/2007-4308b1-00-index.htm
