Antivirals for Treatment of Influenza: A Systematic Review and Meta-analysis of Observational StudiesFREE
Submit a Comment
Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
Abstract
Background:
Purpose:
Data Sources:
Study Selection:
Data Extraction:
Data Synthesis:
Limitations:
Conclusion:
Primary Funding Sources:
Methods
Data Sources and Searches
Study Selection
Data Extraction and Quality Assessment
Data Synthesis and Analysis
Role of the Funding Source
Results
Literature Flow
Oseltamivir
Zanamivir
Oseltamivir Versus Zanamivir
Amantadine and Rimantadine for Influenza A
Discussion
Appendix: Search Strategies
EMBASE (1980 to Week 452 010), MEDLINE In-Process and Other Nonindexed Citations, MEDLINE (1950 to 16 November 2010), the Cochrane Library databases, Chinese Biomedical Literature Database, and Panteleimon (to November 2010)
SIGLE (to November 2010)
CINAHL (1981 to November 2010)
LILACS (to November 2010)
Supplemental Material
Supplement Table 1. Subgroup Analyses for oral oseltamivir versus no antiviral therapy
Supplement Table 2. Subgroup Analyses for oral oseltamivir within 48 hours or after 48 hours
Supplement Table 3. Subgroup Analyses for inhaled zanamivir versus no antiviral therapy
Supplement Table 4. Subgroup Analyses for oral oseltamivir versus inhaled zanamivir
Supplement Table 5. GRADE evidence profile for oral amantadine versus no antiviral therapy
References
Information & Authors
Information
Published In
History
Keywords
- Adverse events
- Antimicrobials
- Antiviral therapy
- Antivirals
- Diagnostic medicine
- Drugs
- Health care
- Health care facilities
- Hospitalizations
- Hospitals
- Infectious diseases
- Mortality
- Patients
- Pharmacology
- Population statistics
- Prevention, policy, and public health
- Research and reporting methods
- Research assessment
- Research design
- Signs and symptoms
- Vital statistics
Copyright
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Antivirals for Treatment of Influenza: A Systematic Review and Meta-analysis of Observational Studies. Ann Intern Med.2012;156:512-524. [Epub 3 April 2012]. doi:10.7326/0003-4819-156-7-201204030-00411
View More
Get Access
Login Options:
Purchase
You will be redirected to acponline.org to sign-in to Annals to complete your purchase.
Create your Free Account
You will be redirected to acponline.org to create an account that will provide access to Annals.
Interpretation of Low and Very Low Quality Evidence
We described many of the issues raised by the authors of the two comments such as the large potential for selection bias quoted by Jones and colleagues in our systematic review (1). While we agree that survivor treatment selection bias can be eliminated by modeling “it can simultaneously lead to a different bias in the opposite direction because sicker patients may be more likely to initiate treatment at any given time” (2 ). The inclusion of data from abstracts was part of our a priori protocol, independently of study results. We did this to reduce potential publication bias of studies that do not favor oseltamivir. We disagree that our results are “unreliable”. We prefer our interpretation of “low or very low confidence or certainty in the estimates of effect” because reliability addresses a different concept.
Heneghan and colleagues are apparently against systematic reviews of observational studies to evaluate potential benefits of interventions and quote the Agency for Healthcare Research and Quality (AHRQ). AHRQ regularly supports such research. We believe that data from observational studies may be helpful for interpretation of RCTs and supplement data from RCTs. They may provide information about broader and special populations and address outcomes that are challenging to ascertain in RCTs. We do not criticize RCTs as a study design which is emphasized by our other work and GRADE. In fact, we would nearly always encourage the conduct of large RCTs and, therefore, asked for “trial preparedness” in our article (1). Furthermore, we have also emphasized the need for making data available (RCT) (1, 3).
We agree that larger sample sizes from observational studies leading to narrow confidence intervals can be misleading and it is unfortunate that people still draw conclusions based on whether a result is ‘significant’ or ‘not significant’. We however, have not interpreted results based on significance alone or described them as reliable. We refer the authors and readers of the comment to the conceptual underpinnings of GRADE (4) which we used to assess the quality of evidence, taking into account criteria such as risk of bias, inconsistency, and imprecision to grade the confidence in effect estimates from systematic reviews. Our interpretation of “low or very low confidence or certainty in the estimates of effect” describes that the true effect may be (low quality) or is likely to be (very low quality) substantially different from the observed estimate of the effect, concepts that are different from reliability (4). Heneghan and colleagues question why we used the term “may reduce”. The term “may” expresses a contingency and not certainty. Thus, we do not see how we expressed “optimism”. Suggestions and research, as there is little, regarding how else to phrase such observations in research syntheses are currently explored by the GRADE Working Group as part of the DECIDE project (www.decide-collaboration.eu). Our terminology is also similar to what is currently investigated by the Cochrane Collaboration.
We agree that there is inherent risk of improper citation, but this phenomenon can occur with any research and we believe that by including statements about the overall quality misquotes will be less likely. We disagree that one of our co-authors has “misquoted” the results. Dr. Uyeki described, almost verbatim, the text of our systematic review (http://www.bt.cdc.gov/coca/calls/2012/callinfo_022812.asp).
Clinicians, policy makers and other stakeholders have health care questions that require answers - whether the evidence is from RCTs or observational studies. Unfortunately, the quality of the evidence from RCTs on this topic is likely to be low and could remain low if the unpublished data were available. We believe observational studies should be used to inform decision making when RCTs do not provide high or moderate quality evidence until such higher quality is available. The lead authors of this article conduct systematic reviews on many topics with adequate collaboration from content experts to ensure the work is well informed. We place emphasis on the underlying quality of the evidence, along with other factors, such as values and preferences, to support informed recommendations and decisions.
References
1. Hsu J, Santesso N, Mustafa R, Brozek J, Chen Y, Hopkins J, Cheung A, Hovhannisyan G, Ivanova L, Flottorp S, Sæterdal I, Wong A, Tian J, Uyeki TM, Akl EA, Alonso-Coello P, Smaill F, Schunemann HJ. Antivirals for Treatment of Influenza: A Systematic Review and Meta-analysis of Observational Studies. Annals of Internal Medicine. 2012;156:512-24.
2. Glesby M, Hoover, DR. Selection Bias in Observational Studies: Examples from the AIDS Literature. Ann Intern Med. 1996;124:999-1005.
3. Schünemann H, Ghersi D, Kreis J, Antes G, Bousquet J. Reporting Research: Are we in for better helath care by 2020? In: Gigerenzer G MGM, editor. Better Doctors, Better Patients, Better Decisions: Envisioning Healthcare 2020. Cambridge, MA: MIT Press; 2011. p. 83-102.
4. Guyatt GH, Oxman AD, Schunemann HJ, Tugwell P, Knottnerus A. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. Journal of clinical epidemiology. 2011;64(4):380-2. Epub 2010/12/28.
Re:Risks in presenting observational results of therapeutic interventions
We did not criticize RCTs as a design in our review (the contrary is true) and we did not disagree that all data should be made available by whomever produces it. The other issues are addressed and transparently described in the review. Holger Schunemann
Conflict of Interest:
See primary article on which I am corresponding author.
Comment on Neuropsychiatric events in Hsu, et al (2012)
In their meta-analysis of observational studies, Hsu et al. report that treatment with oral oseltamivir may result in fewer neuropsychiatric adverse events (NPAE) than no antiviral therapy according to the results of five studies. We would like to comment on NPAEs and oseltamivir use. We conducted a preliminary analysis [1] of neuropsychiatric adverse events (NPAEs) from 31 randomized controlled trials (RCTs) reported in the Japanese summary basis of approvals (SBAs) of oseltamivir for treatment of adults, treatment of children and for prophylaxis [2]. Unfortunately NPAE events were not actively collected in the treatment trials because events thought to be related to influenza were not reported as adverse events leading to very low estimates of around 0.5%. In an analysis of the prophylaxis trials however, odds ratios were: 7.95 (p=0.0387) for psychotic/suicidal events, 1.23 (p=0.0472) for headache, 3.19 (p=0.0259) for pain in extremities, and 5.48 (p=0.0221) for earache. These data indicate that oseltamivir may cause psychotic/suicidal reactions and other types of pain in the body. In a systematic review of prospective cohort studies [3] the three included studies were all conducted specifically to investigate the association of oseltamivir and neuropsychiatric events in Japan with a total of 14,000 children participating. Proportions of children with abnormal behaviours were 6.4 to 9.4 % in the non-oseltamivir groups and 11.0 to 12.6 % in the oseltamivir groups. Overall pooled odds ratio was 1.55 (95% CI = 1.21, 1.98, p=0.0005). In two of the studies [4.5], the largest associations were observed during the first day of influenza with odds ratios of around 4.0 to 7.0 for abnormal behaviours, delirium or unconsciousness. Finally, we are concerned that the quality of all five studies meta- analysed by Hsu et al is very low. For example: (1) two studies did not focus on the analysis of neuropsychiatric events; (2) the included events were generally non-specific, not predefined and not systematically collected; (3) duration of follow-up was very long potentially masking treatment related adverse events that occur soon after commencement of oseltamivir; and (4) no adjustment was made for the possible confounding effect of other medications such as antipyretics. The limitations of these studies could explain why their findings are inconsistent with those obtained from higher quality prospective studies that suggest an association between neuropsychiatric adverse events and oseltamivir.
References
1.Hama R, Jones M, Hayashi K, Yanagi T, Sakaguchi K. Oseltamivir: A systematic review of adverse effects in randomized controlled trials: presented at the 16th JSPE and 5th ACPE joint meeting Tokyo 29-30th Oct. (2010)
2.Chugai Pharm Co 2001. (then Roche Japan). Summary basis of Approval (SBA) of oseltmivir (in Japanese); Tamiflu capsule for treatment (2001), Tamiflu dry syrup (2002), Tamiflu capsule for prophylaxis (2004) http://www.info.pmda.go.jp/info/syounin_index.html
3.Hama R, Jones M, Hayashi K, Yanagi T, Sakaguchi K. Oseltamivir: A systematic review and meta-analysis of adverse effects in prospective cohort studies: presented at the 16th JSPE and 5th ACPE joint meeting Tokyo 29-30th Oct. (2010)
4.Yokota S, Fujita T, Mori M et al. Epidemiologic survey of influenza- associated complications (I); Clinical assessment of symptoms and signs, and medication. J Japan Pediatric Soc. 111 (2007): 1545-58.
5.Fujita T. Fujii Y, Watanabe Y, Mori M, Yokota S. A Pharmacoepidemiological Study on the Relationship between Neuropsychiatric Symptoms and Therapeutic Drugs after Influenza Infection. Jap J Pharmacoepidemiol 2010; 15: 73-92. Available from: http://www.jstage.jst.go.jp/article/jjpe/15/2/73/_pdf
Conflict of Interest:
Both authors are investigators for a UK National Institute of Health Research grant for a systematic review of neuraminidase inhibitors. RH provided scientific opinions on eleven adverse reaction cases related to oseltamivir following application by their families for adverse reaction compensation and he receives royalties from two books published in 2008 titled "Tamiflu: harmful as was afraid" and "In order to escape from drug-induced encephalopathy".
Risks in presenting observational results of therapeutic interventions
We disagree with Hsu and colleagues presentation of results from their meta-analysis of observational studies. [1] Properly reporting the results of meta-analyses of large datasets of poor quality evidence is tricky. Large sample sizes alone will generate effect estimates with narrow confidence intervals, but if the underlying data is of poor quality, there is no guarantee these effect estimates are trustworthy. Hsu and colleagues graded their evidence as low- or very low-quality, yet still presented statistically significant numerical results and concluded that oseltamivir "may reduce" mortality and hospitalization. This optimism is misplaced, for the "confounding and selection, reporting, and publication bias" they note has the potential to not only weaken, but reverse their effect estimates. In fact, our recent systematic review of placebo-controlled randomized controlled trials (RCTs) found no effect on hospitalization (odds ratio 0.95 [95% CI, 0.57 to 1.61]).
The risk inherent in their presentation of results is improper citation. Agency for Healthcare Research and Quality (AHRQ) guidelines state: "poor-quality evidence from observational studies should not be used or relied on, even if it appears to address gaps in the trial evidence." [2]
Unfortunately, like Hsu et al., a previous Cochrane review of influenza vaccines reported both impressive confidence intervals suggestive of benefit from vaccination and caution that the observational studies were of poor quality and likely unreliable. But in subsequent vaccination guidelines, the confidence intervals were cited without mentioning the limitations. [3] It is too early to calculate the extent to which the Hsu et al. meta-analysis will be likewise misquoted , but it has already occurred once, in a recent presentation by one of the paper's co-authors. [4]
Hsu et al. criticize RCTs of influenza antivirals for multiple reasons including a lack of precision of effect estimates. Whilst observational studies are suitable to detect rare or late adverse effects of treatments, they are not the answer to the problem of lack of precision of RCT data. Given that up to 60% of randomised oseltamivir data remain inaccessible the answer is for independent reviews of all evidence before embarking on any further investigations. The problem with the oseltamivir RCT evidence base is less an issue of study design but rather data secrecy. [5] Many of the outcomes Hsu et al. are interested in could be answered if oseltamivir's manufacturer fulfilled its promise to share its full study reports.
References
[1] Hsu J, Santesso N, Mustafa R, Brozek J, Chen YL, Hopkins JP, et al. Antivirals for Treatment of Influenza: A Systematic Review and Meta- analysis of Observational Studies. Ann Intern Med [Internet]. 2012 Feb 27 [cited 2012 Mar 5]; Available from: http://www.annals.org/content/early/2012/02/27/0003-4819-156-7-201204030- 00411
[2] Methods Guide for Comparative Effectiveness Reviews: Selecting Observational Studies for Comparing Medical Interventions. Available at http://www.effectivehealthcare.ahrq.gov/ehc/products/196/454/MethodsGuideNorris_06042010.pdf (accessed 11 March 2012)
[3] Jefferson T, Di Pietrantonj C, Debalini MG, Rivetti A, Demicheli V. Inactivated influenza vaccines: Methods, policies, and politics. Journal of Clinical Epidemiology. 2009 Jul;62(7):677-86.
[4] Uyeki T. 2011-2012 Influenza Season: Antiviral Medication Recommendations. 28 Feb 2012. Available from: http://www.bt.cdc.gov/coca/calls/2012/callinfo_022812.asp (accessed March 13, 2012).
[5] Jefferson T, Jones MA, Doshi P, Del Mar CB, Heneghan CJ, Hama R, et al. Neuraminidase inhibitors
Conflict of Interest:
Peter Doshi is funded by an institutional training grant from the Agency for Healthcare Research and Quality #T32HS019488 and declares no potential conflicts of interest. Tom Jefferson was an ad hoc consultant for F.Hoffman-La Roche Ltd in 1998-1999. He receives royalties from his books published by Blackwell and Il Pensiero Scientifico Editore, none of which are on neuraminidase inhibitors. He is occasionally interviewed by market research companies for anonymous interviews about Phase 1 or 2 products. He is a consultant in a legal case regarding oseltamivir. CH PD and TJ have published on Antivirals and received UK National institute of Health Research Grant funding for the update and amalgamation of two Cochrane Reviews: neuraminidase inhibitors for preventing and treating influenza in healthy adults and children
Comment on Hsu, et al (2012): Survivor treatment selection bias
In their meta-analysis of observational studies, Hsu et al. report that oral oseltamivir reduces mortality with an odds ratio of 0.23 (95% CI, 0.13 to 0.43); low-quality evidence. Hsu et al.'s result is based on three studies, one of which is only published as a conference abstract, and lacks detail, and another, Liem et al[1] conducted logistic regression and Kaplan-Meier analysis from the onset of symptoms to 30 days since illness onset. This then creates an important bias that has the potential to reverse their findings. Similar problems exist in the other study (Hanshaoworakul et al[2]). Although median time from symptom onset to treatment administration was 2.62 days for survivors and 4 days for non- survivors, logistic regression was used for analysis meaning that the time -dependent nature of antiviral exposure was not considered. Survivor treatment selection bias (also known as survival bias, immortal time bias or time-dependent bias[3]) inevitably occurs in observational studies which incorrectly treat time-dependent exposures as time-fixed exposures[4]. The exposure time misallocation is at least a median of 6 days from onset of illness (the time from onset of symptoms to hospital admission) for the Liem et al study[1]. This delay in treatment introduces a selection bias because patients who die early or are very sick, for example, may get less opportunity to begin antiviral treatment while other critical procedures take priority. In a cohort of deceased Japanese influenza patients we found that survivor treatment selection bias was large enough to change the treatment effect from an incorrect protective effect of antiviral treatment on mortality to a correct detrimental effect and we estimated a six-fold increased risk of deterioration leading to death for patients exposed to oseltamivir[5]. This effect of survivor treatment selection bias is consistent with that reported by Beyersmann et al[4] who have shown mathematical proof that time-dependent bias is in the direction of prolonging the outcome in the exposed group. They also suggest time-dependent bias can be avoided by proper hazard-based analyses. As Hsu et al. did not obtain original study data to carry out a proper re- analysis of time-varying exposures, their reported results are unreliable. This applies not only to mortality but for all outcomes (e.g. hospitalization) that may be associated with a time-dependent exposure.
References
1. Liem, N., Tung, C., Hien, N., et al., Clinical Features of Human Influenza A (H5N1) Infection in Vietnam: 2004-2006. Clinical Infectious Diseases, 2009. 48: p. 1639-1646.
2. Hanshaoworakul, W., Simmerman, J., Narueponjirakul, U., et al., Severe Human Influenza Infections in Thailand: Oseltamivir Treatment and Risk Factors for Fatal Outcome. Plos One, 2009. 4(6): p. e6051.
3. Laupacis, A., Observational Studies of Treatment Effectiveness: Some Cautions. Ann Intern Med. , 2004. 140: p. 923-924.
4. Beyersmann, J., Gastmeier, P., Wolkewitz, M., Schumacher, M. , An easy mathematical proof showed that time-dependent bias inevitably leads to biased effect estimation. Journal of Clinical Epidemiology, 2008. 61: p. 1216-1221.
5. Jones, M., Hama, R., Survivor treatment selection bias in a cohort of deceased influenza patients exposed to antiviral medication 2012 Manuscript submitted for publication.
Conflict of Interest:
All three authors are investigators for a UK National Institute of Health Research grant for a systematic review of neuraminidase inhibitors. RH provided scientific opinions on eleven adverse reaction cases related to oseltamivir following application by their families for adverse event compensation and he receives royalties from two books published in 2008 titled "Tamiflu: harmful as was afraid" and "In order to escape from drug-induced encephalopathy". CDM provided expert advice to GlaxoSmithKline about vaccination against acute otitis media in 2008-2009. He receives royalties from books published through Blackwells BMJ Books and Elsevier and has recently updated a Cochrane Review on physical interventions to prevent the spread of ARIs with WHO funds.