Researchers defend influenza vaccine study

August 2005

We welcome this opportunity to discuss the implications of our recent study of influenza-related mortality among elderly people (age 65 and older). In their critique, David S. Fedson, MD, and Kristin Nichol, MD, MPH, rhetorically ask whether we should question the benefits of influenza vaccination for the elderly. We think the answer to this question is unequivocally “yes.”

We all should question the magnitude of that benefit, but this does not imply that the elderly should not be vaccinated. There is a void of evidence from randomized, placebo-controlled clinical trials in the elderly for influenza vaccines that are re-licensed annually, confusion about the effectiveness of the vaccine and, consequently, substantial disagreement in the literature on the implied burden of influenza on mortality. Because influenza is an important cause of mortality in the elderly and because we had found no apparent reduction in influenza-related deaths despite great strides in vaccination coverage, we suspect there is room for improvement and believe further research is warranted to identify more effective control strategies.

Our study hypothesized that careful statistical adjustment for aging within the elderly population and for increased circulation of virulent A/H3N2 viruses would reveal a declining trend in influenza-related mortality — a reasonable expectation in light of the fourfold increase in influenza vaccination coverage in the United States since the early 1980s. We used the traditional Serfling seasonal regression model to assess adjusted influenza-related mortality among different age groups for the U.S. elderly population for the period 1968 through 2002, as given by seasonal excess mortality above a winter baseline.

Our paper presented two distinct conclusions. First, we could not find the expected decline in adjusted influenza-related mortality from 1980 to 2001, despite an increase in vaccination coverage of the elderly from about 15% to about 65%. Thus, as expected, we found no evidence that the tremendous effort to increase influenza vaccine coverage among the elderly had substantially decreased influenza-related mortality. Secondly, we demonstrated an inconsistency in the number of seasonal deaths in the elderly due to influenza. That would be expected if influenza vaccination prevents 50% of winter deaths as the cohort studies have led many to claim.

Specifically, we found that in recent decades an average of 30,000 deaths among the elderly are attributable to influenza, accounting for 0% to 10% of all winter deaths among the elderly in any season studied. This agrees well with recent CDC estimates, which are also based on seasonal mortality data. Multiple cohort studies, on the other hand, find that influenza vaccination can prevent about 50% of all winter deaths among the elderly population, implying that about 50% of all winter deaths in those studies can be attributable to influenza. Given the annual total of 670,000 elderly people dying each winter in recent seasons, this 50% vaccination effectiveness figure would suggest that at 65% coverage, about 323,000 elderly deaths have been prevented each season and that a residual 170,000 seasonal deaths could additionally be saved if the remaining 35% of the elderly population was also vaccinated. There is a clear difference between the 170,000 residual deaths suggested by cohort studies, and the 30,000 deaths estimated by the CDC and us. Furthermore, during the 1997 to 1998 season, when the vaccine was considered to be ineffective due to a complete mismatch of the A(H3N2) component, there was 59,000 excess deaths, an order of magnitude fewer than the approximate 500,000 influenza-related deaths one would have projected from the cohort studies in that season’s “unprotected” elderly population. This is the “vast disconnect” to which we refer.

Fedson and Nichol argue that such a comparison is not valid, because the 50% Vaccine effectiveness estimate comes from cohort studies conducted in healthy community-dwelling elderly, and they argue these results should not be applied to the general elderly population. However, Nichol’s own paper as well as an older study both reported similar or even larger vaccine mortality benefits in nursing home populations, and so we think it reasonable to apply the vaccine effectiveness figure that cohort studies suggest to the total U.S. elderly population to illustrate the claimed astonishing mortality benefits.

Two hypotheses

In our paper, we offered the hypothesis that the disappointing trends observed could be partially explained by undervaccination of frail elderly at high risk of death. We therefore fully take Fedson and Nichol’s point, under the heading “Ecologic Fallacy,” that our study could not address disparities in vaccination coverage among different groups. This same phenomenon, however, could likely cause a systematic but hard-to-detect bias in cohort studies that would result in overestimation of Vaccine effectiveness. The commonly used modeling approach to adjust for differences between vaccinated and unvaccinated elderly typically does not include factors, such as end-stage disease, recent hospitalizations and the need for intensive care, that specifically determine the health status during the vaccination season. Therefore, this approach may not effectively adjust for bias relating to a frail, undervaccinated subpopulation of elderly.

Epidemiologists in the United Kingdom recently demonstrated and corrected for self-selection bias in their cohort study, by comparing Vaccine effectiveness estimates during the influenza season to Vaccine effectiveness in “peri-influenza” periods just outside the influenza season when no vaccine benefits could reasonably be expected. They reported that vaccinated elderly were 20% less likely than unvaccinated elderly to die from any cause during influenza periods — and also 20% less likely to die in peri-influenza periods. They therefore correctly adjusted their Vaccine effectiveness estimate to be 0% in terms of prevention of all-cause mortality.

A related approach to identify and control for bias in cohort studies would be to compare Vaccine effectiveness estimates from seasons during which the vaccine component was severely mismatched relative to the circulating strain type, such as the 1997 to 1998 season, in which vaccine benefits were not detectable in a randomized, placebo-controlled clinical trial set in younger adults. So, reasonably, we would expect cohort studies to also measure 0% Vaccine effectiveness for that season but that did not happen (click here to view table 1); the substantial Vaccine effectiveness measured for this mismatched season strongly suggests self-selection bias and, by extension, considerable overestimation of vaccine benefits in the other seasons.

We also hypothesized in our paper that vaccine response may decline with increasing age. Fedson and Nichol also take issue with this hypothesis, citing cohort studies that find no age-related decline in Vaccine effectiveness. But if much of the all-cause mortality effect measured in cohort studies is in fact due to bias, then the results from different age groups could well suffer from this problem to various degrees and their argument would no longer be valid. While we believe there are insufficient data from clinical trials to address whether vaccine benefits decline with age, immunological studies confirm declining immune response to influenza vaccination in the elderly population.

A way forward

So where to go from here? Unfortunately, clinical trials have little to say about the effectiveness with which the vaccine prevents mortality among the elderly. The single-published, randomized, placebo-controlled trial set in the elderly found 58% vaccine effectiveness for prevention of laboratory-confirmed influenza-like illness. But this important study was largely set in a population of “young elderly”: ages 60 to 70. This trial was not sufficiently powered to assess whether the finding of 29% Vaccine effectiveness in “older” elderly or those older than 70 years truly implied reduced vaccine effect with age. Nor was it powered to assess mortality outcomes in this population. These limitations are crucial, because most influenza-related deaths currently occur among “older” elderly. Although clinical trials set in the “older” elderly population would be very useful to test alternative strategies, there is resistance due in part to the common perception of at least 50% all-cause mortality reduction with the vaccine currently in use.

The absence of “gold standard” clinical trials of the elderly, however, leads us right back to the need for cohort studies, including the need to adjust completely with self-selection bias. Most importantly, we hope that authors of such studies will follow the recent lead of Punam Mangtani, BSc, MBBS, MRCP, MSc, MD, and colleagues: to report on data from peri-influenza periods, and make adjustments in Vaccine

effectiveness estimates as needed. Under our disparity hypothesis, a frail, unvaccinated elderly cohort would mean that the unvaccinated elderly had, independent of influenza infectious periods, a substantially higher mortality rate than the vaccinated elderly, something that would lead to severely overestimated vaccine benefits. This is a testable hypothesis and would only require additional analysis of cohort study databases already collected.

We believe the question about the benefit of influenza vaccination on the overall population is extremely relevant to reach the ultimate goal: to ensure that the elderly are adequately protected against influenza. If there is a problem, it should be addressed through research leading to the development of more immunogenic vaccines, improvement of vaccine coverage in undervaccinated frail elderly populations, implementation of strategies for indirect protection through immunization of family and caregivers or through greater use of antiviral drugs.

For more information:

• Simonsen L, Reichert TA, Viboud C, et al. Impact of influenza vaccination on seasonal mortality in the US elderly population. Arch Intern Med. 2005;165(3):265-272.
• Bridges CB, Thompson WW, Meltzer MI, et al. Effectiveness and cost-benefit of influenza vaccination of healthy working adults: a randomized controlled trial. JAMA. 2000;284(13):1655-1663.
• Gross PA, Hermogenes AW, Sacks HS, et al. The efficacy of influenza vaccination in elderly persons. A meta-analysis and review of the literature. Ann Intern Med. 1995;123(7):518-527.
• Kang I, Hong MS, Nolasco H, et al. Age-associated change in the frequency of memory CD4+ T cells impairs long term CD4+ T cell responses to influenza vaccine. J Immunol. 2004;173(1):673-681.
• Mangtani P, Cumberland P, Hodgson CR, et al. A cohort study of the effectiveness of influenza vaccine in older people, performed using the United Kingdom general practice research database. J Infect Dis. 2004;190(1):1-10.
• McElhaney, JE. The unmet need in the elderly: Designing new influenza vaccines for older adults. Vaccine. 2005;23(Suppl 1):S10-S25.
• Nichol KL. The efficacy, effectiveness and cost-effectiveness of inactivated influenza virus vaccines. Vaccine. 2003; 21(16):1769-1775.
• Nordin J, Mullooly J, Poblete S, et al. Influenza vaccine effectiveness in preventing hospitalizations and deaths in persons 65 years or older in Minnesota, New York, and Oregon: data from 3 health plans. J Infect Dis. 2001;184(6):665-670.
• Patriarca PA, Weber JA, Parker RA, et al. Efficacy of influenza vaccine in nursing homes. Reduction in illness and complications during an influenza A (H3N2) epidemic. JAMA. 1985;253(8):1136-1139.
• Thompson WW, Shay DK, Weintraub E, et al. Mortality associated with influenza and respiratory syncytial virus in the United States. JAMA. 2003;289(2):179-186.
• Vu T, Farish S, Jenkins M, Kelly H. A meta-analysis of effectiveness of influenza vaccine in persons aged 65 years and over living in the community. Vaccine. 2002;20(13-14):1831-1836.
• Lone Simonsen, Cecile Viboud, William Blackwelder, Robert Taylor and Mark Miller are with the National Institutes of Health (NIH) in Bethesda, Md.

• Dr. Simonsen is a senior epidemiologist at the National Institute of Allergy and Infectious Diseases at the NIH in Bethesda, Md.