We Cannot Trust the CDC Estimates of Flu Vaccine Effectiveness

The effectiveness of the flu vaccine in the 2022-2023 season was recently estimated by a CDC-supported study. However, criticism from the NIH director highlights significant issues with the study's design and results, particularly concerning biases that may affect the interpretations of vaccine effectiveness.

1. Test-Negative Design Flaws: The study used a test-negative case-control design which has inherent biases. While it aimed to account for healthcare-seeking behavior, it introduced collinearity bias, making the reliability of results uncertain.

2. Confounding Background Risk: Infection risk varies over the flu season, being higher in October to December and lower from January to March. The vaccinated and unvaccinated groups were not evenly distributed across these periods, likely skewing results since vaccination rates increased over time as the risk lowered.

3. Immortal Time Bias: The study excluded cases occurring within 14 days post-vaccination, which is a known source of bias called immortal time bias. This exclusion may make vaccination appear less effective because it undercounts early events that could show the vaccine’s protective effects.

4. Effectiveness Estimates: Initial effectiveness estimates may significantly decrease when correcting for biases. The actual effectiveness could potentially drop from around 30-40% to 20% or lower.

5. Questionable Results in Hospitalization Rates: The study showed lower adjusted effectiveness for hospitalization compared to outpatient encounters, which is atypical. The authors' explanations for this inconsistency are deemed inadequate.

6. Healthy Vaccinee Bias: The study indicates the vaccinated group was generally healthier than the unvaccinated group, which can distort effectiveness estimates, attributing some protection to overall health rather than vaccination.

7. Age-Related Effectiveness: Unexpected findings emerged regarding vaccine effectiveness in different age groups. Typically, older adults show lower effectiveness, yet in this study, they appeared to fare better regarding hospitalization. This inconsistency requires further explanation.

8. Outcomes for Hospitalized Patients: The study assessed outcomes among hospitalized flu patients by vaccination status but lacked adequate adjustments for baseline characteristics. It raised concerns about the higher fatality rates in vaccinated elderly patients without establishing if the vaccine provided any real benefit.

The critique of this flu vaccine effectiveness study reveals substantial methodological flaws that raise doubts about its reliability. These concerns include biases related to time trends, health status, and the study design itself. The findings suggest the need for alternative study designs or methodologies, as randomized trials cannot be conducted. Past analyses of flu vaccine effectiveness have also yielded uncertain results, questioning the overall benefits of annual flu vaccinations. 

https://brownstone.org/articles/we-cannot-trust-the-cdc-estimates-of-flu-vaccine-effectiveness/