A recently published study by JAMA Network Open reveals the effectiveness of COVID-19 vaccines. The vaccines show approximately 55% effectiveness against hospitalizations and a 50% reduction in emergency department and urgent care visits related to COVID-19.
The study gained attention after political appointees from the Trump administration blocked its publication in a Centers for Disease Control and Prevention (CDC) journal. They criticized the study’s design for being susceptible to assumptions that might lead to biased results. Despite this, experts in public health endorse the design as reliable, having been used for decades to assess vaccine efficacy.
Natalie Dean, a biostatistics expert from Emory University, stressed the importance of continuing to publish estimates of vaccine effectiveness, especially given the evolving viral strains and varying immunity levels.
The CDC’s Morbidity and Mortality Weekly Report initially planned to publish the study. However, it was held for further review by acting CDC Director Jay Bhattacharya. This pause aimed to address concerns about the study’s methodology rather than permanently block its publication. Subsequently, the study authors chose to publish through an external journal.
The study employs a ‘test-negative design,’ a method that evaluates vaccinated versus unvaccinated individuals who visit hospitals with respiratory illnesses. This approach assesses the likelihood of testing positive for COVID-19. The methodology has been vetted and used in numerous respected journals, including Pediatrics and the New England Journal of Medicine.
Bhattacharya criticized the methodology for relying heavily on assumptions, potentially skewing results due to factors like prior infections and varying patient behaviors. However, advocates of the design argue it addresses those differences effectively. While recognizing no study design is perfect, they note that no feasible alternative from U.S. Health and Human Services has been suggested for real-time vaccine efficacy analysis.
Recently, the CDC hosted a forum discussing the strengths and weaknesses of such studies. Expert speakers, including Dean, highlighted the methodology’s advantages, although one critic, Martin Kulldorff, contributed to the discourse. Kulldorff argued long-term studies should have been employed for evaluation, questioning the inclusion of diverse disease groups within the study design.
During the forum, audience responses reflected the urgency posed by the pandemic environment affecting study choices.
The Associated Press Health and Science Department produced this content, supported by grants from the Howard Hughes Medical Institute’s Department of Science Education and the Robert Wood Johnson Foundation.