Using epidemiological data to build public confidence

Sure. The work that we’re doing with FDA is part of what’s called the BEST Initiative, which is an active surveillance system that uses claims or electronic medical record data streams to monitor the safety or effectiveness of certain biologic products that the FDA is responsible for monitoring. In conjunction with a whole series of partners that are part of this BEST Initiative, we conduct targeted queries in response to questions that the FDA has — including the always top-of-mind questions such as:

• Are the COVID-19 vaccines safe, and are they effective?
• How do they work on older people, younger people, people with immunocompromised status and people who get a booster?
• Is the booster effective?
• Is the primary series effective?
• Do you need the full primary series, and how long is it effective?

To answer those questions, we run our real-world data through analytic models developed by our team of epidemiology experts. We produce analyses according to pre-specified protocols (that are publicly posted) to send to the FDA, which they compile with responses from their other partners, helping them build a clear picture about the vaccines’ safety and effectiveness.

At this point, most of the focus remains on the COVID-19 vaccines. But this is a very versatile setup that could be used for almost any product that the FDA Center for Biologics Evaluation and Research (CBER) evaluates.

As a researcher, this assignment was difficult because none of the children in that age range in our data had received the vaccine. What we could do was look at the risk of about 17 FDA-monitored adverse effects related to COVID-19 vaccines. Since this new population hadn’t received the vaccine, we looked at the historical incidence of these 17 different conditions and the rates at which they happen, even without the COVID-19 vaccines. From there, the FDA compared results from our analysis to the trial results that the manufacturers submitted to the FDA and used this benefit-risk assessment to understand what we could likely expect moving forward.

The FDA essentially ran a risk/benefit analysis across the U.S., under the assumption that the vaccine was available for use. Based on the background incidence and clinical trial data of these various adverse events within the appropriate age range, both the risks and benefits could be quantified. A comparison of the risks to the benefits of vaccination helps answer the question of whether the benefit of vaccination is worth the risk.

We’re currently monitoring the vaccine in both adults and children. We went through a very comparable exercise when we analyzed the 5- to 11-year-old population in the fall of 2021, before those children could receive a vaccine. We provided a similar analysis of the background risk, and the FDA used that in conjunction with its trial data. This comparison allowed it to conclude that the benefits outweighed the incremental risks of vaccination.

In terms of differences with this age group, we had the advantage of providing historical data in advance of the authorization. We also created rules in our research design to deal with anomalies in the data. For example, claims are sometimes accidentally submitted for the patient’s sibling, which can create errors in the data. While rare, this does happen, so we added these new rules to mitigate such errors. There’s a whole range of anomalies that can occur in the data, and those rules are part of our research design that help address the underlying limitations of claims data that are used for public health surveillance.

Let’s assume that the COVID-19 vacine is close to 75% effective for children at preventing severe COVID-19 disease. This would mean that children who are vaccinated have a 75% lower chance of hospitalization when contracting COVID-19. Our internal work on COVID-19 effectiveness will bring a sample size in the range of 100,000 vaccine recipients and 100,000 non-vaccine recipients, allowing us to measure this effectiveness with good precision. Even if the absolute risk of hospitalization is much lower in kids, the sheer size of the Optum data helps us identify the small differences in hospitalizations among vaccinated and unvaccinated kids.

We have so many different ongoing analyses. We worked through a set of analyses that contributed to the CBER Vaccines and Related Biological Products Advisory Committee at the FDA. Then there’s this effectiveness analysis, which I’m really excited about because the short answer is that the vaccines work. With real-world data, we can see how effective the vaccines are in preventing hospitalization with COVID-19. And there’s still a lot more to learn.

We’re next going to be stratifying the data by age and sex and a whole range of factors that I mentioned earlier about which vaccine you get, how long it’s been since you’ve had the vaccine, and what kind of benefit you get from the booster. We’ll be asking and answering these questions with the real-world data that we have access to through Optum.

These analyses will help us look beyond the tens of thousands of patients included in the rigorous clinical trials that the manufacturers have run, in order to monitor effectiveness over even larger populations, with longer follow-up; this kind of longitudinal study may help us identify additional factors that would otherwise be hard to spot until we see them in the real world.

Not only are we excited about this progress at Optum, but the FDA is as well. We’re on a fast track to bring information public so that people continue to be informed about the monitoring and effectiveness of the COVID-19 vaccine in situations of routine care.

It’s a mix of factors. Initially, the FDA sent out a request for proposals for involvement in the BEST Initiative. They needed a group that was embedded within a data source, as opposed to a group where the researchers were experts at the applied methods. But the data source is external.

Our Optum Epidemiology group offers the best of both worlds, since a few people on our team have faculty appointments, including my own with Harvard. These connections contribute to our ability to maintain academic rigor and operating standards.

Additionally, because Optum has the epidemiology expertise within the data environment, we could offer some advantages in terms of speed. We’re able to see claims for adverse events come through just a couple of weeks after a vaccination or adverse event has happened, which is a shorter data lag than you see with many other claims data sets. Ultimately, our data, scale and expertise all played roles in making Optum an appropriate fit for the FDA’s ask.

It’s personally gratifying as a public health researcher to be working on something that has almost immediate effects on the population, including friends and family. I live near an elementary school that my kids have gone to, and as I walk by and catch up with some of the teachers, they show so much excitement for these vaccine authorizations in children. When the vaccines were first made available to 5- to 11-year-olds, it opened up a whole range of ways to interact with the kids.

We’ve developed a wonderful relationship between Optum and the FDA. The FDA has learned what’s possible through working with us, and I think they’ve been pleasantly surprised with how robust the system is that we have set up — including not only Optum, but all the participants in the BEST Initiative. It’s proving out what a modern biologics and vaccine monitoring system can look like.

I think the FDA is looking forward to the future of trying to see how far they can extend this work into other areas of research, supplementing what already exists, and applying it to a whole range of other products that are part of their responsibility.

The progress being made by Dr. Seeger’s team, along with the support from other branches of Optum, has helped inform analysis behind many of the FDA’s research questions. These steps forward through the pandemic have been crucial and continue to lay the groundwork for future vaccine and treatment decisions beyond COVID-19.