Efficacy vs Effectiveness

"Efficacy and effectiveness are related to each other, but they're not the same thing," vaccine experts say, "it’s crucial not to mix them up."

Efficacy is the degree to which a vaccine prevents disease, and possibly also transmission, under ideal and controlled circumstances. Effectiveness meanwhile refers to how well it performs in the real world.

You might assume that 95 out of every 100 people vaccinated will be protected from Covid-19. But that’s not how the math works.

“Vaccines don’t save lives,” says A. David Paltiel, a professor at the Yale School of Public Health. “Vaccination programs save lives.”

Vaccines don’t protect only the people who get them. Because they slow the spread of the virus, they can, over time, also drive down new infection rates and protect society as a whole. Scientists call this broad form of effectiveness a vaccine’s impact. But even a vaccine with extremely high efficacy in clinical trials will have a small impact if only a few people end up getting it.

Pfizer and BioNTech announced that their vaccine had an efficacy rate of 95 percent. Moderna put the figure for its vaccine at 94.5 percent. Sputnik claimed their efficacy rate was over 90 percent, and the Johnson & Johnson vaccine say they're 66% effective.

“These are game changers,” says Dr. Gregory Poland, a vaccine researcher at the Mayo Clinic. 

From the headlines, you might well assume that these vaccines will protect 95 out of 100 people who get them. But that’s not actually what the trials have shown.

Exactly how the vaccines perform out in the real world will depend on a lot of factors we just don’t have answers to yet.

Here’s what you need to know about the actual effectiveness of these vaccines:

The fundamental logic behind today’s vaccine trials was worked out by statisticians over a century ago. Researchers vaccinate some people and give a placebo to others. They then wait for participants to get sick and look at how many of the illnesses came from each group.

In the case of Pfizer, for example, the company recruited 43,661 volunteers and waited for 170 people to come down with symptoms of Covid-19 and then get a positive test. Out of these 170, 162 had received a placebo shot, and just eight had received the real vaccine.

From these numbers, Pfizer’s researchers calculated the fraction of volunteers in each group who got sick. Both fractions were small, but the fraction of unvaccinated volunteers who got sick was much bigger than the fraction of vaccinated ones. The scientists then determined the relative difference between those two fractions.

Scientists express that difference with a value they call efficacy. If there’s no difference between the vaccine and placebo groups, the efficacy is zero. If none of the sick people had been vaccinated, the efficacy is 100 percent.

A 95 percent efficacy is certainly compelling evidence that a vaccine works well. But that number doesn’t tell you what your chances are of becoming sick if you get vaccinated. And on its own, it also doesn’t say how well the vaccine will bring down Covid-19.

It’s possible that the effectiveness of coronavirus vaccines will match their impressive efficacy in clinical trials. But if previous vaccines are any guide, effectiveness may prove somewhat lower.

The mismatch comes about because the people who join clinical trials are not a perfect reflection of the population at large. Out in the real world, people may have a host of chronic health problems that could interfere with a vaccine’s protection.

'The Centers for Disease Control and Prevention' has a long history of following the effectiveness of vaccines after they’re approved. The agency posts information on its website about its plans to study the effectiveness of coronavirus vaccines. It will find opportunities to compare the health of vaccinated people to others in their communities who have not received a vaccine.

The clinical trials run by Pfizer and other companies were specifically designed to see whether vaccines protect people from getting sick from Covid-19. If volunteers developed symptoms like a fever or cough, they were then tested for the coronavirus.

But there’s abundant evidence that people can get infected with the coronavirus without ever showing symptoms. And so it’s possible that a number of people who got vaccinated in the clinical trials got infected, too, without ever realizing it. If those cases indeed exist, none of them are reflected in the 95 percent efficacy rate.

People who are asymptomatic can still spread the virus to others. Some studies suggest that they produce fewer viruses, making them less of a threat than infected people who go on to develop symptoms. But if people get vaccinated and then stop wearing masks and taking other safety measures, their chances of spreading the coronavirus to others could increase.

“You could get this paradoxical situation of things getting worse,” says Dr. Naor Bar-Zeev, an epidemiologist at the Johns Hopkins Bloomberg School of Public Health…

Food for thought!