Shop Talk
November 25, 2020
It looks like vaccines are on their way, which should give us something to be grateful for on Turkey Day. Earlier this month, Pfizer reported preliminary results from its COVID-19 vaccine trial and announced that it’s more than 90% effective. Since then, two other companies announced that their vaccines are 95% (Moderna) and 90% (AstraZeneca) effective. After a collective hip-hip-hurray, we started to wonder how Pfizer concluded that their vaccine is 90% effective. Using only 3 values reported in the press, we computed it ourselves. In this issue, we’ll show you how we did it and explain what 90% effective means for number needed to treat (NNT).
The Pfizer vaccine trial was a randomized controlled trial (RCT) with an experimental group that got the vaccine and a control group that got a shot of salt water. Participants were followed over time, and the number of new COVID-19 infections was recorded.
From the press release, we learned the following:
43,538 participants were enrolled in the trial
94 got infected with COVID-19
The vaccine was at least 90% effective
Assuming ½ of the participants got the vaccine (n=21,769) and ½ got the sham (n=21,769), we fiddled with the numbers to compute a control and experimental event rate (CER, EER) that gave us a 90% effect. Recall that CER is the rate at which a bad thing happens in a group that is NOT treated. EER is the rate at which a bad thing happens in those who receive treatment.
If we assume that 86 out of 21,769 unvaccinated controls got COVID-19, the CER is 0.40%
CER = 86/21,769*100 = 0.40%
If we assume that 8 out of 21,769 vaccinated participants got COVID-19, the EER is 0.04%
EER = 8/21,769*100 = 0.04%
So, things are looking good for this vaccine. The EER is 10 times LOWER than the CER.
Now we compute the relative risk reduction (RRR).
RRR = (CER-EER)/CER*100
RRR = (0.40 – 0.04)/0.40*100
And, bang… There it is à 90%
Now we have a rough idea of what the Pfizer data looked like.
94 participants got COVID-19
8 of them were vaccinated
86 were unvaccinated
Let’s wrap up by computing number needed to treat (NNT). Recall that NNT is the number of people who must be treated to prevent 1 bad outcome. Here, the bad outcome is coming down with COVID-19.
NNT = 1/(CER-EER)§
NNT = 1/(0.004 – 0.0004)§
NNT = 278
So, we’ll need to vaccinate 278 people to prevent 1 case of COVID-19.
If that number surprised you, you’re not alone. We thought it was big, too. Why so many people? After all, isn’t the vaccine 90% effective?
NNT is affected by CER, which is the likelihood of getting COVID-19 even if you’re NOT vaccinated. Among those who did NOT get vaccinated, most did NOT get sick. Thank heavens! In fact, over 21,000 unvaccinated people did NOT get sick. When the chance of disease is not that high, you have to vaccinate a lot of people who would NOT get sick in order to prevent illness in those who WOULD get sick.
So, get ready to roll up your sleeve. We’ll need to vaccinate a lot of people to get this thing under control.
In case we’ve sapped your optimism, let’s consider the alternative.
To push down the NNT, we’d need to see a higher CER. That means more cases of COVID-19, more quarantine, more sick people, fuller hospitals, and more death. Given those options, we’ll accept a high NNT, stay 6’ away, and wait for our turn to get a shot in the arm.
Thanks for reading. You can find all our blogs and support our work at https://www.buymeacoffee.com/ptworkshop/posts. Rumor has it that turkey and dressing have no effect on the COVID-19 event rate, even when consumed without a mask. So, gobble it down.
§Note that control and experimental event rates must be in decimal format to compute number needed to treat.