Dilly-Dally Delay?

Note to Scottish readers. Jab is the word that Sassenachs use to mean jag.

Two months is a long time in COVID-19 science

On 8 December 2020, the first analysis of the protective effects of the Oxford/AstraZeneca vaccine were published in The Lancet. (I shall refer to this paper as OX/AZ1.) A mistake in vaccine delivery had meant that a number of subjects were initially given a lower than intended dose. This led to there being two treatment groups: LD/SD (low dose followed by standard dose) and SD/SD (standard dose followed by standard dose). The efficacy of the vaccine was confirmed but, bafflingly, the LD/SD combination showed greater apparent efficacy than the SD/SD dose. The authors had this to say:

Efficacy of 90·0% seen in those who received a low dose as prime in the UK was intriguingly high compared with the other findings in the study. Although there is a possibility that chance might play a part in such divergent results, a similar contrast in efficacy between the LD/SD and SD/SD recipients with asymptomatic infections provides support for the observation...(Ref 1 p10)

See Complicated COVID comparisons for a discussion of these results.

Less than two months later, the team were able to present an updated analysis based on further data that had accumulated. In this second report (which I shall refer to as OX/AZ2) they concentrated in particular on the dosing interval. They now had this to say:

The analysis presented here provides strong evidence for the efficacy of two standard doses of the vaccine (SD/SD), which is the regimen approved by the MHRA and other regulators. Following regulatory approval, a key question for policymakers to plan the optimal approach to roll out, is the optimal dose interval, which is assessed in this report through post-hoc exploratory analyses. Two criteria which contribute to decision-making in this area are the impact of prime-boost interval on protection after the second dose; and the degree to which the vaccinated individual is at risk of infection during the pre-boost period, either due to reduced efficacy with a single dose, or rapid waning of efficacy prior to the second vaccination. (Ref 2)

In the conclusion of the abstract to the new paper the authors state.

Vaccination programmes aimed at vaccinating a large proportion of the population with a single dose, with a second dose given after a 3 month period may be an effective strategy for reducing disease, and may be the optimal for rollout of a pandemic vaccine when supplies are limited in the short term. (Ref 2)

The interval between these two stories is rather less than the 12 weeks now being proposed between the original vaccination and the booster. In this blog, I propose to consider to what extent the new analysis answers this question of the optimal dosing interval and justifies this conclusion.

A Dabbling Dilettante

I have no experience in vaccine trials. My experience in clinical trials is limited to therapeutic interventions. I shall make some critical remarks of the AZ/Oxford results but the reader should be clear of two things. 1) My perspective is that of an interested amateur 2) Were I to be offered the AZ/Oxford vaccine, I would accept, whetever the proposed dosing interval.

Plotting the Problem

Figure 1. Hazard rates over time

Figure 1 shows the sort of situation one might envisage. What is plotted are hazard rates of COVID infection, that is to say the sort of instantaneous risk of infection over time an individual might have, given that they have not been infected so far.

Time 0 is the time of adminstration of the first dose. If an individual is not vaccinated the hazard is supposed to be constant. Note that what is illustrated here is the hazard rate due to vaccination. The rate may be varying for many other reasons, for example the current level of infection or measures to contain the epidemic. This is one excellent reason for observing the principle of concurrent control. However, the causal effect of a vaccine that is not given must be constant and that is what is shown here.

Vaccination leads to a rapid reduction in the hazard and administration of the second dose leads to a further reduction. It seems to be the case that a longer interval leads to a greater reduction of risk. Obviously, until the booster dose is given there is no difference between the two policies. Hence, the two vaccine curves are the same up to this point. After four weeks they separate. For a given individual, it seems that there is a trade-off, as described in OX/AZ2. In the long term there is greater protection by delaying the second dose but there is only the protection afforded by the first dose during the period one waits to be given the booster.

Note that these are not meant to be real values but just illustrations of a general point. The sort of thinking illustrated in the figure helps us to understand what we need to know but making good that deficit of knowledge requires data and analysis and this, of course, is what the authors of OX/AZ2 were trying to provide. I shall consider that analysis in due course but for the moment look at what we might hope to expect from clinical trials designed to answer the very question: what is, 'the optimal for rollout of a pandemic vaccine when supplies are limited in the short term'?

Trial attempts

The intention to treat approach is a natural one for comparing policies. I shall now consider various alternative designs. I shall assume that the case for overall vaccine efficacy has already been made and that there is no need for an unvaccinated control group and instead that two groups will be compared early and late..

Design one. Parallel group. Double blind. No limit on supplies.

In this designs all subjects are given the first jab and are randomised to receive the booster shot either early (after four weeks) or late (after twelve weeks). In order to maintain blinding, however, the method of dummy loading will have to be used, and those assigned to the early regime will have to recieve a dummy shot at twelve weeks and those assigned to the late shot will have to receive a dummy shot at four weeks.

Important point: All cases will be counted according to the group subjects were assigned to, whether or not they got the booster shot. In particular, cases will not be removed from the late group if they occurred between four and twelve weeks,

Design two. Cluster randomised. Double blind. No limit on supplies.

The problem with Design one is that it does not fully address the 'herd immunity' question. This is something that applies (if it applies at all) at the level of whole communities. Thus to compare effects on transmission of infection, the effect on whole communities would have to be studied. This would imply assigning whole communities (say local authority areas) at random to one policy or another and comparing them.

Important points: First, all cases within the communities after initiation of the policy would have to be compared between communities. Second, cases within communities could not be treated as independent. Variation would have to be judged by comparing different communities treated under the same policy. See To Infinity and Beyond for a discussion.

Design three. Cluster randomised. Open label. Supplies are limited.

Figure 2. Hazard rates over time showing the cross-over point at 24 weeks.

However, even design two does not address the practical problem of limited supply. Much of the discussion of delaying the administration of the booster dose has considered the avantage of being able to give a first shot to more subjects. Figure 2 is the same as Figure 1 but with the cross-over point added: the point at which the hazard rate for the late booster policy is lower than for the early booster policy. (The reader is reminded that these plots do not represent real data.) If one looks at Figure 2 it is clear that there is a trade-off to the individual as regards timing (if a delay eventually provides more protection). Given the values represented by the plot, the early booster has the advantage between weeks 4 and 24 and the later booster thereafter. Neither has any disadvantage in terms of protection to one jab only. However, the differences between the two pale into insignificance compared to the advantage of having a single shot compared to having nothing at all. Here the single jab curve needs to be compared to the no vaccine one.

Now suppose that given limited supplies, the policy of giving the booster late allows more subjects to receive their first shot early. This could be compared by randomising communities of similar size to receive a given amount of vaccine and then proceed to vaccinate as many persons as possible using the policy and therefore subject to the constraint that the booster deadline to which the given community had been randomised would have to be adhered to.

Important points. First, I judge that such a trial would have to be run as open label for practical reasons. This implies that great care has to be taken in handling measurement and interpreting data. See Blind Date for a discussion. Second, on the other hand, it could also be argued that since the practical aspects of two policies are being compared, blinding might interfere with practicalities that no longer made it desirable. If scarce rescources include personnel, having to deliver dummy shots, which would be needed to maintain blinding, may interfere with implementing the chosen policy.

Model behaviour

A particular disadvantage with designs two and three is that they would be nearly impossible to run. That is a serious objection! Nevertheless, they remain useful conceptually in allowing us to judge what we can get by what we would like to get. If the parallel group trial allowed us to estimate the relative protection over time, then by varying assumptions and applying a suitable model, strategies could be compared over various scenarios. An evidence-based medicine purist might demur to accept this approach but I would part company with them there. The background infection rate has changed rapidly and considerably during the pandemic and without making some assumptions and accepting some 'translation' of results, reasonable decisions cannot be made.

Making do with what we have

Unfortunately, we do not have such data and therefore such data cannot be considered in the OX/AZ2 study. Subjects were not randomised between dosage intervals. The data are the result of an accident of administration, as the authors make clear. That might seem to be regrettable, but had the accident not occurred, then nothing whatsoever could be said about the effect of interval on risk of infection. The situation thus may be similar to one treated by Efron and Feldman in a famous paper (Ref 3) where patients showed different degrees of compliance. Doses of drug actually taken thus varied and this, in a sense, formed a natural experiment. As Efron and Feldman were able to show, however, there was also a 'dose-response' curve by compliance in the placebo group. As Efron and Feldman put it

The compliance-response regression for the Treatment group shows a smooth increasing effect of the drug in cholesterol level with increasing compliance. However, a similar, though less dramatic, compliance-response regression is seen in the Control group. This article investigates the recovery of the true dose-response curve from the Treatment and Control compliance-response curves. A simple model is proposed, analyzed, and applied to the LRC-CPPT data. Under this model, part but not all of the true dose-response curve can be estimated.

The MHRA itself had done an analysis of the relationship between dosing interval and vaccine efficacy, which they reported on 30 December 2020 and I discussed in Of Weighting and Waiting This was based on the data then available and mainly considered antibody levels rather than infections .

In OX/AZ2 the authors are able to add further data and also look at infections. They present many interesting analyses and I shall not cover them all or all points concerning them. The interested reader is urged to read the original paper.

In my further analysis, I take it as being accepted without question that those given the second shot of the vaccine at a given interval must be compared to those in the control group given their second shot at the same interval. This is analogous in spirit to what Efron and Feldman did in their paper.

Cautious about causes

In OZ/AX2 the authors do not, as far as I am aware, directly compare the apparent efficacy of doses of the vaccine according to the interval since the first. What they have instead is the following statement.

In the SD/SD group, after the second dose, efficacy was higher with a longer prime-boost interval: VE 82.4% 95%CI 62.7%, 91.7% at 12+ weeks, compared with VE 54.9%, 95%CI 32.7%, 69.7% at <6 weeks.

This is not a direct comparison and what we can note is that the upper 95% interval for efficacy at < 6 weeks is 69.7% and so higher than the lower level at 12+ weeks of 62.7%. In other words, the intervals overlap.

However, what we are really interested in is the confidence interval for the difference between the two. The condition that this would not include zero is (approximately) that the 83% intervals do not overlap. (See Straining for Effect for a discussion.) What Figure 3 shows is a mean-mean plot for the risk difference scale of the sort covered by Jason Hsu in his classic book on multiple comparisons (Reference 4).

Figure 3. Mean-mean plot comparing risk differences according to interval between the first and second dose of vaccine. No adjustment for multiplicity has been made.

There are four differences compared to the local control for each of the time intervals. These difference are the first step to making unbiased comparisons between time intervals. If these four differences are compared to each other, there are six possible pairwise comparisons. (This double difference, first to control, then between time intervals, is very much in the spirit of network meta-analysis.) I have used the risk difference scale rather than the vaccine efficacy scale, as it is easier to work with. Each comparison is represented as a point in the two-dimensional space with the values for the four treatments plotted in the X and the Y dimensions. The diagonal line rising from bottom left to top right is the line of equality. The further a point lies from the line, the greater the difference. Also shown are the confidence intervals as diagonal lines slanting at right angles to the line of equality. For five of the comparisons, where the lines are drawn in red, these intervals are intersected by the line of equality. The corresponding differences between time intervals are not 'significant' at the conventional 5% level. For one of the comparisons, however, the one the authors of OX/AZ2 highlight, that of the longest time interval compared to the shortest, the difference is indeed significant. The line for this confindence interval is drawn in blue.

In calculating these comparisons I made no adjustment for multiplicity. If I do, I get the result illustrated in Figure 4. It can be seen, as is only to be expected, that the confidence intervals are wider. However, the same comparison is (just) significant.

Figure 4. Mean-mean plot comparing risk differences according to interval between the first and second dose of vaccine. A Bonferroni adjustment has been applied.

In conclusion

The OX/AZ2 report concludes with a passage as follows, which I have already quoted:

Vaccination programmes aimed at vaccinating a large proportion of the population with a single dose, with a second dose given after a 3 month period may be an effective strategy for reducing disease, and may be the optimal for rollout of a pandemic vaccine when supplies are limited in the short term.

This strikes me as being probably reasonable from the public health point of view although it would, in my opinion, require further modelling to justify it. Note that it is not necessary to believe that the longest interval provides better protection than the shortest interval for this to be true. Suppose the longer interval gives the same protection as the shorter one once the booster has been given. From an individual's perspective it is logical to have the booster sooner. However, from the public health perspective, if more individuals can be given their first shot if the second is delayed for others a longer interval may make sense.

I say may make sense because everything depends on the calculations and we have yet to see these.

References

1. Voysey M, Clemens SAC, Madhi SA, et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet. 2021;397(10269):99-111.

2. Voysey M, Costa Clemens SA, Madhi SA, et al. Single Dose Administration, And The Influence Of The Timing Of The Booster Dose On Immunogenicity and Efficacy Of ChAdOx1 nCoV-19 (AZD1222) Vaccine. Lancet preprint

3. Efron B, Feldman D. Compliance as an Explanatory Variable in Clinical-Trials. Journal of the American Statistical Association. 1991;86(413):9-17.

4. Hsu JC. Multiple Comparisons Theory and Methods. Boca Raton: Chapman & Hall/CRC; 1996.