Exemplar project/tutorial. Sample size calculation for multi-arm Clinical trials using R

One of the most important aspects of any Clinical study is determining the required sample size for the study to work out in the end. This is a critical in multi-arm and multi-stage Clinical trials where sample size calculation is a bit more complex. However in this exemplar tutorial i will try to simplify the process using R[1] and packages 'MAMS'[2] and 'multiarm'[3]. For this tutorial we will use RStudio IDE [4]

So lets start:

The function we will be using is very straightforward mams() and set arguments for it.

Here are some syntax rules to learn first.

• K - Use this argument to set the number of experimental arms (note that these do not include any control arms). So if we set the K to 3, the actual number of defined arms in a study are at least 4, because we need at least one control arm.
• J - with J we can se number of stages (lets keep it at 1 stage for now).
• alpha - Use this to set the level of statistical significance. The general standard is 0.05 in academic world, which mean all values of p<0.05 will be considered as significant. So the design of the sample size calculation is based on 0.05 threshold. If we were to lower the threshold the larger the required sample size would be and vice versa.
• delta - This is the effect of interest we want to identify. Note this is defined in communication with Medical experts, but there are also statistical scales on what the small, medium or large effect is. The smaller the effect of interest we are looking for, the larger the required sample will be.
• delta0 - The effect that we consider as not interesting. This is very important for the algorithm to differentiate a spectrum of uninteresting vs interesting effects.
• sd - sample size is also determined using the standard deviation of the effect. Keep in mind, the larger the standard deviation, the larger variability in the data and the larger the required sample size will be needed.
• power - This is on of the most important arguments. Setting the right power can define the success or failure of the sample size calculation. Most researchers think that 80% or 0.8 power would be enough, but this is not the case most of the time. I will initially set this argument to 90% or 0.9.
• For now just notice how i set p=NULL and p0=NULL. Will talk about this later.

One of the biggest mistakes at this point would be to output the results directly. We have to account for potential uncertainty around the dropout rates. Typically the dropout rate is defined in communication with medical experts running the Clinical trial, but let say for this tutorial the they gave me a feedback that expected dropout rate is 10%. I typically like to add a bit more dropout on top what the experts would say to be more certain. So i will consider a potential dropout to actually be 15%. You can observe this on the part of the code where i multiplied the output results using onsestage$n *1.15 and onsestage$N *1.15. These would correspond to the sample sizes with added 15% dropout rates.

You can see that the initial per arm calculated sample size was 102 and 408 overall ( as i said this package K is meant for experimental arm and plus one control arm mean per arm * 4)

But this would not be a good estimate if i didnt add 15% - the dropout rate, so the final per arm required sample size is actually 118 and 472 overall. So to interpret this we have to get back to the code - K was 3, J was 1. For a one stage 3-arm design with 0.44 effect and standard deviation of 0.9 it would be needed to have at least (note the word at least) 118 subject in each of the arms or 472 overall to achieve 90% Statistical power. I said at least because the sample could be larger and it would still achieve the 90% power or more

Now lets try to build a two stage design with 1:2 allocation ratio.

Before i start explaining the entire block of code, take a look how i set again p=NULL and p0=NULL. If i didn't set these and still used the delta argument i would receive an error which would to tell me to specify this.

Now to the entire block of code...

So this time, K=4, J=2 for 4-arm two stage design. I will keep the same alpha, but for multiple stage designs and multiple groups i prefer two approaches, either to have a a high powered calculation or add a multiple correction method. For this block of code, i will add the 0.98 power and will complement that with lowering the effect size a bit compared to the defined one (0.44). This will make sure the sample size i calculate can compensate for some of the uncertainty around the multi stage design. Let see the results...

The final sample size with added dropout rates is 152 per arm and 608 overall for stage 1 but for stage two its actually 1320 overall with accounting the allocation ratio. One important aspect that i used in both designs is the ceiling() when calculating the final per arm sample size. this function will round the values to have integer number of subjects, but on the top side. Do not use round() function for this as you might end up with -1 subject calculated if the decimal places for rounding are bellow 0.5 for the last subject.

Next lesson - using another pacakge - 'multiarm' and adding the multiple comparison correction. Note to install this package we will need to use devtools and install_github() fuunction.

Next - creating the one stage 7 arm design with Dunnett's multiple comparison correction. Check the code-block bellow...

Notice how i had to add 7 sigmas eventough K=6. Exactly as in the 'MAMS' package K=6 means this is at least 7 arm design because we need at least 1 control group which is not included in K. This means i have to define sigma or standard deviation for 6 experimental arms + 1 control arm. I will make all sigmas 0.9 and have other setting same as before, but you may notice new syntax now. 'beta' is used to define the level of power but in a reversed setting compared to 'MAMS' where we had power argument set to 0.9 or 0.98. Here we have 'beta' which is used to set 1-power. This means if i set beta=0.1, the power will be 0.9. The power argument in 'multiarm' is used to define the type of Statistical power (eg. Conjunctive vs Disjunctive). For this tutorial i will use Conjuctive power argument.

The most important part of the syntax in this case is the argument for multiple comparison correction which is essential with 7 arms. To define the correction simply type : 'correction=' . I will use Dunnett's correction which is well optimized for multi-arm Clinical trials, so 'correction=dunnett's' will do the work.

Lets finalize the calculation by adding the 15% dropout rate and view the results...

The per arm required corrected sample size is 176 and 1232 overall for the study. Now lets try another correction method - Bonferroni multiple testing correction method.

As you can see i changed the correction argument to 'correction=bonferroni'. The Bonferroni correction is more conservative than Dunnett's, so we can expect a larger required sample size using this method.

Lets check the result...

As expected the required sample size is larger with Bonferroni correction, so the final result for this hypothetical one stage 7 arm trial is 184 per arm or 1288 overall.

In this article different multi arm design packages use principle were shown in RStudio and using R programming language. In the next tutorial in Sample size calculation series, Simulations for Sample size requirements will be the main theme. Thank for reading!

By Darko Medin,

Biostatistician and a Data scientist

References :

1.https://www.r-project.org/

2.https://cran.r-project.org/web/packages/MAMS/MAMS.pdf

3.https://mjg211.github.io/multiarm/

4.https://posit.co/