The secrets of a successful manuscript publication- Part 3: Importance of sample size and statistical analysis plan

Continuing from part 2, let us now look at the importance of sample size and statistical analysis plan

Sample Size: You need to provide the statistician details of the focus, objective, and hypothesis of your study and share some available literature on the topic so that the statistician can suggest an appropriate sample size with a rationale for the same. It is always useful to provide the statistician with recent literature and publications on studies whose patient profiles are similar to the patient profiles you intend to include in the study so that appropriate and justifiable sample size can be arrived at. Also, do not leave it all to the statistician as he/she is not the researcher- it is you. The researcher needs to sit with the statistician to guide him/her as the statistician would require answers to quite a few questions as he/she works on the sample size. Not all journals ask for the rationale for sample calculation for a retrospective study, but many do. However, it is an absolute must in prospective studies and most journals will ask for the details of how it was calculated, what was considered, what was not considered, and why. The sample size should always be accompanied by a confidence interval and margin of error. These are statistical terms and any biostatistician would be aware of them and provide these details along with the suggested sample size.

If you wish to target specific journals for your manuscript, it is good to go through the journal guidelines at this stage to know if sample size calculation is required. Even if the journal does not specifically ask for it, having a sample size calculation and rationale for the same, makes the manuscript more robust and might be looked at more favorably by the editors.

At the end of all the above work, you now clearly know what data and information are required if you wish to submit the planned study for publication in an indexed journal. Thus, there is clarity on what is already available to you, what is useful, what should be discarded, and what is still missing.

Statistical analysis: The biggest pitfall that gets overlooked is the analytical plan. It is often assumed that the analysis will be done by the statisticians once the data is available. In fact, the analytical plan is most important even before writing the protocol or allocating funds to the study. The analytical plan will determine the inclusion and exclusion criteria as also the robustness of the data.

Let me give a couple of examples.

I know of a prospective study that was quite novel and multicentric, with random allocation but was refused by all leading journals because it captured too much data. Despite a sample size of 1500, the protocol did not define the treatment regimen and left it to the discretion of the treating physicians, due to which at the stage of analysis it was realized that there are a humungous number of subgroups, effectively leaving each subgroup with small sample sizes, which made it impossible to arrive at any meaningful conclusion of the primary objective. Another prospective study about the effectiveness of a procedure was returned by several journals because the inclusion criteria were not robust, as a result of which the baseline characteristics of the included subjects were very diverse in terms of the primary disease. Thus, each subgroup in itself had a very small sample size.

The analytical plan should also discuss how the primary and secondary objectives and subgroups will be analyzed. For each parameter that will be evaluated in the study, the protocol should mention what statistical tests will be used to evaluate the outcomes. Which subgroups will be studied, what is the basis on which subgroups will be created, and which parameters in which subgroup will be evaluated. After the plan is discussed, you might often find that you need to redefine the patient profiles for inclusion in the study.

Only when in-depth answers to all the above questions are ready and have been discussed and debated among the investigators multiple times, should even the protocol writing begin. Most importantly, the statistician should a part of all meetings and discussions between the investigators. The statistician should also be allowed to have a look at the protocol before it is sent to the ethics committee or other relevant authorities for approval. We have now adequately covered the study types and designs for researchers to know what to consider before even planning a study, which is eventually intended for publication.

In the next and final part of the series next week, we will look at writing the manuscript.