Implementing ICH E9(R1) – still progress to be made?

I haven’t been on LinkedIn much recently but had some spare time yesterday. Two ‘random’ postings caught my interest that referred to publications of recently completed clinical trials and I thought I’d take a look to see how they incorporated the estimand framework.

1.??????A post on the treatment of mild to moderate psoriasis using oral apremilist, showing results from an extension study.

https://www.dhirubhai.net/posts/melinda-gooderham-3544a754_psoriasistreatment-psoriasis-activity-6993752507357286401-nfXD?utm_source=share&utm_medium=member_desktop

What originally caught my eye was a discussion started by Dr. Alexander Schacht on the graphic. I went to look at the actual article and noted that there were two versions of the graph.

Rather than commenting on the graphic itself, my criticism is of the use of NRI (non-responder imputation) in the analysis that produced the second graph. Or at least the labelling of the method itself.

I am assuming that what happened is that any patient who did not get to a time point (presumably dropped out of the study) was called a non-responder.

Imputation means to replace the missing values of a variable with plausible values.

It is not explicitly explained in the (albeit short) article under what circumstances we are trying to impute, that is, what was the treatment effect of interest. Is the imputation based on what would have happened if the patients had in fact continued treatment? Seems unlikely. Is the imputation based on what would have happened if the patients had taken no further treatment? Perhaps, but would they all have immediately become non-responders as soon as they stopped treatment? Seems unlikely. Is it actually that the endpoint itself has been re-defined to include lack of completion as part of the definition of treatment failure? That is, the endpoint is a composite endpoint? This is more likely but then we are not imputing a value for such patients, we are deriving an endpoint. So let's not call it imputation.

I then went back to look at the publication of the original study (also published in 2022). I found that “missing values were imputed using the multiple imputation method”, but again no mention of what treatment effect estimate they were trying to impute for.

Neither the extension trial or the original trial explicitly makes use of the framework, leaving it unclear what the studies were trying to estimate.

The article can be found here: https://www.jaad.org/article/S0190-9622(22)02968-1/fulltext.

2.??????A post highlighting a study on the evaluation of Simvastatin as a disease-modifying treatment for patients with Parkinson Disease.

https://www.dhirubhai.net/posts/exploristics_clinicaltrials-biostatistics-statistics-activity-6993872533108224000-Lirr?utm_source=share&utm_medium=member_desktop

The primary outcome was assessed at 24 months.

It is great to see that more and more clinical trial publications are providing the protocol and SAP as supplementary material, allowing the reader to see the detail that there isn’t room for in the publication itself. This publication is one of these.

There was no specific mention of the estimand framework within the protocol or SAP. I did find the following in the protocol, suggesting a treatment policy strategy: “The trial analysis is based on intention to treat, so all participants will be encouraged to continue with study visits and assessments as per protocol even if trial treatment is discontinued prematurely” and “Data collected prior to withdrawal from follow-up will be included in the study analysis”. This is great! It is not mentioned in the study synopsis or aims and objectives however, nor clearly in the publication.

However, the SAP states “The mITT evaluable sample for the primary analysis will include all participants who are randomised, provide baseline outcome data, commence to the higher dose phase of the study and provide 24-month outcome data”. So, participants who dropped out of the study early were not included in the analysis, which is not in keeping with a treatment policy strategy?

The publication can be found here: https://bit.ly/3Wu4Zu6

Of note, there was a lot in the provided SAP that I did appreciate and even learn from. No doubt Dr. Alexander Schacht could comment on Figure 2 of the publication.

-------

For a much more thorough look at how we, as an industry, are getting on with the implementation of ICH E9(R1) see this publication:

-?????????Cro?S,?Kahan?B C,?Rehal?S,?Chis Ster?A,?Carpenter?J R,?White?I R?et al.?Evaluating how clear the questions being investigated in randomised trials are: systematic review of estimands?BMJ?2022 (https://www.bmj.com/content/378/bmj-2022-070146)

(For honesty, and in case anyone goes looking, I must put my hand up and acknowledge that I too need to work harder in this area. A recent clinical trial publication that I was involved with also makes no use of the estimand framework. One of my personal goals is to make sure that this doesn’t happen again.)

ICH E9(R1) has been finalised for two years now and uptake and implementation has been high. Some people are still struggling though, particularly those without easy access to knowledgeable statisticians, despite the topic being relevant to many disciplines as it affects the study objectives. There is a lot of freely available information out there.

For example, with regards to incorporating estimands into the protocol, see this recent publication written by some of the members of the Estimand Implementation Working Group (https://www.efspi.org/EFSPI/Working_Groups/EFSPI_EFPIA_EIWG.aspx), of which I belong to:

-?????????Lynggaard, H., Bell, J., L?sch, C.?et al.?Principles and recommendations for incorporating estimands into clinical study protocol templates.?Trials?23, 685 (2022) (https://doi.org/10.1186/s13063-022-06515-2)

I have provided study specific support in implementing ICH E9(R1) as well as general training to many clients so do get in contact if you need help. ?