An Insider's Guide to Clinical Research - By Stanford Online

Case Study 1: Wine v/s Water for Type II Diabetes Mellitus

Participants were randomly assigned to drink 5 ounces of red/white wine in the test group and the control group drank the same amount of mineral water daily for 2 years in a phase III, parallel RCT cascade trial

The study in Annals of Internal Medicine stated that red and white wine could improve the control in blood sugar level in people with slow alcohol metabolism.

This trial was not easy to perform, as per Dr Regina, this trial was an observational study and hence one cannot get causation from an observational study. There is a bias in this study when people who chose to drink wine, already like wine and have a status to afford the same. Wine drinkers are much more educated, have more money and might have a better lifestyle which could be the reason for better control in blood sugar levels. Hence, one cannot claim that wine alone affects the health of the participants.

The drawbacks of this study were

1.?????? The researches chose more than 2 primary outcomes, which gave false positive results and they did not adjust the p value for multiple testing.

2.?????? Also, the data was tortured by excessive testing.

3.?????? 5 primary outcomes inflated the Type I errors.

4.?????? The authors pre-registered in clinicltrials.gov the study which kind of locked their parameters and outcomes. However, they switched the outcomes by cherry picking data and this was very evident.

5.?????? All primary outcomes were false positive and statistically significant

6.?????? All secondary objectives were statistically insignificant

7.?????? Cherry picking data and p hacking were very evident to the peer reviewers

Case Study 2: Hookworms for Insulin Resistance

Hookworms were used in this study after observing areas where parasitic worms were endemic. In these areas, the metabolic diseases like Type II DM have low occurrence rate.

Hence. A 3 Arm, 2 years long, Phase I b study was designed to assess the tolerability and safety of hookworms in patients with Type II DM for 2 years. The intervention was a dose of 20 hookworm larvae or a dose of 40 hookworm larvae applied to the arm. The control group was exposed hot chili pepper sauce, which was a great sham control as it made the control group feel as if the worms were really entering them. Both the groups were told that they might experience tingling or burning sensation, this cancelled out the bias by making the control group believe they also got the doses of hookworm.

It was a double blinded study which further ensured the removal of bias. There were no SAEs observed, mood of participants was slightly better in the test group. Insulin sensitivity improved in the low dose hookworm group. However, the test group with low dose hookworm had average weight loss of 5 kg.

Case Study 3: Competition for improving sleep and exercise

Medical interns are generally sleep deprived during internships and get fewer physical activities; therefore, micro randomized trials were conducted to assess the effectiveness of gamification through smartphones. The teams entered a competition of daily step counts as one of the physical activities. The steps significantly increased by about 106 steps per day. The teams were updated daily with opponent’s progress and thus motivation was used to increase physical activity.

This study used micro randomization which randomized the interns 12 times in 12 weeks. This smart design of the trial allowed every team to play control group and it was a clean automation. However, again the data was missing here and cherry picking had occurred.

Take home message:

1.?????? Micro randomization represents a novel study design for testing just in time adaptive interventions

2.?????? Clinical trial should be pre-registered before first patient enrollment.

3.?????? Throwing out portions of data is another form of cherry picking and leads to false positives.

Case Study 4: Superpower glass for kids on the autism spectrum

Kids were asked to try on glasses connected to the parent’s mobile phone through an application. ?The glasses have a prism in the corner which interacts with people talking to the person wearing the glasses. The prism spots the face of the person in front and shows a green box to the glass wearer, the application on the phone detects the person’s emotions and categorizes it using the software installed in it. The application categorizes the emotions e.g.: happy, sad, angry, etc. this data is then translated to an emoji which is shown in the green frame. The participant can see this emoji and understand the emotions of the person sitting in front of them. The researchers performed cross over randomization and exploratory studies; however, the sham control was not up to the mark as per the peer reviewers. Another observation by the reviewers was about all the digital studies, they have observed a trend that researchers are registering the digital studies after completion of the trial and not before first participant enrollment. Another observation was that the parents of the participants were also involved and hence the inclusion of the bias was expected, however the researchers took care of this kind of error by performing Bonferroni correction.

Potential problems that weakened the evidence were:

1.?????? Cross overs are best for interventions with no lasting effect, which was not the case here.

2.?????? Participants who got the device first must have continued to improve even after the cross over was done and arms were interchanged, as the device would have a long-lasting effect, if it was indeed effective, this increases error in the control group readings.

3.?????? As families were involved, there could be a placebo effect due to family and loved ones spending time together.

4.?????? Families could have been given a sham intervention with instructions for equal family time to minimize error in both groups (test and control)

5.?????? Families found the device too challenging to use, hence 24% of families dropped out while others used the device only 2 times a week versus 4 times a week as per the protocol.

6.?????? Families has usability issues with the device in the early exploratory stage of the trial, however the researchers kept at it and started the trial while still recruiting the patients for the exploratory phase.

I learnt a lot through this webinar, so thank you again Dr Regina and Dr Sainani

Take home message:

1.?????? Observational studies of lifestyle habits are often confounded, making randomized trials of these behaviors very important.

2.?????? Number of statistical tests performed are directly proportional to chances of occurrence of false positives.

3.?????? Outcome switching is a form of cherry picking the data and can lead to false positive findings.

4.?????? Early phase studies pave way for larger, more definitive studies.

5.?????? Available data and a pre-registered/published protocol increase trust and transparency

6.?????? Blinding researchers and participants and using a good control sham intervention increases protection against placebo and nocebo effects.

7.?????? Data gets published even when it is statistically not sound, usually p hacking is easy to hide from peer reviewers. Hence, we need better peer reviewing and better-quality control.

8.?????? Last but the most important one is that some RCTs are robust while others are flawed and mis stepped and therefore we need Real World Data (RWD) and Real World Evidence (RWE)

9. The best learning through the webinar was the difference between micro-randomization and cross over randomization:

Micro-Randomization:

-????????? It is a very new concept

-????????? It cannot be done by hand

-????????? It is done for changing the conditions of a group (test or control) every week or every day

-????????? It is difficult to analyse the data generated by micro-randomization, hence good statistical techniques are required.

Cross Over Randomization:

-????????? Can be applied randomize only once, for a single switching of arms

-????????? Conditions to which both arms are subjected remain the same

-????????? Multiple changes cannot be done

-????????? Data is not too complex to analyse

?

?

?