How long should I observe my test subjects in the evaluation study?

In theory:

The current (as of June 11, 2024) DiGA guideline from the Federal Institute for Drugs and Medical Devices (BfArM) does not contain any concrete requirements regarding observation periods and data collection times within the scope of the evaluation studies. According to the BfArM, a baseline data collection, i.e. a collection of data before DiGA use, is useful. In addition, the data collection periods and possible follow-up observations after the intervention phase should be described and justified in the scientific evaluation concept. The basis for these justifications can be the systematic literature review and/or pilot study data that were already collected with the DiGA during the systematic user data evaluation (pilot study).

In practice

As of June 04, 2024, 55 applications are listed in the DiGA directory, 35 of which are permanent. Most DiGA manufacturers chose an observation period of 12 weeks (n=30) to prove the positive healthcare effect of their DiGA. The shortest observation period was 7 weeks and was successfully used by the manufacturers of the DiGA HelloBetter Stress and Burnout to prove the primary medical benefit in the form of a reduction in stress levels. The longest observation period was 12 months and was successfully used by the manufacturers of DiGA Kaia Rückenschmerzen - Rückentraining für Zuhause to prove the primary medical benefit in the form of a reduction in pain intensity.

Our experience

The BfArM requires a description of the baseline data, the observation periods, and the follow-up periods, but allows the manufacturers flexibility in justifying the specific observation periods. This justification is of central importance for both the evaluation concept and the study protocol and therefore requires a comprehensive literature search. It should be shown that comparable studies (i.e. comparability regarding the selected study population, the intervention used, the outcomes investigated, and the study design used) have achieved significant effects in the specified observation period. The search for these comparable studies is often lengthy and requires a structured search process. Sufficient time should therefore be devoted to the argumentation of the observation period as part of the systematic literature search.

Above all, it is important to plausibly justify an observation period that fits the selected indication and the specific intervention. For example:

A DiGA to improve the lifestyle of patients with diabetes mellitus addresses the improvement in blood glucose levels (HbA1c) as the primary endpoint. The HbA1c value indicates the percentage of blood sugar that has been bound to the red blood cells within the last 8 to 12 weeks. This requires an observation period of at least two months. DiGA manufacturers such as vitadio have chosen an observation period of six months, as this is sufficient to detect significant changes in the HbA1c value and other metabolic parameters.

A DiGA to improve the self-management of patients with an addiction addresses the improvement of the state of health by reducing the symptoms of addiction (e.g. alcohol consumption, smoking). DiGA manufacturers aiming to achieve a change in behavior through their application should allow sufficient time to achieve this. The more complex the behavioral change is, the more time it takes for it to become automatic and the longer the observation period of the scientific study should be. Therefore, longer observation periods can be seen in DiGAs that address an addiction problem. This is exemplified by the NichtraucherHelden-App or Smoke Free - Rauchen aufh?ren which used an observation period of six months, or the manufacturers of vorvida which addresses patients with mental and behavioral disorders caused by alcohol and used an observation period of six months as well.

If the observation period is shorter than the subsequent prescription period, it is also advisable to justify the transferability of the effect(s) achieved to the prescription period.

To successfully complete the study within the planned evaluation period and with a realistic observation period, it is essential that everyone involved works hand in hand. As a scientific institute, we can offer the entire research process in close cooperation with Clinical Research Organization (CRO) GREENBAY research GmbH in a seamlessly planned and continuously coordinated workflow—from study planning and implementation to data analysis and reporting, to the publication of the results in a manner appropriate to the subject and target group. Our DiGA expert Tonio Sch?nfelder will be happy to answer any questions you may have.

The WIG2 Institute’s DiGA-FAQ addresses the questions that have been frequently asked in our previous collaboration with DiGA manufacturers.

DiGA-FAQ // Evaluation study // Question 4: How long should I observe my test subjects in the evaluation study?

? zur deutschsprachigen Version des Artikels