Practical Application of Medical Impact Measurement

By Carlos Eid, MD

Views are strictly personal

One of the major challenges that medical affairs professionals face is how to demonstrate their value and impact to the organization. Traditionally, the metrics used to assess the performance of field medical associates (MSLs) were based on quantitative indicators, such as the number of visits, the duration of interactions, or the coverage of key opinion leaders. Additionally, new metrics started surfacing like number of actionable insights, advocacy based on agreement with scientific statements, etc... However, these metrics do not capture the impact on behavioral change or change in patient outcomes objectively. Therefore, there is a need for a more comprehensive and robust approach to measure the impact of field medical activities, one that considers both the process and the impact of the engagement. There has been enough talk about metrics and impact measures, some have made it too complex and others too simple, and personally I feel we are missing a practical, pragmatic approach (which may not be perfect) but a good starting point if we really want to drive change!

Therefore, in this article, I will share with you a personally devised, practical example of how you could implement an impact measurement framework for your field medical teams (based on my own experience and views strictly). I will share how to define your desired outcomes, identify your key actions, collect and analyze your data, and communicate your findings. I will also illustrate how to use this framework to monitor and improve your performance, as well as to showcase your value and achievements to management and cross-functional partners.

First, let me quickly remind you of the impact framework I proposed months ago where our thinking needs to shift from transactional interactions and a focus on the number of activities to a bigger focus on the patient outcome that we hope to impact.

Based on this figure, focus on key 5 steps:

1. Choose the patient outcome (big ambition)/output (steps along the way) you want to improve
2. Identify the Impactful Contributions that are needed to move the needle on these outcomes/outputs
3. Figure out how to accomplish them by engaging with the right stakeholders at the right time using the right channels
4. Generate insights from these interactions to help identify key opportunities, key barriers and to help monitor your progress
5. Measure if there is any impact on the patient output/outcome due to potential behavioral changes

If you are asking yourself what “Impactful Contributions (ICs)” are, take a look at the figure below and note that we use the term contributions as some of these activities might not be primarily led by MSLs or have a shared accountability.

Let’s jump to the "practical example"

To help with this I will start with creating something that we will call the Medical Impact Plan (MIP). For this I will make an assumption that any CRM can incorporate this template and its functionality.

STEP 1: Start with the patient outcome/output.

PS: Please note that this example is purely fictional, given for the purpose of explaining the model and not related to any current strategy or tactics in my organization.

The disease I will tackle is COPD. As a patient outcome I will choose “National reduction in hospitalization rate for COPD exacerbations by XX% over X years”. This is longer term, so I will couple it with a shorter-term patient output (that will ultimately impact the outcome over time) “XX% increase in GDMT (guideline directed medical therapy) implementation in COPD patients discharged after a COPD exacerbation”.

(PS: you may decide to have more than one patient output for a disease area, but we will choose only one for demonstration purposes)

STEP 2: Decide what impactful contributions can help achieve this.

I will assume that we will need to (and these could look too basic, few or simplified versus the ones you will eventually choose):

1. Co-Develop or support medical educational programs on overall COPD management and guideline implementation targeting XXX HCPs across XX different regions. (exact numbers should be based on concrete insights and ensuring that we are educating the right stakeholders leveraging the right speakers aiming for maximal impact on patient management)
2. Support the creation of clinical pathways and hospital protocols to enhance guideline implementation and disease management upon patient discharge (XX hospitals)
3. Support the generation of RWE and IS on COPD management (X projects)

Reminder: Implementation Science focuses on how to effectively integrate evidence-based interventions into real-world settings. It studies the methods and strategies to ensure that these interventions are adopted, implemented, and sustained in practice. Real-World Evidence involves collecting and analyzing data from real-world settings (outside of controlled clinical trials) to understand the effectiveness and safety of interventions        

STEP 3: Based on the strategy and needs the team will have to then:

1. Decide on the number of ICs (how many A,B,C) and flesh out the relevant milestones (if needed). Number of ICs and milestones will differ based on desired patient outcome/output, lifecycle, targeted stakeholders, team size, company size, budget, etc…
2. Choose the right stakeholders that can help them get the ICs achieved – these can be KOLs, DOLs, non-HCPs, rising stars etc… (PLEASE do not fall in the trap of having the "traditional list of KOLs only" and focus on who can help you get the IC completed in order to impact outcomes)
3. Engage and partner with the relevant stakeholders internally and externally, generate insights, aim for behavioral change, track your progress and assess your impact

Example of an IC with milestones linked to it:

A.?Co-Develop or support medical educational programs on overall COPD management and guideline implementation targeting XXX HCPs across XX different regions.

o?? Milestone 1: Identify speakers/steering committee

o?? Milestone 2: Content, delivery channel and target audience

o?? Milestone 3: Execution and impact measurement

If we assume that the above Medical Impact Plan can be incorporated in their CRM, MSLs can now engage and link their engagement to either of the three impactful contributions, and most importantly document the progress in the “notes” and “progress” tabs. Once extracted, the table could look like the below:

STEP 4: Once all milestones are achieved (if milestones are needed of course) and an impactful contribution is marked as fully accomplished the MSL will be prompted to add

1. Details/proof/quant metrics (like survey scores in the case of education)
2. Whether this IC is to be replicated (for additional HCPs or different areas)
3. Then he has to send it to his manager who will assess and approve it.

?

The example below shows a case where the MSL decided to have 5 educational activities for IC A (ie each is treated as an individual Impactful Contribution not a milestone, where the milestones are the gradual steps to complete an IC).        

In summary if the MSL decided 5 individual activities for IC A, the development of 3 protocols for IC B, and 1 evidence generation activity for IC C, then his total is 9 ICs to be completed.        

?

What can an impact dashboard look like?

A dashboard that brings all this to life can look like the below (more of a leadership view and not individual MSL view), but certainly you can also consider:

1. Individual team members’ reports/dashboards
2. Summary of notes per IC per quarter
3. # of engagements leading to IC completion
4. Etc…

How is that helpful?

1. We will no longer have activity metrics just for the sake of it, and it becomes clearer as to WHY we are engaging with these specific stakeholders, WHAT are the key outcomes and HOW is this impacting the progress of our Impactful Contributions (that will eventually impact the output/outcomes).
2. All completed impactful contributions are linked to metrics, proof of completion, surveys – and not only a tick the box exercise where we fulfil activities or visits without a clear why or so what (with an established linked to the outcome/output we started with)
3. The generated reports will help MSL managers in their evaluation and coaching by being able to ask key questions like:

I see you had 15 engagements this quarter for IC B, and based on the notes and progress, there are still no discharge protocols embracing the updated guidelines in these centers. Why do you think this is the case? Are we targeting the right decision makers? Is there anything I can help you with?

Time to protocol creation in hospital A took less time and less engagements than hospital B, what are the key differences, best practices or challenges?

We did 10 educational activities, however the feedback from customers is neutral and there was no difference in pre and post surveys, What can we do differently? Are we choosing the right speakers, content and audiences?

So what??? Have we impacted the patient outputs or outcomes??!!

The exercise doesn’t end here, as (per the above) we just measured activity metrics and the completion of Impactful contributions, which we “assume” or “hope” to impact patient outputs/outcomes. Now, we need to take this to the next level and try to map the achievement of these ICs with the progress of the patient output, which in this specific case is: XX% increase in GDMT (guideline directed medical therapy) implementation in COPD patients discharged after a COPD exacerbation and ultimately the rate of COPD hospitalizations post discharge.

These can be measured using claims data, registries, databases, market research, real world evidence and other methods (including AI) that are possible and already being implemented by several Pharma companies. Only caveat is that we expect to see a faster impact on the patient output as the patient outcome will require more time. Therefore, you focus on the output on a yearly basis and outcome long term. (Long-term vs short-term impact of medical affairs...rings a bell?)

An example chart could look like the below where you map the yearly # of ICs the team is doing vs the cumulative % change in patient output.

What's next... The MOST IMPORTANT step: Course correction

What happens if the patient output is not being impacted? This is when you should consider reflection and course correction.

I propose 3 levels of course correction (with examples of questions that you can reflect on and ask)

?

Scenario 1: Impactful contributions are not progressing or being completed

1. Are we engaging the right stakeholders?
2. Are we conducting the right activities?
3. Are we actioning the right insights?
4. Are we capturing the right feedback?

Scenario 2: Patient output is not being impacted

1. Have we chosen the right ICs?
2. Should we consider alternative ICs?
3. What more can we do to change behavior?

Scenario 3: Patient outcome is not being impacted

1. Have we chosen the right patient outputs to impact and track?
2. Do we have the right strategy in place?

Final Thoughts and Conclusion

Measuring the impact of medical affairs has long been a challenging endeavor. While we have made significant strides in quantifying various metrics such as the number of activities, percentage coverage of Key Opinion Leaders (KOLs), number of visits, subjective advocacy assessments, and customer satisfaction, these measures alone do not provide a complete picture.

What I am sharing with you is not intended to be the ultimate solution; rather, it is a different perspective (which some might already be thinking of or adopting). My aim is to link strategy to desired patient outcomes and focus on activities that serve this outcome (or “north star”) with course correction (a step we often miss). Additionally, I propose adding a few measurable (and possible to visualize) layers between what is easy to measure (activities) and what is difficult to measure (patient outcomes). These layers are:

1. Impactful Contributions: These are the significant actions and initiatives that could directly influence the desired outcomes. Examples include successful protocol creations, impactful evidence generation activities, and/or meaningful/impactful educational programs.
2. Patient Outputs: These are intermediate outcomes that reflect the benefits to patients, such as better patient diagnosis, enhanced quality of life, better GDMT implementation

Finally, I would like to thank all the colleagues who reviewed my article as a draft and shared their valuable insights. Let's keep striving to highlight the impact of medical affairs and aim for better patient outcomes.

?

The ideas and concepts presented in this article are the intellectual property of the author. Readers are kindly requested to respect the author's work and not claim these ideas as their own. Proper citation and credit should be given when referencing or using any of the information presented in this article.