Activity, output or outcome, how can AHPs measure quality?

The financial squeeze is tightening on NHS trusts, and as more service budgets and staff vacancies cross the fiscal event-horizon to become ‘savings’, service leads and clinicians are desperately searching around for metrics to justify a stay of execution.

Unfortunately, allied health professionals (AHPs) and their services can easily fall prey to the dreaded excel spreadsheet of recurrent and non-recurrent savings. And not because there are inherent prejudices against our professional group, but because often we do not have the quantitative metrics to mount strong defensive cases that translate into sound financial impact.

It is of particular note this year as the national priorities for AHPs are the themes of ‘Quality’ and ‘Safety’. Both of which depend on us having suitably robust, sensitive, costed and relevant metrics to be able to monitor any change in their state over time. And if we accept the premise that delivering both quality and safe care is fundamentally better value, then having such metrics is fundamental in the current constrained climate.

In the title I state three broad types of metric; activity, output and outcome. Outcomes are what we ultimately deliver, and they are the sum total of our activity and output. And if the activity and output can be costed, then so too can the eventual outcome.

Complexity can arise when it is not always possible to measure an outcome. If this is the case then we may rely purely on activity metrics instead, safe in the knowledge that the desired outcomes will follow. The activity becomes a proxy for outcomes. An example would be hand hygiene audits. This is reliable as we know that if staff members are physically seen washing their hands in the correct way then we can be assured their hands will be clean. The link between the activity (correct hand-washing), the output (cleaner hands) and the outcome (reduction in hospital acquired infections) has been well established in research over many years. Of course this is a very simple example, as soon as we stray into the realms of interventional clinical activity and output, then outcomes (and their costs) all become much cloudier.

“The NHS has endless metrics” you might cry, and this is very true. Any clinician reading this will be very familiar with completing regular audits, surveys, risk assessments, tick-boxes, EPR forms and the like. Most without obvious or immediate relevance to enhancing quality or safety, and all come at a cost. And we are all familiar with that most dreaded (and factually incorrect) phrase ‘If it’s not written down, it didn’t happen’. In reality this should instead be rephrased ‘If you don't count it, then it doesn't count’. As any form of activity, output or outcome in the NHS requires evidence before it’ll be ‘counted’. Again, this approach is not something I agree with, as a very valuable activity can result in an intangible, immeasurable, short or medium term outcome (e.g. sitting and properly conversing with a patient), but sadly it is a truism in the current NHS cultural landscape.

So, what exactly is the problem? Quality and safety are the outcomes we are all aspiring to deliver, and whilst we all know what quality care looks like when writ large, it is much harder to agree on its specific, niche components. In that way it is a ubiquitous phenomenon. And therefore if we do not know exactly what outcomes constitute quality care, then we will not be able to accurately use activity or output as proxy metrics either. I believe this is the challenge we face as AHPs. Capturing outcomes at a service level are difficult, particularly if viewed through a financial lens, and whilst we can easily capture any activity we complete (patients assessed, equipment ordered, home visits completed etc etc) if we do not know (or cannot evidence) specifically what outcome these are contributing to, then it’s all for nothing from a capturing impact and funding perspective. Fine at times of plentiful budgets, but much less so when widespread cost reductions are needed. And at a system level the challenge becomes greater still.

The other problem of only capturing activity and output (not outcomes) is that whilst it demonstrates the demand on your service, at the same time it is a record of the service’s cost too. As all activity consumes financial resources, and without verified outcomes to offset this fiscal outlay, you are simply offering an imbalanced equation of expenditure only, without any quantifiable value or income being generated. Any finance director searching for apparent non-added value to cut will likely start here.

So, what is quality? The key components of quality care in my mind are timeliness, effectiveness, professionalism, accessibility and equity. My list mostly chimes with that set out by the World Health Organisation. And yet the NHS has a hard time measuring many of these components.

Timeliness appears to be the easy one, as waiting list data is published continuously, and it is clear that this is an area where the NHS is struggling across most services. For AHPs specifically, we know that waiting lists are growing and this is due to the combination of increased demand, lesser investment in AHP service provision (compared to acute nursing or medically led care) and a workforce shortage. Even worse, timeliness is one of the metrics that is most easily gamed, as the definition of when a patient is ‘seen’ can be fluid and handily open to interpretation. Plus, it quickly becomes another proxy measure for activity, as individual patients are seen more quickly, so the total number of patients seen increases. And of course seeing patients quickly, rarely results in the best overall care experience. Goodharts law is that any metric that becomes a target ceases to be a good metric. And for AHP interventions in particular we are not best served by a focus on timeliness. Our commitment to recovery, functional restoration, independence and rehabilitation all often require timeframes incongruent with a focus on rapidity, acute delivery and quick resolutions. As such, despite timeliness being one of the easier metrics to measure, it is often a poor indicator for establishing the quality of AHP services.

Effectiveness is defined as “the degree to which something is successful in producing a desired result”. And this requires knowing what the desired result (or intended outcome) is meant to be before you can evaluate its success. In an interaction between clinician and patient this is fairly easy, as all AHPs will utilise some form of clinical outcome measure or PROM (patient reported outcome measure) to guide their practice and impact. The outcome measure having been selected based on assessments and in discussion with the patient about what they want to achieve. Yet, elevating this to a service, trust or system level becomes foggier. How do our specific interventions snowball and impact the waiting lists, onwards referrals (or lack of), discharge rates, step-down intensity etc. Of course, the impact of AHPs on all of these factors will be considerable. But where the AHP owned impact lies is hard to unpick, if possible at all. And as most services are multi-professional then changes in a services effectiveness will often be viewed through the biomedical lens with most attribution to nursing and medical colleagues. A greater emphasis on meaningful recovery metrics and return to function as outcome measures will naturally play to the strengths of AHPs and I would argue patients too, but trying to move a data model that has historically gorged on activity inputs rather than outcomes is tough to envisage. The system also needs to explicitly regard patient recovery and function as markers of success, this will enable future effectiveness to be judged accordingly. Otherwise if there continues a preoccupation with ‘activity’ then success can only be judged in this context. To their credit the mid-south Essex ICB is attempting to redefine their outcomes through a subsidiarity model and this approach offers real hope for AHPs and service-users.

For me professionalism concerns two aspects related to the workforce. The first is the extent to which organisations enable their healthcare staff to deliver on their professional responsibilities, and the second is the clinical ratio of unregistered to registered staff. Both of those pillars are addressed by some combination of the following; ensuring capacity/demand is realistic (safer staffing levels for AHPs perhaps), monitoring the ratio of unregistered to registered staff to prevent it escalating to unwieldy levels, and ensuring staff have and value their supervision, appraisal and CPD opportunities. These are all metrics that can (and should) be reported at a system level and again the system must tie this intimately to their own definition of effectiveness. The AHP workforce is one that see’s attrition of its more senior clinicians all too frequently, and ensuring we are able to retain this expert, knowledgeable and skilled workforce should be an absolute priority for every healthcare system. We cannot talk about productivity, clinical excellence and prioritising patient outcomes if we are not serious about retaining our most experienced AHPs in clinical roles. Prioritising and being explicit about valuing professionalism at a system level will be a powerful motivator for organisations and should be a stated outcome in one guise or another.

Lastly accessibility and equity overlap with and empower each other, and all the previous components must be viewed in the context of our patients and service users being able to access the care they need, in the way they need to, at the time they need to. Equity and accessibility both require our AHP workforce to reflect the communities it serves, to ensure we are culturally, emotionally and socially aware. We have data and metrics that map our populations, and we have data that informs where we can expect to see specific health conditions. We must use both data sets to focus AHP services on areas of most need in an effort to reverse the ‘inverse care law’, and access metrics could be a more informative activity metric than focusing on discharge metrics. Again it refers to the earlier point of what our ultimate, desired outcomes are? Is it simply throughput of patients, or is it about meaningful improvements in the key markers of health and wellbeing. I would argue that currently we still prioritise numbers seen, rather than outcomes achieved. And as I have alluded to this is likely due to complex, multifaceted, longer term outcomes being harder to ‘cost’ whereas most activities and outputs are easier to put a price on. If this perspective can be shifted at a system level, then it will be of benefit to us as AHPs and complement our holistic view of health. It is the responsibility of AHPs to challenge the metrics currently used and offer suitable alternatives in conjunction with patients. What is important to AHPs will be important to our patients and service users.

In summary, discussions about quality and safety are downstream of the more important discussion about metrics. We cannot measure, monitor or aspire to value, quality or safety without utilising metrics. And the metrics we use dictate the services delivered, the healthcare staff charged with delivering them and their impact on patients. If we want the overarching outcome of our AHP services to be the good value provision of improved health, well being and function in patients, then all activity and output metrics must be questioned. We must start with clearly defining the outcomes we want to achieve and then routinely question how each activity or output metric contributes to said outcomes. If a metric cannot be explicitly linked to an intended outcome, then it should be scrapped.

As AHPs we must make narrative waves in the murky pool of quality and safety and at the same time be brave enough to suggest a complete change of the water.