NHS PTS, CQC Ratings and the Effectiveness KLOE

The newer CQC Inspection regime for ambulance services providing Patient Transport appears more focused and comprehensive. The DH Consultation on expanding the existing regulatory system to include the publication of ratings for independent ambulance services concluded in October 2016 and a response to the consultation was published on the 17th September. Around the halls of the ESS show at the NEC last week were some discussions on the subject of even-handedness, and whether NHS and Independent sector ambulance providers are currently receiving unequal treatment, and would the new rating system for the independent ambulance sector change anything. My own take on the sentiments raised in these caffeine filled halls was that the inspection regime is objective, but NHS providers are perceived to have the ability to negotiate more favourable PTS contract performance standards which, under the proposed ratings framework, could translate into lower ratings for the independent sector. I have never seen evidence of this, but want to shine a spotlight on the variation in PTS contract standards anyway, while the system is being developed.

A CQC inspection can significantly affect a provider’s reputation (and so it should) and may impact commissioning decisions, however I suspect this is immaterial to the service users who cannot exercise choice. The main objective of the regulator is ensuring services to patients are safe and of high quality and overall, I think the recent published inspection reports look comprehensive and tightly bound within the KLOE framework. One notable exception, are services effective? which correlates to provider contractual performance, and to patient outcomes, specifically KLOE (E2)

 Effectiveness (and satisfaction) from a patient’s perspective has been repeatedly linked with reliable on-time arrival for their hospital appointment. The quality and consistency of on-time arrival performance is directly related to the cost of the service and currently aligned to locally negotiated performance standards, as well as the routine operational performance of the PTS provider.

Two wrongs do not make a right rating

 But what if the CQC publish numerical contract performance data that is unclear, confuses the public or is unintentionally misleading? Without a common standard or benchmark to use, it is unlikely the public will see through the jargon. The proposed new rating system would use this variable data, and then layer onto it a highly visible single rating using KLOE criterion, which it turns out even the most innovative provider cannot meet, The required resources needed to improve or validate a rating are not available. (There is a current illustration of the numerical data problem highlighted further on in the article).

 Under the new rating system, it is the E2 Effective criterion which appears to have a disconnect and it is not possible for the provider to comply with, as currently proposed. The E2 sub-criterions listed below are extracts from the appendix to the CQC Provider Handbook, and underpin inspectors observations used in the Effective KLOE. This qualitative rating system is used in conjunction with the newly published numerical data on contractual performance. These are used then aggregated with other sub-criterion, and then other KLOE to calculate the “overall” rating to be published.

Here are extracts the Effective criterion for each proposed level of provider rating.

Just where can PTS ambulance providers’ access performance standards on similar services or have their performance recognised by credible external bodies? just where are common NHS PTS standards benchmarked and published? Who are the external bodies referred to?

PTS standards in context

Most members of the public, and all NHS staff “get” the NHS 4hr A&E target, the 18-week RTT pledge, or the 8-minute standard for Emergency ambulances, they are national, simple and published widely. Which patients (and NHS staff) are really clear on PTS contract performance standards? (known as Local Quality Requirements, in the NHS Standard contract).

Before taking a view, on this one new element of the much improved inspection regime – it is worth sharing and understanding iterations of older PTS performance standards, because to the surprise of many, the PTS national standard of performance was originally mandated in 1974. It was set by the (then) DHSS via HSC(IS)67, when adopting the recommendations of the Ambulance Operational Research Unit at the Cranfield Institute. These standards became known as Organisational Control, or ORCON standards. Senior (in longevity) NHS ambulance folk may recall that they applied to the 999 service, fewer current service providers are aware that mandated national standards were also set for NHS Patient Transport Services. Back then, PTS accounted for 20 million journeys per annum or 90% of all ambulance journeys.

Figure 1 Cranfield Institute 1974

Note within the ORCON standard, the maximum number of patients carried per inward trip is, (2) and from outward is (8).

Prior to 1974 mini buses operated by ambulance services were few in number but grew in part, due to the Mini-Bus Act of 1977, Transport Act of 1978 and Transport Act of 1985, which relaxed bus drivers’ hours’ obligations and deregulated bus driver licencing and ensured the ambulance mini-bus became ubiquitous. The oh-so obvious cost effectiveness of carrying more patients on each trip could not be ignored. From a patient perspective, the impact was a less personal and a less reliable service. In a depressingly similar economic situation akin to, but not quite as biting as the current era of austerity, NHS PTS budgets were under pressure. In the late 70’s and early 80’s and as now, PTS needed to perform more effectively. Parliament looked at NHS PTS in detail through the Royal Commission (Merrison Report) and the Rayner Scrutiny’s. In 1984, the K?rner Report investigated why NHS Ambulance Trusts did not measure NHS PTS performance using mandated standards and they argued "the results were not worth the considerable effort needed to produce them” In 1990, the National Audit Office summarised “The standards set in 1974 for non-emergency patients have not been generally followed”

 By 1991, the NHS Management Executive published HSG (91) Ambulance and other Patient Transport Services: Operation, Use and Performance Standards. This document removed the national mandated standards and opened the way for local performance standards to be negotiated.

  Figure 2 NHS Management Executive 1991

Lost Focus - Quantity over Quality

Whilst I agree that mini-buses’ capacity to collect 6, 7 or 8 or more out-patients on inward journeys into a hospital can be cheaper when these vehicles are used optimally, there is no evidence they succeed in meeting inward performance standards. The compound risk of patient delay increases exponentially as the hospital day progresses. The underlying design imperative of the Orcon standard for carrying 2 out-patients on inward bound journeys has been lost. This measure was considered to reduce patient anxiety, and supported OP clinics efficiency with more on-time arrival of patients.

 As a practical example, providers delivering PTS services at NHS Children’s hospitals today can repeatedly evidence high levels of contractual performance, given the protocol that only one patient and the child’s parents travel in a vehicle at any one time, and the contract is budgeted for in this way.

 The ORCON approach has been eroded over time as cost pressures on the NHS increase and it seems in parallel, the majority of PTS provider’s ability to deliver consistently against agreed contractual targets. Both Independent sector and NHS Ambulance Trust providers have both been challenged to meet patient expectations for consistent on-time arrival. Competition has escalated the adoption of digital technology and at least one by-product of this technology upgrade is that PTS providers have delivered more advanced performance reporting, which unfortunately has had the effect of self-certifying that agreed contractual performance standards are not being met.

Business development teams working on NHS PTS tender bids can use route-planning software to optimise vehicle utilisation and fill up theoretical mini-buses en-route to a hospital, but up to now they have often been writing cheques that PTS ambulance crews cannot cash in real time. Elderly and frail patients do not always move with the times. This clearly affects patients experience and consequently, the smooth running of hospital services.

 The CQC are now publicly rating provider efficiency without evaluating if the locally negotiated contract standards are lower quality than average or more tightly specified. I have not seen evidence in recent reports that CQC inspectors have a formula that can identify high performance against a low quality contract, or poor performance against a higher quality one.

 The first obligation to publish NHS PTS performance was set down by the Health Act 2009 and amended in 2012. This act mandates that NHS providers, and Non-NHS bodies need to publish a Quality Account (QA) with NHS Choices by 30th June each year. Before the QA is published, Healthwatch and local stakeholders should be invited to scrutinise and comment on the service provided in order to influence local services and service improvements. Not that I was aware of the obligation when it was announced, but the Department of Health (DH) introduced QAs in 2010 as part of the national commitment to improve quality and transparency in the NHS.

 As a component part of a wider research project into Patient Transport, I methodically reviewed each and every published Quality Account for patient transport services in England between 2009/10 and August 2017. The fragments of results provide some useful insights which help to highlight why the CQC risk unfairly impacting organisational reputations when publishing Effectiveness ratings using the planned methodology.

Only one Independent ambulance provider has submitted a QA in the reference period to NHS Choices, some NHS Ambulance Trusts submit intermittently, others publish on their own websites. No PTS provider submitted what could genuinely be termed “meaningful” quality accounts under the definition. In the research I describe the difficulty in reviewing these QA, when providers prefer to use qualitative patient surveys, without the patient knowing precisely what contract performance standards (effectiveness) are involved. Using the Friends and Family Test has its value, but it is not intended to be directly correlated against contract performance. Often in published QA accounts, PTS providers use oblique performance data or peripheral information, such as how many phone calls are taken in a year. No PTS provider has transparently provided full on- time arrival information in the QA, which would allow an ordinary citizen to measure the reliability of the service. If PTS providers ran the railways, there would be no train timetables, no data on punctuality, merely significant comment about how busy the station is, how caring staff are and what a challenging rail network they are part of. I am suggesting the reluctance to publish punctuality data is because few providers, public or independent sector, succeed at meeting the agreed inward performance standards.

Mind the Gap

Piecing together some small fragments of published data, it is possible to observe the scale of the variances – how should patients use this information?

 Figure 3 J Sheehan University of Kent 2017

In 2015, Provider 3 had a 30-minute window (-15 to +15) to achieve a 90% target of patient arrivals and achieved 84.56%.

In the same year, Provider 1 had a 60-minute window (-45 to +15) to achieve 90% and achieved 88%. 

Apart from the inequality, in a theoretical scenario, either provider working to this contractual standard, could convey 90% of inward bound patients late for their hospital appointment, and still meet the contractual standard. This is clearly not outcomes based.

When the CQC inspected Provider 2 in February this year, they reported a (2015/16) target of 82.9% of patients to arrive, (-120 – 0 mins) which is a window four times bigger than that of Provider 3 and twice that of Provider 2. The target shown below in the CQC report was different to the target published in the QA – but I suggest this is just a typo, with the material point this provider was rated as Good for meeting this low standard – and rated as requiring improvement on the other 4 KLOE.

  CQC Inspection report 1/2/2017 rated here Effectiveness as GOOD

Service users or Healthwatch in Provider 2’s CCG, would be within their rights to raise questions asking why they are subject to a lower quality service and yet this is still rated as a Good service. Provider 3, with a similar percentage result as 2, but with a much higher standard to attain (by a factor of four), was also rated good for effectiveness. The CQC inspectors clearly need support in identifying the quality of service that has been contracted.

It might be that the criterion weighting formula used for calculating individual KLOE ratings and the overall organisational ratings needs refining and weighted more heavily toward timely arrival. This seems worthy of further investigation to me. I think PTS patients, if asked, will say that arriving in good time for their appointment is the key issue, and should carry significantly more emphasis.

Helpfully, the CQC promotes a “single shared view of quality”, it also suggests in the Shaping the Future publication, that “variations in quality will be taken into account”. Given the significant differences in local performance standards, just how can the CQC evidence they are taking contractual performance into account and translating variations into published ratings for providers? I would like to see the formula published and subject to normal scrutiny.

Let’s be fair to the CQC, these standards are not easy to compare as multiple variations exist, and each comprise of large data sets. There is no available evidence to suggest either public or Independent sector PTS providers are more adept at negotiating standards downward. Nor is it possible to obtain contractual costs and align them with contractual standards, due to FOIA commercial exemptions.

So, let’s also look at this from a patient’s perspective, a patient who has transport arranged for a hospital appointment, who is anxious, and enquires about punctuality. Would that patient be more reassured with a coherent answer such as, we get 95% of our patients to their appointments in good time or an opaque reply along the lines of “after a journey of not more than one hour, 88% of our patients arrive between 45 mins early and 15mins late”. This is not a good example of patient-centred commissioning, and translates poorly into the required language for an annual Quality Account.

So when provider reputations are at stake, would it be more equitable for the CQC inspection report to comment on, or rate a providers’ effectiveness against…

•         A clear and transparent NHS PTS national performance standard.  
•         And, link the CQC “well led” rating, to publication of a timely and meaningful Quality Account on NHS Choices.  

 Can we deliver more effective PTS services? – YES we can.

 PTS scheduling operations and vehicle despatch have already moved into the digital age, with on-line booking, satellite tracking, mobile data, SMS messaging and dynamic route-planning but my research clearly suggests that as a community of PTS providers we are struggling to meet the ORCON quality standards designed 40 years ago. Using some measures, it is clear we have taken backward steps. Cost pressures in the NHS, high demand from an older population, tougher commissioning, more incisive reporting have all had the cumulative effect of labelling PTS as problematic. Some artificially low contract pricing by ambitious market entrants has eroded the reputation of the independent sector. The morale and prospects of PTS staff are not good enough, and most disappointedly, collectively we have not improved the service to frail elderly patients. I do not believe we have yet integrated digital technology into the design of performance standards, only into the PTS despatch operation. This is the new development age for PTS. By redesigning and redefining the standards into outcomes based measures we can break the cycle of struggling to meet 40-year-old standards, designed for two inbound patients on a PTS vehicle, whilst acknowledging the cost pressures in the NHS which need the cost benefits of higher capacity vehicles.

The unchanging constant since 1974 is the need to convey patients safely to an appointment in comfort, within a caring environment, and in good time. To improve services, I do not think we need newer technology, we need to use what we have at our disposal more effectively and engage with commissioners and patients. Crucially we need to engage Healthwatch to discuss and champion outcomes based local quality requirements or OBLQR.

Lets complete the picture; Come on AACE, come on IAA et al, why not collaborate for the common good and deliver to the regulator a set of commonly supported, outcomes based standards, fit for a modern PTS service? Don't you think patients deserve it ?

Following the example of the 30:30 standards for renal patient transport, it is entirely plausible for a working group to get together from a community of providers and agree high-quality national standards for core PTS patient groups and service types.

In the meantime, and, I am keen to collect further evidence of successful NHS Patient Transport Services that consistently meet contractual performance criteria, specifically if an occupant ceiling is used for inbound patient journeys, or CQUINs are aligned to outcomes based metrics.