Airport planning casts aside best appraisal practice

Fair warning – this is a deep-dive post that gets into the more technical elements of transport project appraisal and the Department for Transport’s Transport Analysis Guidance (TAG).

These were the words that (almost) broke me:

“Ultimately, it would be open for us, and the SoS to take it [TAG] into account as a material consideration. However, given the differences between the relevant parties and its lack of application elsewhere, there is too much uncertainty in its application for it to be useful in determining this specific appeal at this time.” – London City Airport appeal, Inspectors’ Report, p.156

With this cutting ruling the Planning Inspectors writing up London City Airport’s appeal proceedings tossed out hundreds of hours of cumulative work across five airport planning applications by my colleagues and I at the New Economics Foundation. Not only had we advocated that an assessment guided by TAG should be given weight, we had conducted parts of that assessment ourselves.

We did this partly to demonstrate just why the airports, their consultants and lawyers, were fighting against giving TAG weight so aggressively. The numbers weren’t in their favour.

I’m writing this blog in the hope that our time wasn’t entirely wasted (and because it’s way too technical/boring for a NEF blog!).

We’re far too lax with standards when it comes to economic appraisal in the planning system

The crux of the argument I (representing NEF) advanced in multiple planning forums was this:

The planning system enforces almost nothing in the way of best practice appraisal standards when it comes to economic assessment. Ultimately what typically prevails is precedent, witness credentials, and a bit of know-how from planning decision makers themselves when it comes to economic statistics.

The adversarial system we operate in the UK sees the scheme applicant submit their own impact assessment, the ‘environment statement’. This assessment will often be presented as an ‘independent’ assessment made by expert consultants. But, as many consultants are available on the marketplace, the Applicant can choose the consultant they believe most likely to produce the desired outcome – approval. Often the consultant will make errors (as we all do), or worse, they might build a questionable overarching narrative out of selective data.

How, then, do these errors and misrepresentations get challenged? If the local authority opposes the application (or parts of it) they may commission their own consultants (vulnerable to similar compromises, of course). But LA resources are always extremely limited and they are risk-averse. At the City Airport inquiry, the Borough of Newham presented no economics witness. More often than not, at least part of the job of holding an applicant to account falls on the public, and organisations (NGOs, campaign groups) representing the public interest (or a portion of it).

The public can’t challenge dodgy analysis without agreed standards

Search the Proof of Evidence presented by London City Airport’s noise consultant for the word ‘guidance’ and you’ll get 75 hits in 96 pages. Search the Airport’s Need/Socio-economics proof (88 pages) and you’ll get 2 hits. One of them is a reference to why a particular guidance doesn’t apply.

If no methodological guidance underpins an assessment, how can the public hold it to account and scrutinise claims?

The trouble is, when Mr/Ms Jones of Chuffington Place, Nowheresville presents their evidence, they immediately face an uphill battle. They will need to work hard to convince the Inspectors that their view carries at least as much credibility as Dr Fancy Suit, 40-years in the field, representing the applicant.

If they are going to get anywhere, then it is going to be rather helpful if there is at least some sort of accepted, robust, yet suitably flexible, assessment framework from which to start. This is particularly the case because, while scheme proponents usually describe their economic methods in loose terms, they don’t actually share their models, and they rarely share enough methodological detail for an external analyst to try and replicate their numbers. It is the assessment standard that allows proper checking.

In the absence of guidance, precedent prevails, even if it’s a bad precedent. Over the past couple of years I’ve had a go at documenting the performance of previous job creation forecasts made by airports in the planning process. All of the ones I’ve looked at significantly over-estimated the number of jobs that would ultimately result. In some cases they were off by thousands of jobs.

If an application descends into ‘he said-she said’ I’d argue the group with the biggest budget has a sizeable advantage. A common refrain from Inspectors is the line “there are [were] no other figures before us to assist” (City Airport Inspectors’ report p. 154). On employment this is not quite true, NEF has repeatedly argued that proposed expansions would likely produce a negligible, or potentially negative, net impact on jobs when impacts beyond the direct footprint were considered. However, at NEF we simply don’t have the budget to conduct granular employment forecast modelling. NEF’s work has relied primarily on small contributions from local individuals. Our work on most planning processes had budgets that would have been consumed in 2-3 days work by the top consultants and barristers employed by airports.

TAG should set the standard for airport planning applications

Clearly there’s a wider problem in the planning system. We need a rethink of how we do economic assessment. But, in the case of airport expansion, the DfT’s TAG is already there, regularly updated, and the product of consultation and expert input over many years. It isn’t perfect, but it’s a good start when it comes to economic assessment.

I found in the process that TAG was often misrepresented as something you either do, or do not do, i.e. “we didn’t do a TAG assessment”. This ignores its role simply as a best practice guide which can be leaned on when assessing key scheme impacts. Some of the less relevant chapters, addressing impacts which are not material in a particular case, can be parked.

Across multiple planning appeals NEF argued that assessments informed by the TAG methodology should be given weight. The airports’ representatives argued it wasn’t relevant. TAG was only meant – they said – for decisions relating to the use of public funds, it was not relevant for a private sector-funded project like an airport expansion.

The exact wording of government documentation was, when we started out, ambiguous. The TAG home page actually states "Projects or studies that require government approval are expected to make use of this guidance". A planning application requires government approval, right? Especially an appeal which has gone to the Secretary of State.

Apparently not. Barristers argued it for hours. In the first test, Bristol Airport’s appeal, we lost. The Inspectors sided with the applicant in that TAG did not apply to privately funded applications. A big part of the rationale seemed to be that “the panel is [was] not aware that any of the other recent airport expansion schemes undertook a WebTAG assessment” (p.76).

I was frustrated. This looked like (bad) path dependency. Even if TAG was not mandatory, the DfT clearly regarded it as best practice. Surely an application of such importance should be enforcing best practice?

At the Luton Airport inquiry we doubled down on this argument. But the inspectors held their line. Rejecting TAG was now precedent, and precedent sticks in the planning system. Although, in somewhat contradictory fashion, the inspectors did accept that a higher standard of economic assessment would have been useful…!

Between Luton and London City’s appeal, the DfT updated the TAG aviation unit. It looked (though this is purely speculation) like they’d been following events. It struck me, they might even share some of NEF’s frustrations at the disregard for TAG in the airports planning process.

The new guidance was stronger, it said “we expect this guidance to be useful to other appraisal practitioners considering the impacts from non-government aviation interventions”. I was (naively) confident, that assessments presented by NEF at the City appeal using the TAG method could not be so easily dismissed. I was wrong.

This brings us back to that quote from the Inspectors I shared up top. It was devastating, but it did represent progress. The new line from the Inspectors clearly suggests it is “open for” the inspectors to take TAG-based assessment into consideration. But they chose not to. The rationale for choosing not to is, at least in part, “the differences between the relevant parties”…

This rationale was a hard one to take. I am certain now, after several rounds of analysis in different contexts, that a TAG assessment of an airport expansion in the UK in 2024 produces a far less glowing endorsement of expansion compared with the ‘analysis’ the airports’ consultants are currently presenting… (which is based on no particular methodology). I’m doubly confident, because a similar type of assessment approach was recently conducted on a potential expansion of Schiphol Airport in the Netherlands, and reached the same conclusion (twice) - a negative net impact. The economics of aviation and airports has shifted dramatically in Europe in recent years.

So clearly, legal teams acting on behalf of the airports have every reason to strongly oppose attribution of weight to TAG assessments. Far from being a reason to disregard the DfT’s view of best practice, disagreement between parties is a reason to enforce it. The methods sanctioned for economic assessment should not be determined by vested interests or bad precedent.

What next?

The next two decisions in the pipeline are much larger expansions at Gatwick and Luton Airports. They are different because they are Nationally Significant Infrastructure projects applying directly to the Planning Inspectorate for a Development Consent Order from the Secretary of State for Transport. It would, clearly, be odd for the Secretary of State to dismiss a methodology designed and endorsed by her own department for understanding whether a scheme is in the public interest. NEF has contributed to both processes our analysis grounded in the TAG-based method. Gatwick’s consultants, to their credit, have already accepted the relevance of TAG and undertaken their own (albeit erroneous) analysis. Luton’s consultants roundly rejected TAG. The fact that two airport teams diverge so significantly on this point is unusual. The Inspectors’ reports and the government decision letters will make interesting reading.

As for the wider planning system, and airport expansions going through the ‘normal’ local planning route... something needs to change. These are huge, environmentally destructive proposals justified entirely on the basis of their economic benefits. Economic assessment must be transparent, best appraisal practice must be mandatory, and the public must have a standard to which they can refer when holding proponents to account. ?