Tire testing comes under intense scrutiny

Last week, Nokian implicitly admitted that it rigged magazine tests of tires. In this column I want to look at how and why it might have done that and then look at some of the implications of the admission.

Nokian admits rigging tests

First, the facts as we know them. Finnish newspaper Kauppalehti last week published an interview (Finnish language) with Nokian CEO Ari Lehtoranta in which he appeared to admit that Nokian had in the past supplied ‘special’ tires to magazines who carried out ‘independent’ tests.

The company then published a press release on Friday making the same implicit admission.

In a Tweet he said Friday had been the worst day of his life.

The buzz around the tire industry is whether this is likely to have the same impact in tires as the VW emissions scandal is having in the car world. I’ll answer that question toward the end of this piece

First, some background

Anyone who has been involved in comparative tire tests knows that tire makers take the results very seriously. A strong result in an influential test has a direct, measurable impact on sales. Sales of a well-reviewed tire can triple in the weeks around publication. Equally, many buyers will shun tires rated as ‘not recommended’ whatever discounts might be offered. Dealers have been known to cease stocking certain brands and products, following poor test results.

These test – especially those published online – are gaining in influence as people use the internet to research their tire purchases.

A good example of this came a couple of weeks ago when ACE published a tire test in which tires from GT Radial performed poorly. GiTi responded vigorously to a review by ACE which appeared to show that the GiTi label score was not justified by their test results.

Running a tire test

If you want to run a tire test, you can do it properly, or you can do it on a budget.

First is the track. You can hire a professional track such as MIRA or IDIADA, or you can politely ask Conti or Goodyear, or Nokian if they will lend you their track for a couple of days (usually at no charge).

Second is the cars. Again, you can hire cars for a few days or ask the track supplier to also supply some cars, or maybe ask a vehicle maker to supply some vehicles with a custom paint-job. Often if the test is being run on a tire company’s test track, they will have cars available.

Then there is the team of technicians required to swap tires around from one vehicle to another. Again, you can supply those people and their equipment, or you can use the technicians employed by your tire testing partner.

Almost always the magazine staff carry out the tests, but the cars need to be instrumented to extract the results. Does the magazine provide calibrated telemetry systems, or do they use the systems owned by the tire maker?

Finally, there are the tires. You can either request tires from a dozen or so different manufacturers, or buy them from a random tire store and ship them to the test venue.

A fully independent test, hiring in all the facilities and people can cost many tens of thousands of euro.

Low cost means high risk

Clearly, there is a temptation for magazines to save costs. Bringing in a partner who can provide a track, cars, technicians and telemetry, as well as hospitality, is the low-cost option.

Low cost also means high risk. The risk is that the tire partner will manipulate the situation to their own advantage. At the most simplistic, technicians can change inflation pressures. That’s easy enough to spot. But more subtly, instrumentation engineers can rig instruments.

At the other end of the scale, it’s almost impossible to spot if management arranges for their manufacturing team to supply tires with a different tread compound. I hear rumours that a well-known tire maker was excluded from a magazine test earlier this year due to discrepancies between the tested tire and the magazine's control test. That control test was carried out precisely to identify a rigged tire.

Rigging the tread compound

In its most simplistic form, a tread compound has three dimensions: wear life; fuel economy and grip.

Compound engineers can play these three against each other. A Formula 1 race lasts something like 300km, but tires might be changed two or three times during that distance. The tire treads have been formulated to give maximum grip and good fuel economy at the expense of a ridiculously short life.

No tires can be commercially successful with that kind of life. Even high performance bikers expect a life of a couple of thousand km, in addition to incredible grip.

Returning to the tire test, almost all European tests nowadays measure the three label criteria: pass-by noise, wet grip and fuel economy.

Measuring lifetime, the third side of that performance triangle, is expensive, inaccurate and subject to a range of factors from weather to driving style and road surface. I think only ADAC routinely measures lifetime in its bi-annual tests.

If the magazine has brought in a tire partner, there is the opportunity for that partner to create some special tires for the test with improved wet grip and fuel economy, but which might last perhaps 5000km instead of the more commercially viable 30,000 – 40,000 km.

As I noted earlier, everyone in the tire testing world has – to put it kindly – heard persistent rumours that tire makers frequently take advantage of such opportunities to deliver exceptional performance in these independent test.

I’m not going to name any names, but every one of the big brands has been connected with such rumours at one time or another. Nokian  just became the most honest because it has admitted the behavior.

Tests are a big deal

Ari Lehtoranta of Nokian told analysts late last year that tire test results are the main driver of sales for his company. Ask any tire marketing executive and they will tell you that good results in these tests do more for brand-building than TV advertising, sports sponsorships or even discounts, and that the benefits of a good result are increasing as more people research tire purchases online.

ADAC tests are most influential

Many motorists in the heart of Europe: Germany, Austria and Switzerland make their tire purchase decisions exclusively on the results of their favoured tire test.

The biggest player in this is the German motoring club ADAC which works with its Swiss (TCS) and Austrian (?AMTC) counterparts twice a year to run the biggest, most professional and most influential tire tests in Europe.

They published their results of Summer tires for 2016 last week. Size 1 | Size 2

Next in line in importance are the big German motoring magazines, Auto Motor und Sport and Auto Bild.

All of the above are prepared to spend heavily on tire tests to ensure their readers get unbiassed results. ADAC has its own proving ground a few km south of Hanover. I’ve met the testers at all these magazines and they take their jobs very seriously, which means removing every opportunity for tire suppliers to distort the results.

Then we get to the lesser motoring magazines. Some of these are less engaged with the tire industry and are less aware of the risks of manipulation by their tire partners.

Many of the Scandinavian magazines fall into this category. There simply is not enough money for some of the smaller magazines to put the same precautions in place as ADAC or AutoSport do.

And, I have no doubt, this is how Nokian found the opportunity to distort test results.

Impact on Nokian

Returning to the question at the top of the article, I doubt that it will affect Nokian much, or the rest of the industry. Magazine tests carry no legal weight and the results often have a large subjective element. Furthermore, there is little doubt that most other tire makers, if they have seen the opportunity, have tried the same kind of distortion.

Official testers take precautions against this kind of thing, such as the control checks mentioned above where a well-known tire maker was called out in its efforts to ‘do a Nokian’ earlier this year.

I hope that this will show the more budget-conscious tire testers that there are dangers in working too closely with a tire supplier and that future results will be more accurate.

However, the incident brings out a wider point. Much like the doping scandals in various sports and the VW emissions scandal, this kind of revelation brings the industry under a fierce spotlight.

ISO tests also under scrutiny

Even the legal testing regime for labels and suchlike is open to interpretation.

A few years ago, after the EU label law came into force, the main tire makers in Europe did some round-robin tests.

The aim was to check many different tire brands for compliance with the new labeling laws.

When I heard about it I thought that perhaps they were expecting to find that the European tire brands were all fully compliant whereas some imported brands were less so.

The results of those tests were never published. Originally I thought that was because a big-name brand had found that one of its premium-brand rivals was non-compliant.

That really would have been a scandal – a series of premium brand tire makers publishing false label claims.

In fact, that is exactly what the round-robin test results did appear to show. But the explanation is a lot more subtle than simple non-compliance by a premium-brand tire maker.

Compliant, or non-compliant?

Tires tested in a rival’s laboratory were found to be non-compliant, but those same tires tested on ‘home ground’ fell within accepted limits.

This is still a sensitive issue, but it appears that the explanation is that testing culture varies from lab to lab. There is space in the testing specifications for two labs to produce different results from the same inputs.

Cross-lab calibration

This is a well-known issue among test engineers – and it is not restricted to tires. If two labs on opposite sides of the world test the same product, there is often a significant variance from one lab to the other.

The results might depend on temperatures, calibrations, time of day, humidity, positioning of specific items or one or more out of thousands of tiny details.

Global companies compensate for these small, subtle differences by having their different R&D labs cross-calibrated and operating identical, standardised procedures throughout the world. Technicians from one lab frequently visit others to watch and experience their operating procedures. The best of them run calibration tests on a regular basis to ensure that results from one lab are compatible with results from all others.

That’s how a single global company manages inter-lab variations within its own organisation.

Round-robin tests found inter-lab variations

The round-robin test program among the big-name tire makers was like the first of these cross-calibration tests. It showed – unsurprisingly in retrospect – that one lab does not produce the same result as another.

For better or worse, test engineers from Continental do not regularly visit Michelin’s labs to learn from and match their procedures, or vice versa.

As a result, it should come as no surprise to an experienced test engineer that a lab in Ladoux will produce different results from a lab in Hannover, or Luxembourg, or Rome, or even Akron or Shandong Province.

The issue is not professionalism or competence or even accuracy. It is cross-lab compatibility, standardisation of procedures and that unidentifiable thing: lab culture.

The take-away from this is that when writing the specifications for complex tests, such as wet-braking, or fuel economy, the test conditions have to specified to an unfeasibly tight degree, allowing minimal room for variance from one lab to another.

In the tire industry, that’s all-but impossible.

Standardisation is impossible

Different companies use test drums with different diameters and surfaces for  good internal reasons. No-one would propose that all companies should standardise on a specific make and model of test drum and the associated instrumentation.

Test drums are expensive to buy and to run. If the testing authorities were to require that a test drum should have specified diameter and instrumentation, one lab might have to re-equip while another does not, which introduces competition issues.

In the end, the impact of a small tire maker trying to distort the results of a small magazine tire test may bring too much focus on testing procedures to the point where the whole business of label tests could be discredited.

What is difficult for test drums is impossible for test tracks.

Implications for global labelling regulations

China is now introducing a labelling scheme, based on the EU standards. I dread to think how far test results in China could be distorted, since the testing culture there is far less advanced than European test culture

The VW emissions scandal has focussed a very bright spotlight on emissions testing. I am not sure the EU labelling tests and associated procedures could stand up to the same intense scrutiny. I am certain that such scrutiny would highlight vast amounts of distortion if applied to China’s very limited tire tests

I doubt Nokian – or any other tire maker– had that in mind when they sought to cheat the magazine editors and those customers who trusted the results, but their ill-advised actions will surely bring the spotlight of scrutiny on all tire test procedures and results.

My thanks to Jonathan Benson at tyrereviews.co.uk/ for his support in compiling this article. He knows a great deal more about the business of testing tires than I do. And he's a much better driver than me.

--------------------------

David Shaw runs Tire Industry Research. He is a well-known and authoritative commentator on the global tire industry, from the challenges of China to Sustainability and other complex issues, he helps those interested in the tire industry to understand how it really works.