Levelling Up

Alright folks, here we go. Drum roll….For my next trapeze trick, I am going to try to shed some light on the so-called autonomous driving ‘levels’. Many of you will already have heard of the standard with the catchy handle of “J3016: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles”, first published in 2014 by the Society of Automotive Engineers (SAE)[1]. It defines six levels of automated driving, from L0 to L5, and is commonly used in the automotive industry, by journalists, and even occasionally by customers. It is a very useful document for engineers, but (with great respect to the SAE and its authors) a truly terrible one for the poor consumer trying to figure out what the hell is going on with this fascinating technology. So I’m going to have a shot at shining some light on the document, and hopefully help folk understand what’s going on beneath the geeky engineer-speak.

I admit that I dive into this subject with all the enthusiasm of a bad swimmer diving into shark-infested water, who notices as he’s about to leap that the water is also teeming with barracuda, and that a fair few poisonous jellyfish are also floating around. For this topic is a truly tricky one. Not so much because it’s technically difficult, but because many people have very strong opinions about autonomous driving, right down to the very vocabulary used. Many billions of dollars ride on interpretations of words, and this can lead to remarkably dogmatic reactions. Worse, it is almost impossible to discuss the subject without saying something about a certain Californian EV maker (sorry, Texan EV maker, mea culpa). Reactions to anything one says in public about Tesla are remarkably polarized – there seem to be no neutral reactions, and negative reactions can be remarkably strong. I was tempted to write this piece without referring to this particular OEM at all. But that would be both remiss, and slightly cowardly of me. So yes, we’ll talk about Tesla below, so I’ve got the flame-proof overalls standing by, in case I say anything objectionable. I’ll get the old ‘I’ve never worked for Tesla, nor do I have any shares in Tesla’ declaration out of the way here….and as we’re in full-disclosure mode, also mention that I am a massive fan of the company as an EV pioneer. Right, social-media-self-protection water-wings in place, let’s dive into these sharky waters. Big breath, arms out wide….here we go!

SAE J3016 is quite a difficult document to really understand. One of the traps is that is seems simpler than it is. The SAE even publish a nice simple poster-like 1-pager (see headline image) to lull you into a false sense of understanding. A quick read, and you think it’s easy. L0-L2 – ADAS. L4/L5 - Waymo, driverless cars stuff. Things with Lidars. Easy. But L3 might make you blink a bit. And a Tesla has something called “Full Self Driving?”, so that’s got to be L4, right? Err….no. And when Mercedes-Benz announces that their latest S-Class has ‘L4 technology’, then that car should soon be able to be a robotaxi like a Waymo, right? Wrong. So what seems like a simple 0-5 guide does not help us easily understand what’s really going on here. Things get even more confusing when people try to make parallels between the SAE levels and definitions like ‘hands off’, ‘eyes off’ and even ‘brain off’.

To make this more bearable, I’m going to divide the explanation into two Acts, like an Opera or one of those super-long old movies. ‘Cos I’m a nice guy, I’ll even build in an Intermission to give your brain a rest.

Health warning to the true nerds: I’m going to try to keep this reasonably untechnical. If you are an engineer looking for a far more rigorous analysis, I cannot recommend strongly enough to check out Philip Koopman’s excellent ‘SAE J3016 User Guide’ at https://users.ece.cmu.edu/~koopman/j3016/. Thanks, Mr Koopman – he’s helped me keep on the straight and narrow many a time.

Act One: Easy Stuff.

Right, let’s get the easy ‘levels’ out of the way – L0, L1, L2 and L5.

L0. This is basically a bunch of warnings – blind spot warnings, annoying things that beep to tell you that you are moving out of a lane without using the turn (indicator) switch etc. ‘Momentary’ braking intervention is allowed – so the genuinely life-saving Automatic Emergency Braking that many cars are now equipped with (and is now effectively a market requirement in Europe, thanks to the latest EuroNCAP rules) falls under L0. A car with L0 will typically have a bunch of cameras, probably ultra-sonic sensors and (especially if it has AEB), radar.

Fig 1: AEB or Automatic Emergency Brake testing - an L0 function. Image credit: EuroNCAP.

L1. This level allows the car to ‘intervene’ and take control of the braking and throttle OR the steering temporarily to help avoid an accident. So the car can briefly control the stop/go OR the left/right stuff (us boring engineers will say ‘longitudinal acceleration’ or lateral acceleration’) but not both at the same time. So, we can get handy features like Adaptive or Active Cruise Control, where the car will manage the brakes and throttle to keep us a set distance (or time-gap) off the rear bumper of the car instead. OR it can apply torque to the steering column to prevent the car drifting out of lane – often called Lane Centering or Lane Follow. But – by definition - it can’t do both at the same time. In terms of technology, the L1 car will look very much like an L0 car – a bunch of cameras, ultrasonic sensors and radars. But under the skin, the car will have electrical/electronic systems that allows an electronic ‘brain’, usually called an ADAS domain controller, to take temporary control of the brake hydraulic pressure OR the power steering. Again: not both.

L2. You’ve guessed it. L2 is like L1, but this time that ADAS domain controller ‘brain’ can handle brake/gas pedals AND the steering in an emergency situation. Now we can do quite sophisticated things. The car can now ‘lane follow’ – take a curve in the road. It can do automatic overtaking – if Active Cruise is on, and we close up on a slower car in our lane, I can ‘request’ the car to overtake it by (say) flicking a turn indicator. By the SAE L2 definition, the car’s ADAS domain controller ‘brain’ can now turn the steering wheel, ‘press’ the gas pedal and smoothly move into the next lane to overtake that car. Technically the cars will look identical to an L1 car, but the ADAS domain controller will have much higher compute power than its L1 little brother. Almost all OEMs now offer cars with L2 systems. The best-known and most (in)famous are Tesla’s “Autopilot?” and “Full Self-Driving Systems?”, Cadillac’s Supercruise? and the recently introduced Ultracruise? and Mercedes-Benz’ Drive Pilot?. But almost all OEMs now have their version of L2 ADAS – from Nissan’s Propilot ? to ?Honda’s Sensing 360? to Hyundai’s Smartsense?. So the L2 category covers a very wide spectrum of functions, with no little or no standardisation of commercial names…and this causes problems, which we’ll come back to later.

Fig 2. She's got her hands in her lap, but she's still driving. It's L2. Image credit: FR24 News.

So, to sum up so far, L0-L2 are all what are known as ADAS or Advanced Driver Assistance Systems. Not Autonomous Driving or AD. Certainly not ‘self-driving’ – a phrase the SAE specifically recommends to not use, on the grounds of it being confusing.

“Whut?” I hear you say. “The damn thing can handle throttle, brakes and steering – so it can drive, right? I saw a dude in a Tesla drive itself right across my town. I watched a video of the CTO of Mercedes-Benz on a test track in an S-Class and he had his hands in his lap! This guy Twohig’s talking rubbish…”

This brings us to a (hang on while I select caps lock) VERY IMPORTANT POINT. Why are these systems ADAS and not AD? Because you need a human driver in the driver seat. The SAE does not leave this up to the imagination. From a legal point of view, the driver of the car is the license-holding, adult, driver. This is not my opinion; it is baked into the standards’ definition.

Phew. Right. Thanks for hanging in there so far. Heavy stuff, I know. It’s about to get worse.

L5. Right. I’m skipping over L3 and L4 for a moment to do the easy one. This is a full ‘self-driving’ car – or Autonomously Driven Vehicle (ADV) to use a more technically correct term. The full 'Total Recall' Johnny Cab deal. It’s a car that can drive itself, anywhere there is a surfaced road, any time. It never needs human intervention. Now this car is a very different animal to the L0-L2 beasts above. It does not need a human driver – ever. The ‘rider’ or occupant could be a child, could be fully vision-impaired, or could be asleep. Hell, the occupant could be a piece of lumber or a pizza. If this vehicle has an accident, don’t bother trying to sue the pizza, or asking it if its insurance is up to date. It is the legal responsibility of its maker….so we are in a whole other (but very clear) legal, technical, and even – dare I say it? – ethical – universe.

Fig 3. Arnold looks quite confused. He's probably trying to figure out if Johnny Cab is L4 or L5. Image credit: Tristar Pictures.

INTERMISSION.

Please go get a tea or coffee and a biscuit or cookie. Stretch, mediate or empty the dishwasher and come back refreshed. It’s about to get a bit funky.

Act Two: Hard Stuff.

Let’s jump back to fill in the tricky stretch from L2 to L5, shall we?

L2+: Easy. Doesn’t exist. The SAE makes no allowance for L2+, L2.5, L2.x or L2-and-a-teeny bit. But these terms are often - forgiveably - coined by automotive engineers or marketing people trying to explain that their L2 system is a ‘high-function’ L2 systems, with advanced features like Tesla’s “FSD” or Caddy’s Supercruise?, not the basic L2 stuff that you’ll find in any Honda or Nissan. And journalists gratefully take up the term, trying the best they can to explain to their readers why the advanced (and expensive) system on a Benz S-Class is better than that on a Hyundai. But sorry, they are all L2. Even the best systems, that really can appear to be in complete control (when all goes well) for long periods of time, are L2. Confusing, I know.

L3: Right, this is where things get rough. The SAE defines L3 as where AD (as opposed to ADAS) starts:?note that it’s color-coded green on their SAE graphic. But then they slip in a sneaky little blue box and say ‘when the feature requests, you must drive’. In other words, the car can drive itself – until it can’t. It can - at any time - decide that it’s not comfortable maintaining control, and ask the human driver to take over.

Fig 4. Ah, so much hidden meaning in that innocuous little blue-green box. Image credit: SAE.

L3 is where the J3016 standard (and again, great respect to its authors) goes west and - unintentionally - can cause great confusion. Crucially, it does not even attempt to define the warning time the L3 car will give its human driver. So when the car says “Hey Dad, I don’t like the look of that scary cow wandering on the freeway ahead of me. I’m a bit freaked out. Can you please take over?” it does not specify whether us human Dads (or Moms) have 30 seconds, 3 seconds or 30 milliseconds to react, stop admiring the scenery or that passenger and grab that wheel. And this is where we get into grey areas of ‘hand off’ ‘eyes off’ and ‘brain off’. L3 has had a very variable existence. While there are a bunch of L0, L1 and L2 cars in production, there are currently almost no L3 vehicles. Some folks claim that Audi and Bosch already had L3 technology ready to go in their top-of the line A8 models as long ago as 2017. But they never turned it on. Why? For the obvious legal reasons. If there is a crash, who is liable? The answer is sufficiently vague to have scared off everyone – especially the cautious Legal Department of VAG, possibly still a little gun-shy after ‘Dieselgate’.

Now M-B seems to be on the edge of turning the systems on in their 21MY S-Class…very cautiously, starting in Germany, certainly not in the scarily litigious US of A. But Honda have stolen a march on them, and announced that their Legend is the ‘worlds’ first production L3 car’. But Honda have dipped a very cautious toe indeed into quite shallow water: 100 cars only, sold only on their home turf of Japan. Such caution – especially with geography – clearly demonstrates the first major problem with L3: it’s as much (maybe more) of a legal challenge as it is a technical one.

Fig 5. The word's first L3 car - Honda's Legend EX.

But L3 does have a big technical challenge also. It’s called ‘fallback behaviour’, and it’s a particular issue on high-speed roads. Let’s say that the car is driving on a freeway, autobahn or autoroute, in L3 mode, at 130 kph (80 mph) – a function often called ‘Highway Pilot’ or something similar. It’s a two-lane roadway, and there is no hard shoulder. Stopping is illegal. These conditions are very common, all over the world. The car is well designed – it’s got a full Driver Monitoring System (DMS) with a high-quality interior camera, able to track the driver’s gaze. Obviously, it does not rely just on some simple ‘hands on wheel’ torque detection – that would just be silly, right? Now, let’s say it detects a problem…maybe something as simple as a flat tire. It tries to wake up the driver. It beeps, shakes the wheel, shakes the seat, does all that it can to wake the hairless ape in the driver’s seat. But its driver gaze monitoring systems can see that he/she is not awake. Maybe fast asleep? Maybe fainted? Maybe a heart attack? This is sometimes referred to as a ‘hands off and eyes off’ condition. The SAE does not give a standard of how long this ‘eyes off’ condition is acceptable – seconds? Minutes? Half an hour? So what does our poor, touchingly anthropomorphic L3 car do? In town, this is an easy problem. If we were in 'Traffic Pilot' territory – on urban, speed-restricted roads, it would be a piece of cake. Brake the car gradually to a stop in the road, bang on the warning hazard flashers, unlock the doors and wait for the ape’s other ape friends to come put it on a low-loader. But we’re on a 2-lane autobahn or freeway, remember? You can’t just ‘stop in lane’- to do so would be to invite a heavy truck to come in the back window at 100 kph: not good. So an L3 car has to be able to keep driving, to the next exit – which may be 30 km ?(~20 miles) or even 50 km (~30 miles) away. It then needs to be able to take the slip road or exit, and drive into the nearest town or quiet country road. Now it needs to be able to identify a safe spot where it can stop, finally put on those blessed flashing lights and call for help. So, in short, it basically needs to be able to ‘drive itself’.

Readers, we just described L4. So in short, building a safe L3 car is technically just as hard (and just as expensive) as building an L4 car – i.e. very. If you’ve really cracked L3, you’ve more or less cracked L4. ?

These legal and technical/cost barriers to L3 mean that many industry experts now believe that L3 will only exist widely after L4 has been truly mastered. So L3 is really a very problematic zone of the SAE chart- and all of that difficulty is captured in that cute little blue/green box in their 1-pager…

L4: OK, relax, this is easier than L3. These are the truly autonomous cars (or ADVs) that you see driving around, operated by the likes of Waymo, Cruise, Aurora, Argo Ai, Baidu, WeRide, Pony.ai and others. They will NEVER ask a human being to take control – unlike L3. They do not need a human driver to operate them. It is not an accident that the worlds’ first public-road L4 drive (Google’s ‘Firefly’, 2015) was with a fully legally-blind occupant, Mr Steve Mahan, on board. Waymo were making the point to clearly separate their efforts from L2 and L3. These cars are incredibly sophisticated. With no exception, they all use the familiar triptych of sensor – optical cameras, radars and Lidars. Crucially, they have highly sophisticated redundant systems – steering, braking, drive units, electrical power supplies etc.

Fig 6. Mr Steve Mahan and Google make history - the world's first open-road L4 'drive' in Austin, Texas in 2015. Mr Mahan is seriously visually impaired. Image credit: Waymo.

Let’s pause on that word ‘redundancy’ a moment. Hardware redundancy is often underestimated in the discussion/confusion between L2 (ADAS) and L4 (AD) systems. And here I have to go back to that famous Texan T-company again. You may well believe that your Tesla will one day be able to ‘drive itself’ without Lidar – after all, you can drive, and you are equipped with just two relatively crappy little cameras mounted in the front of your head. You may well believe that the ‘FSD? computer’ is really powerful enough that one day, the OTA updates will make it ‘clever enough’ to drive – and that your Model S will morph somehow from an L2 caterpillar into a beautiful L4 butterfly robotaxi, out there earning you tons of cash while you kick back. And the debate between believers and sceptics often rages around the very visible and glamourous topic of Lidars. But let’s talk about a far less sexy topic – steering racks. In an L2 (or L3) car the backup system is case of a power steering pump (or motor) failure are your biceps. If the EPAS motor in your Tesla fails, no biggie: you just apply a bit more steering effort and you will be able to safely steer it to the kerb and safety. But in an L4 (or L5) vehicle there is no human backup. You might be asleep. Or a kid. Or non-seeing. Or a pizza. A pizza cannot apply the backup torque to the steering wheel to steer the car to safety. So you need fully redundant steering – two pumps, or motors, each with their own electrical supply. So while I allow that your Tesla S might conceivably one day download the Mother of All Software Updates and be able to drive ‘itself’, it’s highly unlikely to ever receive a new steering rack or brake booster OTA. Sorry for the bad news…

Fig 7. Tesla Model S steering rack. You'll need one hell of a bandwidth to download one with two assistance motors...

OK, what’s the difference between L4 and L5 again? Those two words – anywhere and anytime. An L4 car does not claim to be able to drive anywhere or anytime. It can legitimately ‘decide’ that it’s not safe to drive – either on roads that do not fall within its Operational Design Domain (ODD – sorry, more engineer geek-speak), or, say, in weather conditions that don’t allow its sensors to 'see' clearly enough. In this, it’s analogous to humans – my Mom would not be comfortable driving on certain mountain roads I know in the French Alps. I myself have been forced to pull in and park a car in white-out fog in the UK, and once in torrential rain in the Japanese Alps. For these reasons, many AD experts believe that L5 may never exist – that it’s a stretch too far: and with arguable benefits above and beyond L4.

“Hold up, there!” I hear you say…the coffee from the intermission might have worn off. L4 sounds like L3 here. So an L4 car can ask you to take over if it’s too foggy? So that’s what those safety drivers are doing in the Waymo/Cruse/Argo cars I see driving around? Nope. It can’t. Again, by definition, once it’s started the driving task (within its defined ODD) an L4 car will NEVER ask a human being to take over. Ever. The safety drivers are present in these cars simply because they are all test prototypes. There is human being in the front seat to intervene if the under-development software or hardware were to have a problem. So if you want to see true L4 in real commercial operation, on open roads, as far as I know you still have to travel to Chandler, Phoenix Arizona, and take a ride in a Waymo One ‘rider only’ L4 car: with no human being in the driver’s seat.

Fig 8: to the best of my knowledge, still the world's one and only true, commercial, rider-only L4 service on open roads: Waymo One's operation in Phoenix, Arizona. Watch this space in San Francisco and other cities. Image credit: Waymo.

L WTF? OK so far, so good. We’ve got the ADAS L0-L2. We’ve understood L5, and why it might never exist. We’ve even navigated the swamps of L3, without being caught in the ‘hands off’ quicksands. But there are a few other confusing things you might run into.

Auto Valet Park, for example. What’s that? You get out of your Benz and it parks itself. This smells like L4, right? There is no human being in the car, yet it’s parking itself. ?It can’t be L2 or L3, as there is no human to intervene if things get bad. And you are right, this is a horrible grey area. This is why Mercedes-Benz speak of ‘L4 technology’ in the 21MY S-Class – because their Auto Valet Park function uses Lidar (normally associated with L4 vehicles) to ‘see’ precisely enough to navigate into a tight parking spot at low speeds. Very careful use of words by M-B's communication folks, but adding to the general consumer confusion, unfortunately.

Right folks, you’ve stuck with me so far, and unless you are both a saint a scholar, your head is probably spinning. If only there was a simple way to ignore all these damned levels and simplify the whole kit and caboodle. Guess what? There is…it’s called ‘Roy’s Razor’.

Fig 9: Need something to cut through the fog of the various levels? We have right what you need...step this way.

Named in honour of Occam’s Razor, I must give full credit here to the excellent Autonocast podcast, with Ed Niedermeyer (notorious Tesla fanboi), the always-amazing Kirsten Korosec, and Alex Roy, famous contrarian, driver-philosopher and director of 'Race Across Amercia'. If you are even vaguely interested in ADAS/AD (and you are…because you are still with me) I strongly recommend this ‘cast, available in all reputable PodStores.

Alex Roy, in one recent episode of the podcast posited the following question, which his co-presenters dubbed ‘Roy’s Razor: in short, it is “Can I safely sleep in the back seat?”. If the answer is YES, it’s an ADV or autonomous vehicle – L4 or L5. If the answer is NO, it’s an ADAS system, somewhere between L0 to L3. The beauty of this seemingly simple question is that it’s an infallible, no-BS, binary test, as faithful as litmus paper.

Alex could equally have asked “Do I need a driver’s licence?” or “Am I kid?” or “Am I seriously vision-impaired?” or indeed “Am I, in fact, a pizza?”, but I’ll stick to the original version in homage to Roy and co-hosts.

CONCLUSION

The SAE J3016 standard was (and remains) a very useful guide to professionals in the ADAS and AD industries. But it is really not useful to communicate the real impact and use cases of these technologies to non-specialists. In particular, the grey areas around L3, and the lack standardisation in naming ADAS functions allow great confusion to be generated – deliberately or not – by companies overclaiming the capability of their ADAS systems.

The only way to progress of to be super-disciplined in in clearly separating ADAS and AD technologies. Roy’s Razor is a great way to do this.

Thank you for your attention – and endurance. My next article will be shorter and simpler. Pinky promise.

[1] A much more detailed history of the J3016 standard can be found here, along with lots of other useful information:

https://technologyandsociety.org/its-time-to-rethink-levels-of-automation-for-self-driving-vehicles/