How to deal with exponential complexity in automotive engineering

While the automotive industry has been dealing with increasing sophistication in the past years (comfort, safety, regulation…), some were hoping that, with electrical powertrains replacing traditional internal combustion engines, automotive engineering would get simplified. Unfortunately, this is not so simple. This would be underestimating the complexity driven by the shift of the auto industry from hardware to software, with the emergence of many technically advanced capabilities.

On top of electric drivetrain disruption, automakers are facing rising customer demand for seamless Connectivity solutions, Energy Management capabilities, as well as more autonomous features: from partially autonomous Level 2&3 features such as Lane departure warning, Traffic jam pilot, Autonomous parking… to fully autonomous Level 4&5 features like Intersection autonomy. They are also facing increasing constraints & regulations for car software (SW) & electrical/electronic (E/E) architectures, such as safety standards, CO2 emissions objectives, data management & protection of personal data, risks of liability for accidents caused by Autonomous Vehicle (AV) features, or mandatory cybersecurity upgrades.

As a result, software in modern cars can contain more than 100 million lines of code. In comparison, software for a modern commercial airline contains “only” 14 million lines of code. This will grow even further along the capacity of electronic control units and the number of sensors from camera to advanced technology, like LiDAR-based long-range perception presented during 2022 CES.

How to deal with this considerable complexity? To achieve this, engineers need to resolve key questions: why is this complexity fundamentally different? How to best contain it??

THE COMPLEXITY CURVE IS GROWING EXPONENTIALLY

The first challenge: complexity in the mechanical world

Until the beginning of the past decade, auto engineering was mostly about mechanical design, with E/E and SW limited to basic infotainment, body, security, and safety features, that were conveniently siloed and managed through control systems. Most of the complexity was driven by mechanical systems interacting with one another. For N mechanical elements, maximum complexity could be modelized as N elements interacting with N-1 elements, resulting in?Complexity following an N2 curve.

Such complexity was already beyond linear growth under incremental meta requirements. Take the car bonnet example: requirements used to be only about “Open & Close Durability”, “Legal & Safety” and “Optical Quality / Dimensional”. Now on top of that, requirements include “Optimized Weight”, “Aero”, “Engine Breathing”, “High-Speed Stability”, and more.

So far, such complexity has been managed with system engineering and appropriate automation thanks to integrated toolsets and proper curation of the Body of Knowledge. Much more modern & efficient systems can still be deployed to support meta system requirements, design, analysis, verification, and validation, leveraging carry-over approach with the example of Model-Based Systems Engineering (MBSE) and tools like accelerated CAE and digital simulation.

The new challenge: complexity in the new SW and EE world

Automakers also have to deal with new software-centric advanced systems that are fully interconnected with the mechanical stacks, including ADAS or Energy Management interacting with powertrains and chassis systems. For example, a software should activate a braking system, dynamically handle individual wheel angle, monitor and share information on tire pressure, manage transfer of forces and vehicle position activating chassis actuators accordingly. Same interconnection is required between on-board and off-board systems, to deliver diagnostic services for preventive maintenance, for example.

Complexity is not driven by parts interacting with each other anymore, but by a broad and versatile combination of interconnected elements (composed of numerous software services from modular software architectures, sensors, actuators, and other mechanical parts), each with their own life cycle versions and status. For N software services & mechanical parts, maximum complexity could be modelized as a combination of N elements with 2 possible statuses (on/off), resulting in?Complexity following a curve in 2^N. Such exponential complexity increases much faster than in the mechanical world: with 10 elements, EE/SW complexity would be measured as 2^10=1.024 vs. 102=100 for mechanical complexity!

Dealing with this new complexity has become critical, notably to deliver new features expected by customers. Development costs have surged, software has already jumped from 20% to 40% of total R&D costs in the past 12 years, while R&D thereby went up from 30% to almost 45% of total expenditures. Besides, engineers are facing several critical issues with great development delays for new vehicles, decreasing quality –SW becoming the primary cause of customer dissatisfaction for autos– increasing security issues, unpractical production processes–non-automation-friendly and space-critical (inflating numbers of controllers as well as wiring)– and lower maintainability.

As a car has turned into a set of several computers on wheels, software enabled features are testing the limits of today's engineering systems, pushing for a complete architecture rethink.?

CONTAINING COMPLEXITY "BY DESIGN" IN MODERN CAR ARCHITECTURE

Thinking the car like a high-tech product

A modern car is a high-tech product comprehending a hundred of computers with different capabilities (from great HMI for infotainment, to state-of-the-art latency for autonomous driving or best algorithm for energy management). However, a car is much more complicated than a computer or a set of several computers in the sense that it delivers many more services than computing or than a telephone, by moving large pieces of metal to carry passengers in proper safety conditions.

As a high-tech product, one can decompose the car into a tech stack:

• On the top, sit digital applications, as well as physical applications for any user (e.g. driver, passenger, repairer) to engage with: digital dashboards, switches, steering wheel etc.
• At the bottom are the passive “metal parts”, that actually move…
• Between the two sits the middleware. Yet, this one is far more complicated than in a computer. On the one hand, the E/E platform takes on user's or external input to drive the orchestration. On the other hand, this orchestration has to go through very complex mechanical systems, such as the chassis, to actually activate the “metal parts”.

With this difference in mind, engineers can transpose many of the frameworks used in computer engineering to the automotive.

?Shifting to a “product & platform” approach

To rethink EE/SW car architecture, it is critical for automotive engineers to change their development philosophy. Until now, EE/SW conception was managed along the lines of traditional vehicle development: building a bespoke integrated system conceived as a monolith within a fixed scope and timeline, following a “project/waterfall” approach. Automakers have to?shift towards a “product & platform” approach?like in Consumer Electronics industry, building a set of constantly upgraded components and systems, decoupled from the actual vehicle programs, to meet the needs of the market leveraging the continuous update of the EE/SW stacks, even after actual vehicles are delivered to final customers.

This “Tech product” approach, relying on product iterations and improvements, allows to deliver relatively simple systems to start with, and add or enrich features and capabilities iteratively with updates independent from the underlying hardware. This is only possible with a “platform” architecture, relying on compartmentalization principles, aiming at reducing complexity and enabling strong decoupling.

To do so, auto engineers must start building a “product map” to manage the constituents of the car (i.e., elementary bundles of self-standing physical or virtual capabilities providing an actual value to an end-user or to another set of components & systems that consumes it). Leveraging?platform?architecture principles?popularized by the Consumer Electronics industry in the 2010s, such product maps should be organized around 3 distinct technology layers:

• Application layer, regrouping all on-board and off-board features, focusing on driver and passengers’ interactions, as well as business-driven applications.
• Service & platform layer, regrouping:

o??Hardware platforms core modules and hardware activators, driving the behavior of the car and its attributes, including energy systems, chassis systems and body articulation systems (hard points).

o??All new transverse services consumed indirectly by the application layer to assemble customer facing features, as well as on-board operating systems, frameworks, and capabilities to manage and orchestrate “elementary” hardware components (such as actuators, sensors, EE physical parts and purely mechanical parts), and off-board (cloud) data and computations capabilities.

• Hardware layer, with low-level electrical/electronic components, electrical distribution system, power connectors, sensors (cameras, radars…), actuators, as well as physical components (such as, for example, for brakes: brake pedal, brake pipes, brake discs…).

Such a holistic map is a powerful tool to identify dependencies between products, and manage the interfaces managed in a repetitive way between the subproducts, leveraging APIs between Services and Features, and Abstraction Layers between Services and Hardware. It clarifies the understanding of delivery ownership, identifying which teams currently deliver / or should deliver which products or groups. It can also be leveraged to visualize strategic decisions for in-house development vs. externally procured standard items or specific partnerships (make/buy/partner strategy), as developing everything in-house is very costly.

Each element of the product map must have a clear 3-year roadmap, allowing progressive value delivery along a continuum, while developing highest value first. Clear dependency between roadmaps must be identified to ensure consistency between platform enablers (shared services, middleware), supported features within customer-facing applications, and consistency with Vehicle Program key milestones.

With this new product approach of their EE/SW capabilities, auto engineers can reconsider their architecture to bridle complexity. From a conceptual standpoint, they can pull 3 levers:

A.???Compartmentalize elements into subsets (e.g., split N into A and B). Even if subsets are interconnected with a multiplicative factor (K), it will still result in lower complexity (2^A + 2^B) * K < 2^N.

B.????Maximize standardization to restrain complexity, by stabilizing element contents through carry-over.

C.????Reduce the number of elements (N -> n), with complexity 2^n < 2^N.

A. Compartmentalize elements

Taking advantage of their newly defined product map, engineers can engage 5 workstreams to contain EE/SW complexity compartmentalizing elements:

1.?????Decoupling EE and SW leveraging Hardware Abstraction Layer (HAL) frameworks

2.?????Centralizing EE architecture with less ECUs?around high computing central processor(s)

3.?????Virtualizing OS & computing capabilities based on serverless concepts

4.?????Modularizing SW with service-based architecture principles

5.?????Integrating secure-by-design architecture principles

1. Decouple EE and SW

Decoupling SW and HW is mandatory to enable continuous software innovation through “Software/Feature Over The Air” updates (SOTA/FOTA). To do so, next generation onboard software architecture should leverage a?Hardware Abstraction Layer?(HAL) to separate the logical platform (base OS, hypervisor, diagnostic services…) from the hardware (circuit board, microchip, discreet electronic components), resulting in an agnostic interface. Android HAL framework is a good example in the phone industry. It defines a dedicated standard interface for each hardware capability: Android HAL Camera, Android HAL Storage, Android HAL Bluetooth, Android HAL Audio… allowing mobile apps to be developed and updated no matter the hardware vendor-based component in the phone.

That requires, on the mechanical side, to compile the list of modules and key hard points of the mechanical “skateboard” platforms, and split them, from a “platform” perspective, between Software layer and Hardware layer (actuators, sensors, non-SW/E/E parts). That way of thinking is different from what one used to see in the pure mechanical world: it creates the conditions to deploy system engineering natively across the whole vehicle stack. One of the best examples of the benefits of system engineering at scale for vehicle development is the “Super Bottle” concept developed across thermal cooling systems in Tesla cars. Such an approach brings mechanical engineering much closer to software engineering. Just like object-oriented programming (a concept widely used among programming languages such as C++, Java, Python, etc.) was created to manipulate normalized conceptual objects, elementary mechanical components are actual objects that can be manipulated by engineers.

One of the main challenges for auto OEMs will be to revisit progressively their component-based sourcing strategy. Many tier-1 suppliers are providing components with a full layered stack ECU acting as a "black box", leading to integration issues, and preventing agility and speed in updated SW features delivery. Let’s take the example of a standard “Seat Heat Module” ECU. Not only does it contain electronic hardware and its related bootloader (software code to install and launch the ECU), but it also comes with many other embedded “middleware” software components, including operating system, memory management, diagnostic, networking, electronic hardware interfaces, cyber capabilities… and, of course, its own “application” software that provides features and functionalities, in our example to control seat heating. To truly decouple SW/HW, components must be sourced only for the set of services they provide (i.e., “passive” electronic hardware and its bootloader), and all intelligent services (operating systems, diagnostic services, cybersecurity services, background software update services) must be unified within the shared “middleware” tech stack on the new platform.

2. Centralize EE architecture

Moving from decentralized & redundant ECUs distributed across the vehicle, to a unified EE architecture centralized around one or few central processors with high computing/memory capacities and advanced gateways, thereby, making it possible to deliver power and data across several domains, sensors, or devices, is mandatory to support advanced capabilities such as ADAS Level3. It is also a strong lever to support further integration & sensor fusion, improve self-configuring / plug & play zones, as well as next generation communication (e.g., wireless). It offers higher potential for standardization as well as significant manufacturing cost reduction. First progressive steps towards such centralization could be to start implementing a few Domain Controllers consolidating inputs from domain-internal ECUs, to aim for lower complexity and lower costs within each domain, using advanced gateways to drive cross-domain interaction.

3. Virtualize OS & computing capabilities

Once EE architecture has been decoupled and centralized, auto engineers can virtualize their central computing capabilities and base software, with open-source orchestration systems like Kubernetes. While this type of technology has brought strong innovation in the Cloud & Datacenter ecosystem, using on-board production-grade container orchestration technology is not science-fiction. In December 2021 an Air Force pilot flew a T-38 Talon with operational flight software utilizing the Kubernetes framework installed on the instrumentation system of the aircraft, running on-flight live updates to the software without any interruption. Such containerization frameworks with the proper level of automation, offering onboard infrastructure as a service, not only will enable rapid software development, but it will also improve the overall system stability, as the framework can switch automatically to a stable version in case of failure. It could also manage scalability, either leveraging central computing capabilities, or switching back and forth between on-board and off-board capabilities, without any impact to the user.

4. Modularize SW

On top of OS & Computing capabilities, auto engineers must design their applications following a modular service-based approach, to promote re-use and accelerate software delivery by reducing development dependencies. A “service” is a shared software component that implements a consistent and related set of either a technical function (e.g., “Software Update Manager” service) or a functional function (e.g., “Localization” service). Such services should be agnostic to technology, able to issue and receive events, deployable in an independent manner, allowing for orchestration to create higher level services. Most importantly, it should expose its resources through APIs, creating an access layer for other services, which strongly contribute to the overall system scalability and agility.

5. Integrate secure-by-design architecture principles

With increased systems fragmentation and connectivity (Software, Hardware, Cloud, OTA) comes a fast growth of cybersecurity vulnerabilities. As auto engineers must redesign their architecture, they have the opportunity to implement native cybersecurity-by-design principles to create safer cars. Identifying, protecting, detecting, responding and recovering from cyber threats while protecting privacy. Those principles are not only to be embedded in the technology stacks, for example protection systems against unauthorized access, or in-component/in-vehicle response mechanisms (e.g., alerting, "limp home", safe shutdown, etc.), but auto OEMs must also upgrade their skills and processes, with new capabilities like a Product Security Operations Center (PSOC) for incident classification and response triggering.

B. Maximize standardization

EE/SW standardization is a major lever for auto engineering to ease complexity, accelerate development and reduce costs. Such standardization could be put in place:

1.?????Implementing a standard EE/SW platform across all brands and vehicles

2.?????Leveraging standard cross OEM open-source platforms or systems

1. Implement a standard EE/SW platform across brands and vehicles

First step comes with the standardization of the Software Development Life Cycle (SDLC) leveraging DevOps principles and tooling, with all SW development teams not only using the same Source Code Management tool, but also the same instance of code repository. Auto engineers must also implement clear tagging of released versions of the code, coupled with a strict version management policy, to minimize the number of software / features versions running in vehicles.

For onboard platform, it means clear versioning of the adjacent electrical/electronic architecture, to allow new vehicles programs to pull the latest version available to be adjusted for each vehicle platform geometry. To be able to only run a very limited number of SW versions in their vehicles, automakers must anticipate upcoming software obsolescence and improve maintainability for the longest period of time. To do so, EE architecture must over specify both the underlying server capacity (computing, memory), and the hardware components as well (actuators, sensors…), no matter which SW option is activated or not by default, to support future SW updates/upgrades.

As over specifying server and hardware is costly, a “one size fits all” core Car OS (base SW and server capacity) might not be adapted to all vehicle segments. For most OEMs, today’s focus is on building a first version of their core Car OS. Once they will have gained sufficient maturity, if they want to maximize cost versus obsolescence, they will need to review their variant strategy for core Car OS, with, for example, a “Low-Tech Car OS” that won’t require much computing capabilities and EE components. Such Car OS will never be able to support advanced ADAS levels but can still support evolutions of SW features for basic transport conditions, light payload, or limited vehicle performance, more adapted to low-cost vehicles.

Finally, for their offboard platform, engineers should leverage SaaS players’ best practices, implementing a multi-tenant architecture, with one single Cloud backend serving several brands and vehicles, using load balancing and automated and dynamic scaling.

2. Leverage standard cross OEM open-source platforms or systems

Open source has been driving the Tech industry for several decades, supported by open collaboration, and fostering innovation. Auto engineers can already extensively use Open-Source frameworks, following the example of Tesla who developed its Car OS based on a full version of Ubuntu (open source, Linux distribution). Complete off-the-shelf standard platforms don’t exist yet on the market, despite recent announcements. A good example is Redhat’s objective to deliver the first continuously certified Linux Platform for Road Vehicles, teaming up with?ARM?on the?Scalable Open Architecture for Embedded Edge (SOAFEE) initiative. Yet, auto OEMs already have strong open-source alternatives, mostly for their Infotainment domain as with Android Automotive OS (AAOS), while other initiatives are rising, with open-source projects focused on the Base SW & HW abstraction layer (Automotive Grade Linux), or in the Autonomous Driving stacks (the Autoware foundation).

If, today, most auto OEMs are still focusing on engineering their own proprietary Car OS to gain software maturity with the objective of building key differentiators on the market, instances in other industries have shown that leveraging full stack open-source platform is not only a strong acceleration lever, but it doesn’t prevent from value differentiation either. Let’s look at the drone industry, where 60% of non-Chinese drone makers have turned to the PX4 Autopilot for Drones open-source community, based open-source HW and SW stacks to manage everything from flight control features and communication protocols to battery management systems. Doing so, they have been able to reduce their time-to-market and their cost, while benefiting from reinforced continuous evolving standards to manage their cybersecurity risks. While they share a common and standard HW & SW stack, each of those Drone OEMs are now focusing their efforts on developing their own value-added third-party applications, be it in consumer or industrial applications.

C. Reduce the number of elements

In parallel of platformization & standardization, the number of elements must always be reduced. This can be achieved by aiming at maximum simplicity, reducing the number of components, systems & features, minimizing customization, and limiting variants while turning it into positive experience. Just like Apple did with its first iPhone when they removed the physical keyboard, with only 1 tactile button (that has been removed since), Tesla has been leading the way in car interface simplicity, with only 2 switches in their cockpit. Besides, just like the latest iPhone 13 has only 4 variants (based on screen size vs. camera type vs. chip performance), combination when specifying a Tesla is limited to 2 Powertrain variants, 2 wheels options, and 4 features for add-ons (excluding exterior & interior colors variants).

Another way to reduce the number of elements is to leverage off-board capabilities to move complexity outside of the car. Most OEMs have started to virtualize key capabilities and moved them to the Cloud, mostly to store and compute data, especially for non-safety related capabilities like navigation calculation. New features, such as ADAS, which are increasingly requiring computation capacity, and complexity while facing the physical limitation of car body space and weight, could be next candidates for offboarding. Yet, real-time requirement and connectivity issues are currently limiting offboarding opportunities. In the future, a solution could be to offboard those capabilities in near-edge infrastructure (starting with highway for example, or city infrastructure) acting as a multi-agent system. This would require creating ecosystems with automakers, infrastructure players and regulators and aligning on key standards regarding cooperative sensing & decision making.

CONCLUSION

As the cars are becoming closer to robots, mastering the exponential complexity of new car software & electrical/electronic architectures has become table stakes for auto OEMs to build their next generation vehicles, opening the door to a new field of possible, unlocking the full potential of artificial intelligence.

It requires a complete architecture rethink, that should also come with maximum standardization and components/systems simplification, to focus differentiation on key added-value services. We believe OEMs can successfully operate this shift if they change their development philosophy, implementing a “product & platform” approach. As operationalizing such a transformation is highly complex, OEMs should implement agile at scale coupled with system engineering. Incremental and iterative approach accelerates delivery, enables steady platform improvement while maintaining business alignment.

To master the full stack of such a high-tech product, upskilling is paramount.

?

?Authors:

Vanessa Lyon, Managing Director and Senior Partner

Mika?l Le Mou?llic, Managing Director and Partner

Jean-Christophe Laissy, Partner and Director

Szymon Walus, Partner and Associate Director

Laurent Alt, Associate Director

Rapha?l Giraud, Project Leader

?