Automotive Software Development: Best Practices and Professional Insights

Executive Summary

This article examines the current software challenges faced by the automotive industry, focusing on hidden errors and performance issues in control units. It explores the historical context of software development, the shift towards in-house software creation, and the increasing complexity of highly integrated control units. The article highlights the necessity of early-stage integration testing and continuous integration environments to manage these challenges effectively. By adopting these practices, OEMs can ensure higher quality, better performance, and a more efficient development process, ultimately leading to reliable and well-performing vehicles on the road.

The overarching objective of the software-defined vehicle is to be prepared to deploy software every time. This requires centralized, networked IT systems and a modern software mindset based on system engineering expertise.

The Current Situation in the Automotive Industry Regarding Software Issues

When examining the current state of the automotive industry, a recurring theme is the prevalence of software issues. However, the specifics of these problems are often not thoroughly addressed in media reports.

There are two core issues identified from these accounts: Hidden Errors and Performance Issues.

Core Issue 1: Hidden Errors in Software

There are numerous hidden errors in the current software versions that are only discovered during vehicle-level acceptance testing on the Hardware-in-the-Loop (HiL) systems. These concealed bugs often surface late in the development process, creating significant delays and complications. Many of these errors could have been detected during earlier testing stages, such as software integration tests. The general core idea in testing is that at later stages, like vehicle testing or HiL, only those errors should be uncovered that arise due to the higher level of completion or integration. For instance, during the integration of control units into the vehicle, issues related to electrical/electronic aspects, such as bus communication or temperature effects on the hardware, should be identified. Functional errors should predominantly (estimated at over 95%) be detected in earlier testing stages.

It makes sense to conduct functional tests before performance tests to ensure that the basic functionalities are working correctly before focusing on performance optimization. By verifying that all fundamental features and functions operate as intended, the development team can be confident that the system's core requirements are met. This foundation allows for a more efficient and effective performance testing phase. If basic functionalities are flawed, performance testing would yield unreliable results, making it difficult to distinguish between issues caused by functional defects and those stemming from performance inefficiencies. Additionally, resolving functional problems first helps streamline the debugging process, as performance issues often compound existing functional errors, making them harder to isolate and fix. Therefore, a sequential approach, addressing functionality before performance, ensures a more structured and manageable development and testing process.

Core Issue 2: Inadequate Software Performance on Control Units

The performance of software on control units is often insufficient. The software is not optimized enough to run efficiently on the control units, leading to performance bottlenecks and system failures. To address performance issues, the entire software architecture must be examined. Evaluating and making meaningful changes to the architecture requires mastering the integration process, including testing, which should be handled in-house for better control and understanding.

When designing hardware requirements, it is often unclear what resources will be needed. Estimates are made based on best knowledge and experience, but during software development, it almost always becomes apparent that these resources are insufficient. This is because the software is not always implemented optimally, and the context has become so complex in recent years that it is no longer possible to linearly estimate resource consumption. Consequently, there is a constant need to scale back on features and optimize usage.

Sustainable software performance management also requires a deep understanding of hardware layers such as processors, memory, access times, latencies, and bus interfaces. An effective performance optimization process is inherently iterative. However, the current situation often impedes the addition of new features, as the iterative process is not well-managed, and resources are frequently underestimated. Addressing this requires a more integrated and informed approach to both software and hardware development, ensuring that performance issues are identified and resolved early in the development cycle.

To understand and address these issues, it is crucial to consider how software integration is currently performed and how these performance problems can be mitigated.

Current Software Integration Model

The traditional model involves the Original Equipment Manufacturer (OEM) delivering software to an integrator, who is usually an external party contracted by the OEM. The integrator’s task is to develop the operating system, integrate the application onto the control unit, and ensure the system functions correctly. This model has worked, albeit with challenges, for smaller control units. However, the introduction of highly integrated control units has made the situation more critical. The volume of software, components, and interfaces has grown exponentially, making the system complexity unmanageable.

In many cases, software and system architecture do not dictate the integration of various applications and components onto a control unit. Instead, these elements are often cobbled together based on project models, which are not testable at the software integration level.

Challenges in Integration Testing

The integration lead typically has requirements outlining how the overall system should function, but there are no comprehensive tests to ensure its reliability. External parties handling the integration cannot perform early integration tests to identify errors when the software is delivered, nor can they ensure that the software works seamlessly with other products, processes, modules, and components.

The lack of detailed knowledge about the system’s functioning means that meaningful tests cannot be designed. Testing at the control unit level is too late in the process. By that stage, other software components may obscure some paths of the software, making it impossible to stimulate certain error states or performance issues externally.

Debugging specific points within the control unit is excessively cumbersome, akin to painting a room through a keyhole. Effective testing on highly integrated control units at the external interface level is unfeasible.

The Ideal Approach

The proper method involves conducting small, targeted software integration tests directly during software integration. This approach allows for immediate verification of interface functionality, response to input signals, error states, and robustness measures. If this is not done, issues remain undetected, leading to the problems observed today.

Proposed Solution

The solution lies in establishing IT and process frameworks that enable continuous integration and testing at the OEM level. Every software check-in and update should trigger immediate, meaningful integration tests within a Continuous Integration (CI) environment. This system should instantly identify software package changes, validate new features, and ensure existing functionality remains intact. If issues are found, they should be addressed immediately.

This method is seen as the only viable way to manage the increasing complexity. Various OEMs that have adopted this approach confirm its effectiveness in handling the intricate software landscapes of modern vehicles.

Why Are We in This Situation?

Historical Context and Evolution

Fifteen to twenty years ago, traditional OEMs followed a standard model for software development and integration. They specified control units, detailing the electrical and electronic interfaces—power, voltage, signals, and actuators. Suppliers received these specifications in the form of requirement documents, which also included functional descriptions, parameterization options for calibration, and instructions for adapting software to various scenarios. Suppliers were then contracted to develop software based on these requirements.

During certain milestones, referred to as prototype phases, the OEM would receive software versions for the control units. The OEM would then flash the software onto the control units and conduct initial tests on HiL systems or sometimes directly in the vehicle, validating functionality and performance relatively easily.

?

Shift to In-House Software Development

As systems grew more complex, OEMs began developing software in-house. They aimed to maintain functional consistency across vehicle generations without constantly re-specifying functions for each new model. In-house development allowed OEMs to implement updates more efficiently, reuse software, and avoid the learning curve associated with new suppliers.

OEMs developed software collaboratively with Tier 1 suppliers, ensuring continuity and quality across vehicle generations. This approach worked well for simpler control units. However, with the advent of highly integrated control units, this strategy exposed significant challenges.

Complexity of Highly Integrated Control Units

Highly integrated control units combined multiple applications, significantly increasing the complexity. Previously, interfaces were straightforward, typically based on CAN communication between sensors, actuators, and software components. Now, software-to-software interfaces are internal to the control unit, obscuring them from external analysis and debugging.

Prototyping and Testing Challenges

OEMs conduct extensive vehicle testing, including hot, cold, and various environmental trials, to validate vehicle performance. Due to the limited availability of prototype vehicles, OEMs have invested heavily in HiL testing systems. However, these systems are difficult to scale. While there is a push to virtualize these integrated control units and perform tests at the virtual level, complexity remains a significant hurdle.

Setting up stimuli for tests on these complex control units is arduous. A more effective strategy involves conducting early software-software integration tests, ensuring issues are identified and resolved at initial integration stages. This approach reduces the complexity and cost associated with late-stage testing.

Conclusion

Addressing the current software challenges in the automotive industry requires a shift toward continuous integration and early-stage testing. By adopting these practices and incorporating a comprehensive understanding of both software and hardware layers, OEMs can better manage the complexity of highly integrated control units, ensuring higher quality and reliability in their vehicles. Ultimately, nothing goes on the road that doesn’t fit together or perform well, but the path to achieving this goal needs better management and optimization to avoid last-minute adjustments and feature removals.

Personal Opinion (Robert Fey)

In my view, the majority of today's issues in the automotive industry related to software stem from the overarching challenge of software integration. What I observe is that the way software is currently integrated is a critical factor in whether these problems are effectively managed. These are not purely technical or software issues; rather, the current approach to integration itself is the primary challenge.

I firmly believe that by changing this approach - creating a seamless workflow and enabling developers to quickly identify and address issues - we can significantly mitigate these problems. Empowering developers to detect and resolve issues swiftly is key to overcoming these integration challenges.

?