A lesson on Software Reliability from the automobile industry

In the 1960s, the automobile industry experienced remarkable expansion. Americans had collectively driven fewer than a trillion miles during this era. Over the subsequent six decades, however, this figure skyrocketed nearly fivefold to a staggering 3.3 trillion miles driven.

While this in itself is remarkable and can spin off a discussion on the astounding rise in prosperity of the United States, there is an interesting dimension that can be brought into perspective. Car safety!

In the 1960s, the fatality rate stood at 5.1 deaths per 100 million miles driven. By the 2010s, despite a significant increase in the number of vehicles on the road, this rate had dropped dramatically to 1.1 deaths per 100 million miles.

Something transformative happened during this period that promoted Car safety (Reliability) like never before.

What factors can be credited with contributing to increased reliability?

In the 1950s, Toyota adopted quality control methods influenced by W. Edwards Deming , shaping modern standards in Japan post-World War II. This evolved into the Toyota Production System , emphasizing continuous improvement, and setting a global benchmark in automotive quality control.

The reliability of automobiles in the US correlates directly with periods when manufacturing design and engineering processes were significantly enhanced.

This improvement was driven by a focus on continuous enhancement and the adoption of new practices that strengthened design, allowed for early failure prediction, and incorporated advanced telemetry systems. As a result, even entry-level cars today boast better safety standards than high-end cars from the 1980s and 1990s.

Enterprise Software Reliability (or the lack of it) today

Today's enterprise software mirrors the challenges of the 1960s automobile industry.

Despite heavy investment in tools like application performance management and open-source test automation, major enterprises still face news of near-catastrophic software failures.

Conversely, tech giants like Google, Amazon, and Netflix deliver notably reliable software. For instance, Gmail's uninterrupted service since 2002 exemplifies this reliability.

Reliability, in the context of organizational transformation and delivering value to customers, can be distilled into ensuring consistently successful and performant requests within technology ecosystems. Every organization is in a state of continuous evolution/transformation, and this leads to a continuous change in their technology ecosystem.

Organizations can confidently claim their systems are reliable, despite the rate of evolution, if they consistently achieve two goals:

Highly Successful Requests: This refers to the ability of systems to consistently and successfully fulfill customer requests or operations without errors or failures. It ensures that customers can rely on the system to perform as expected and deliver the intended outcomes.

Highly Performant Requests: This involves optimizing system performance to ensure requests are handled efficiently and within acceptable response times. High performance contributes to a positive user experience and reinforces the dependability of the system.

What enables Google, Amazon, and Netflix to consistently deliver software of superior quality and reliability compared to enterprises?

The answer lies in embedding reliability throughout the entire software development life cycle (SDLC) .

Google, for instance, famously wrote a whole book on its Site Reliability Engineering approach to software quality. Netflix amongst other things, implements graceful degradation to ensure uninterrupted user experience despite component failures or degradation.

Recently, Joachim Herschmann from Gartner? wrote about "Digital Immune Systems" , and what characteristics a digitally immune system would have.

Notable suggestions in the article include:

Autonomous and AI-augmented Testing: Implementing automated testing systems powered by AI to enhance efficiency, accuracy, and coverage in identifying potential issues.

Chaos Engineering: Introducing chaos engineering practices to proactively test the resilience and robustness of systems by simulating unexpected disruptions and failures.

Improving Customer Experience: Striking a balance between the need for rapid development and deployment (velocity) and maintaining system stability to ensure a seamless and reliable customer experience.

The implications are clear; software reliability cannot be an afterthought, and the onus of operations alone, and like reliability in automobiles it must be an integral part of software engineering.

The need for a software reliability platform

Our extensive experience in the software industry over several decades suggests the following factors that hinder software reliability:

• Design Systems are plagued with un-validated architecture, operational flaws with unknown failure points and inherent weaknesses.
• Coding is usually developed with a focus on functionality and not always keeping Reliability and Customer Experience in focus. This is hampered due to lack of deep and early performance/observable insights during development.

• Testing is complex, siloed and heavily skills dependent making it slow and expensive without achieving the right outcome. It is seen as a cost center rather than a quality center.

• Releases : Customer Experience and Systems stability do not influence release cadence causing an imbalance between velocity and stability.

• Observability is not comprehensive and reactive in nature leading to unmeasured undercurrents impacting customer experience consistently.

• Incident Response is reactive in nature due to lack of proper knowledge management to address incidents swiftly either manually or through auto-remediation.

Our first-hand experience with these challenges and our expertise in addressing them have led us to develop the world's first AI-based Software Reliability platform. This platform aids enterprises worldwide in achieving software reliability.

Software Reliability, with Watermelon Software

Watermelon is the world's first enterprise software reliability platform.

We help all personas across the application lifecycle with our software modules that are purpose built to infuse reliability into applications in pre-production, and measure and predict reliability events in operations and we do this, with zero code.

Currently, the Watermelon Software offers the following capabilities, all in one platform:

History tells us that the more we shift the onus of reliability from operations to engineering, the better quality of software our customers are likely to experience.

Software reliability is a journey driven by mindset, not maturity.

You can start this journey with any of our modules or our complete platform. Wherever you wish to begin, Watermelon Software helps you continuously evolve towards greater reliability

The Team Behind Watermelon

Watermelon Software is founded by Rajeev Vasisht and Harpreet Singh , each with over 25 years of experience aiding enterprises in their reliability journeys.

Our engineering leads, with two decades of reliability practice each, have developed the Watermelon Software platform to address the challenges encountered during their extensive experience.

Connect with any of us or drop us a message at [email protected] to learn more.