Why Corporate Bloat is Killing Your Software Efficiency

TL;DR Summary:

• Boost Quality by Increasing Speed: To improve quality, focus on enhancing speed and efficiency. Higher software delivery speeds correlate with better quality outcomes, challenging the belief that adding processes automatically enhances quality.
• Challenge Every Assumption: No practice should be sacred. Regularly challenge assumptions, especially those surrounding established practices, to ensure they are still valid and beneficial.
• Action Over Documentation: Prioritize actionable changes over extensive documentation. Implement small, manageable changes in low-risk areas and measure the results promptly to prevent stagnation and maintain a focus on tangible improvements.
• The Litmus Test of Innovation: Regularly pushing past the comfort zone to the point of needing to reintroduce some elements is a sign of healthy experimentation. If you’re not retracting steps now and then, you are not venturing far enough.

The Fascinating Productivity Gap

I'm fascinated by a prevalent issue in the software industry: the stark productivity gap—often by a factor of 10x or more—between lean startups and established corporations, even when they have the same tools and follow the same regulations. Take Instagram, for instance. The original website was created in just a few weeks by two developers. In contrast, in the corporate world, even trivial UI updates, with far more staff involved, can take months. This isn't just intriguing; it's a massive opportunity. The cause? Mainly, its the slow, insidious growth of “bloat,” perfectly illustrated by Parkinson’s Laws1: the number of workers grows regardless of the actual work to be done, and work expands to fill the available time. Over time, excessive processes, redundant checks, and unnecessary steps that add no value accumulate and bring progress to a halt. The typical corporate response to quality issues is to add more processes, checks, and staff, which slows things down even further. This approach is fundamentally flawed. It often exacerbates the problem, protects self-interests, and fails to address the underlying inefficiencies. So, how do we address this?

SpaceX's Revolutionary Approach

Let's look at SpaceX, a relative newbie in the well-established and heavily regulated aerospace industry. Despite using technology that’s not exactly breaking new ground,2 they've managed to slash costs dramatically—by about 10x, and they're aiming for another 10x reduction with their upcoming Starship3. What's their secret? It boils down to a strong focus on simplification and sticking to first principles, something Elon Musk is big on. Musk pushes us to question everything we think we know and start fresh from the basics. It’s about having the courage to ditch what’s familiar and the wisdom to know when it’s necessary. He often says that if you’re not putting some things back after cutting them out, you’re not cutting enough. This mindset drives SpaceX to keep pushing the boundaries, constantly innovating and challenging the status quo.

“If you’re not adding things back in at least 10% of the time, you’re clearly not deleting enough.” - Elon Musk

Core Problems

Growth of Inefficiencies

In many companies, especially those with long histories, there is a tendency to cling to legacy processes simply because "that's how it's always been done." This adherence to outdated practices is not just about comfort; it’s a reflection of a deeper resistance to change. These inefficiencies grow and become embedded in the company's operational DNA, leading to bloat that nobody questions, often because challenging the status quo can be seen as a risk to stability or even career progression.

The Fallacy of Oversight

A common misconception in the corporate world is that having more oversight—more eyes on a project or process—automatically leads to better outcomes. This belief leads to the multiplication of check points and approvals that do not necessarily add value. Instead, they often extend timelines and dilute accountability, as no single individual feels wholly responsible for any mishaps, dispersing responsibility across an ever-widening array of stakeholders.

Misguided Solutions and Siloed Interests

The common response to emerging problems is often to pile on more processes, an approach that typically backfires by increasing inefficiencies. This is further complicated by siloed teams driven by perverse incentives, which fail to align with broader organizational goals. Teams like security and quality assurance often end up focusing on narrow issues that do not necessarily translate to overall project success. This reflects the historical pitfalls of perverse incentives, such as during Vietnam's rat bounty program?, where the goal to reduce rat populations actually encouraged their propagation.

Cultural Resistance to Change

Corporate culture can significantly hinder innovation when it rigidly clings to established practices without assessing their effectiveness. This resistance often stems from the belief that adding more processes equates to better quality, despite evidence to the contrary. In contrast, the DevOps Research and Assessment (DORA) metrics present a compelling argument: there is a strong correlation between higher software delivery speed and improved quality?. This indicates that enhancing quality should focus on boosting speed and efficiency. Practices such as Continuous Integration and automation support this by promoting frequent updates, which compel teams to tackle the core issues directly, rather than layering on additional, unnecessary manual processes.

Effective Solutions for Reducing Bloat and Enhancing Efficiency

Emphasizing Value, Speed, and First Principles

To combat the entrenched inefficiencies plaguing many organizations, a shift in focus is crucial. Concentrate on delivering value, maintaining speed, and applying first principles thinking. This means consistently optimizing for the whole rather than getting lost in the minutiae of traditional methods. It's vital to maintain a perspective where no practice is sacred and unquestionable.? Challenge every assumption, particularly those surrounding established practices. For example, if an 80% test coverage is mandated, it's essential to question its necessity and impact. Could lowering it while maintaining quality free up resources and accelerate development?

Avoiding Analysis Paralysis

Action should always take precedence over extensive documentation. Implement small, manageable changes in low-risk areas and measure the results promptly. This approach prevents the stagnation that comes with over-analysis and ensures that the focus remains on practical improvements. If you're not occasionally reversing some of these changes, you're likely not experimenting boldly enough. Contrast this with the organizational observation of 'bike shedding,'? where in the software space it's easier for architects to produce lengthy internal documents on already well-established practices rather than directly engaging with complex legacy code to genuinely identify and address underlying issues (i.e. actually doing the work).

Practical Implementation Examples

Streamlining Code Integration Processes

Suppose long integration times are identified as a bottleneck due to prolonged feature branching and mandated pull requests. Start by collecting actual data, such as PR merge times, and gather anonymous feedback from developers, not just upper management.

Trial solutions like making pull requests optional for minor changes or reducing the requirement to a single approval instead of two. Implement these on a small scale over one sprint. After the trial period, evaluate the impact and decide whether to expand, adjust, or revert the changes based on real-world data and developer feedback.

Adopting New Tools like MS Co-Pilot or GitHub Copilot

Despite offering massive productivity gains—such as a 37% increase according to an MIT study?—companies continue to strongly resist adopting secure LLMs like GitHub Co-Pilot and MS Co-Pilot. Competitors under the same regulations are already using these tools, backed by robust security from Microsoft. Instead, organizations spend months on security reviews, ultimately rejecting these tools under vague safety concerns and red tape, while developers find workarounds with less secure models. The productivity gains from these tools cane be argued to be equivalent to those from the internet, yet overly cautious safety concerns hinder adoption. This situation parallels the absurdity of the Red Flag Traffic Laws?, which limited cars to slow speeds and required a person to walk ahead with a red flag—all in the name of safety, with zero regard for actual productivity or common sense. Instead, companies should immediately trialing these secure tools to get genuine feedback from developers, bypassing potentially biased and lengthy reviews from 'ivory tower' architects or overly cautious security teams.

Footnotes

1. Parkinson's Laws: Formulated by Cyril Northcote Parkinson in 1955, these principles highlight organizational inefficiencies. The first law states that "work expands to fill the time available for its completion." The second law observes that "the number of workers in bureaucracy grows regardless of the amount of work to be done." Both laws demonstrate how inefficiencies and bloat can proliferate, reducing productivity and effectiveness.
2. SpaceX Cost Reduction: SpaceX has dramatically lowered the cost per pound to LEO from approximately $18,000 with the Space Shuttle, to $2,720 with Falcon 9, and anticipates a reduction to around $200 with the upcoming Starship rocket.
3. Reusability and Vertical Landing: The Space Shuttle was an early but costly attempt at reusability. The DC-X, previously demonstrated vertical landing capabilities.
4. Vietnam Rat Bounty: During the U.S. occupation of Vietnam, a program was instituted to reduce the rat population by offering a bounty for each rat tail brought in. This incentive led to unintended consequences as people began to breed rats to harvest their tails, ultimately exacerbating the rat problem rather than mitigating it.
5. DORA Metrics Explanation: The DevOps Research and Assessment (DORA) metrics, derived from extensive industry data, reveal that higher software delivery speeds are associated with improved quality, challenging the traditional belief that more processes ensure better outcomes.
6. Sacred Cow: In business, a 'sacred cow' refers to an idea, custom, or institution held, perhaps irrationally, above criticism or alteration, akin to the literal reverence for cows in some cultures.
7. Bike Shedding: Derived from Parkinson’s Law of Triviality, this term describes the phenomenon where people give disproportionate weight to trivial issues, often while important matters are neglected.
8. Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence: A study by MIT found that tools like GitHub Co-Pilot and MS Co-Pilot can increase productivity by 37%+. (Source: MIT Economics)
9. Red Flag Traffic Laws: Historical laws in the UK from 1865 to 1896 that limited cars to speeds of 2 mph in urban areas and 4 mph in rural areas, and required a person to walk ahead with a red flag for safety. These laws exemplify excessive caution at the expense of productivity and basic common sense.