How to Software, Better - Part 1: First Principles

Previous Article: Introduction

For those unfamiliar with the phrase, first principles are the basic underpinnings of a study - rules, premises, and concepts that are so elemental that there is nothing more basic. For the business of software engineering, I use three distinct concepts as my first principles: Lean, Design Thinking, and Servant Leadership. Of course, there are things more basic than these when talking about software engineering, product management, and human management, so it's a bit of useful fiction to call them first principles, but only infrequently do you need to get all reductio absurdum and dive deeper.

Lean

Lean has its origins in manufacturing and was popularized for use in the software industry with the book Lean Software Development by Mary and Tom Poppendieck (2003). Essentially, Lean dictates that organizations efficiently and relentlessly remove waste from their processes in a form of continuous improvement. Classically, it describes three types of waste using the Japanese words: muda, mura, and muri.

• Muda is effort expended without producing value for the customer.
• Mura is unevenness in effort (lumpy workloads).
• Muri is overworking people, machines, and tools.

Initially, it's helpful to use these classifications to narrow the scope of analysis, but once you engrain the Lean philosophy into your way of thinking, it will be fairly easy to identify waste in any of its forms.

Lean is inherently complementary to Agile. As your read through the Agile Manifesto and its related Agile Principles, you should be able to identify how each objective is supported by Lean's focus on the elimination of waste. You should also be able to easily see Lean as the backbone of DevOps. If not, read the book The Phoenix Project that Gene Kim (et al.) (this is an adaptation of Eliyahu Goldratt's The Goal, the fictional story of a manufacturing operation that incrementally improves by applying Lean principles).

Lean is also complementary to Design Thinking and Servant Leadership. In fact, Lean is so fundamental that I can usually tie every conceivable process optimization back to Lean and consider it the first principle of these first principles.

Design Thinking

Design Thinking was born in the 1960s as designers tried to transform an intuitive approach to designing new products, with highly variable results, into a scientific approach. IDEO formed in the 1970s to teach and apply design thinking and took the idea mainstream in the early 1990s. Stanford's d.school builds on design thinking and teaches a comprehensive approach to product design.

Though it's been around for about 60 years, design thinking is still not widely used in software product development. However, design thinking is incredibly powerful for shortening the time from concept to product, being able to accomplish more with fewer resources, and evening out production efforts as you create and deploy value. These advantages should sound familiar because they very closely echo Lean.

Through the late 1990s and early 2000s, as I progressed from a senior engineer to architect to manager, I intuitively fell into using design thinking to guide my approach to building desirable products (though I didn't realize it had a name). At the time, I told my teams to "build it right the second time" as a sort of abbreviated Rapid Prototyping. I had noticed other software groups spending a lot of time designing every last detail into a specification before writing any code. This meant that their product was going to be late to market even before it started. Invariably, they would end up scrapping some or all of the design and specification (and the code they wrote to them) somewhere around the 35-50% point, making it even later. Instead, I directed my teams to spend a little time on the upfront architecture, just enough to cover the gist of the product requirements, and then immediately start writing a prototype so they could start uncovering the pitfalls and assumptions with their first approach. We did this with the full intent to throw it all away and start fresh with the "perfect" solution, though often many modules survived.

This "do it right the second time" approach made more sense when you had to ship a monolithic product that may never be updated. Today, continuous deployment strategies make this approach obsolete. With microservice architectures, container orchestration, and other modern constructs, it's possible to use design thinking throughout a product, with each enhancement, and with even less wasted effort. Its greatest value is when it's used to tighten the loop between consumer, product manager, and developer to rapidly iterate solutions to satisfy the need and, perhaps even more importantly, stop when you're done (I'll talk about how to avoid overbuilding in a later article).

Servant Leadership

In the Servant/Inverted/Intent-based Leadership model, the expectation is that the Individual Contributors (ICs) know how to do their job best and rely on managers and executives to provide the tools and resources to execute the mission. In turn, Executives are expected to set objectives, describing the "What" and "Why" of the mission. Managers have a slice of the overall mission to execute. They work laterally to coordinate across functions and prevent duplication of effort. They also work vertically the coordinate the efforts of their ICs with the mission objectives of the Executives. This approach lets everyone focus on what they're best at and reduces waste (Lean).

Conclusion

These three concepts form the foundation of my approach to engineering leadership. I'm always searching for ways to remove waste from the process of turning a market need into a technology solution, whether its refining requirements, making sure every person can do their jobs efficiently, or optimizing the overall system of policies, procedures, tools, and people. Lean forms a terrific basis for staying focused on what matters most.

Design Thinking can be used as a guiding process for both product management and engineering management. A great way of avoiding waste is by continuously engaging and confirming with the consumers of your product. This "product" might be the one your company is delivering to the market, but it's also every work product of everyone in the company. Someone is consuming everything that everyone is doing, so it follows that there should be a high level of engagement between them.

Finally, Servant Leadership gives a high level of agency to Individual Contributors and places an emphasis on empowerment and ownership. The Individual Contributors feel personally responsible for what they're working on and can expect a large amount of freedom to figure out the best way to accomplish a task and the resources to execute it. In any organization, these ICs represent the large number of people, so anything you can do to improve their productivity will have a massive multiplier to overall velocity.

Next, in Part 2: The People, we'll talk about the external and internal people responsible for the success of your product and company. I'll emphasize how to lead and orchestrate the development organization to minimize waste.

Recommended Reading

The Goal by Eliyahu Goldratt (Amazon) [The original storybook for Theory of Constraints.]

The Lean Startup by Eric Ries (Amazon) [Measure, Improve, Profit (or Pivot).]

The Phoenix Project by Gene Kim et al. (Amazon) [An adaptation of The Goal for software and IT people.]

The Service Startup by Tenny Pinheiro (Amazon) [My introduction to Design Thinking.]

Turn the Ship Around by L. David Marquet (Amazon) [An excellent example of inverted leadership easily adapted to software engineering organizations.]

Wiring the Winning Organization by Gene Kim and Steve J. Spear (Amazon) [Probably the single best book I've ever read on technology management.]

Justin Wolf wrote his first line of code when he was about five years old. Since then, he's written hundreds of thousands of lines of code for consumer products in assembly language (Intel and Motorola), C, C++, C#, and other languages. In the mid 1990s, he began learning about computer networking while at Cisco Systems and spent a decade working on enterprise and residential networking products. He worked for about eight years at national laboratory on cybersecurity projects and then at a lab spin-out leading product development of a visual analytics platform fusing machine learning with human-in-the-loop analysis. Most recently, he led a large engineering team building massively distributed systems for battery energy storage systems (nearly 100,000 networked devices for the largest deployments). He lives in Southwest Washington state with his family (and pets) and enjoys sailing in the Pacific Northwest inland sea as often as possible.