Software Success: Ideas

Part 1: Ideas

Why do software projects fail and others succeed? I don't know the answer. I'm going to write about it over the course of a few months and hope, with your help, we can think through it and share what we find. I hope that you'll engage in this process. I'd love to hear about what's worked and what hasn't. We've all got our war stories, let's get them out there to take a look. I'm going to start by writing about programming culture, hierarchy and the importance of ideas.

Hierarchy is part of every software project - that big pyramid of complex and sometimes competing relationships, which from a developer’s perspective, can get us what we need or get in the way. There's nothing wrong with hierarchy in general or the use of authority to simplify problems. I can think of great reasons to use authority. Everyone should use the same calendar for scheduling, a coworker harassing someone in the office needs to be disciplined, we need someone to organize purchasing, financials (i.e, our paychecks) , set standards for office hours, creating onboarding and offboarding processes, and educate everyone about professional standards. All of these things need authority and the formality of hierarchy.

But let's flip over and look at the organization from a programmer’s perspective. A group of people who are generally not high up the hierarchy (don’t worry, we’re fine with that). We've been assigned a deliverable, we need to create something that works by a date and inside a budget. The product process, sometimes called Initiation or Discovery creates the overall vision of the product, its goals and strategic importance to the business. It isn’t possible to nail down every detail during that process. If a team can identify a product of value to their business, are reasonably sure it’s doable and can allocate time and resources to it. Well, they’ve done their job.

But in production, our job has just started. We focus on the goal of delivering the product. We have to nail down every detail – we can’t program ambiguity. We need everything broken down into little pieces so we can take it apart, inspect each one and see how it will fit together into software. Then capture those pieces, organize and sequence them and assign them to the development team. Equally important, we need to find the missing information –things that are unsaid and unwritten, the assumptions. The most dangerous word in any software project. They hide in the many decisions that are at the foundation of the project. Not just by the product development team, but by programmers, database designers, solutions architects and capacity planners. We’ve got to hunt assumptions down and slay them. What does this look like to the rest of the organization ? A bunch of programmers with a whole lot of questions. These questions aren't meant as a challenge to authority, we need to know the answers to do our job and our job isn't simply to do what we're told (sometimes, I wish it were). Our job is to turn an idea into a product and we need to fill in every gap between the idea and the software.

This process of questioning is so important because we are searching for ideas, answers, solutions and tools to improve the product and simplify our work. We want the fastest and most effective way from not done to done. We like everything to be simple: simple things are easy to understand and communicate. Simple things can get handed around the team without a lot of explanation. Hierarchy and the complexity of social exchange inside that context is not simple. It is convoluted, subject to interpretation, constantly changing and not easy to standardize. We don't know how to capture the requirements of office hierarchy or social dynamics. We've got enough to do at work and attending the boss's birthday party in a conference room is not one of them (but we will sneak in later to grab a piece of cake!)

So, let's stop here for a moment and address the stereotype of the programmer. We prefer dark rooms, headphones, no interruptions and pointless small talk. We are "introverts", argue a lot, are hard to work with and socially awkward. We'd rather you email us than talk to us, even if you're in next cube. No, we don't want to go to lunch, we need to eat at our desk. Just open the door to our dark rooms and slide in a few pizzas and case of Coke (cold please) and we're good. But we behave this way for a reason: small talk, socializing and birthday parties don’t create software. At times, it can take days to think through a process, link the pieces together, visualize how everything interacts and program it in way that's easy to understand.

Our goals require formal communication that can be copied from an email and pasted into a requirements system. We can't capture requirements from conversations in the hall, or even conversations at our desk. They need to be written down, organized and analyzed. We are solitary because only one person can program one thing at a time. We eat at our desk because we're freaking out that we're behind on the schedule. And, yes, we are always behind on schedule, sometimes even before the project starts. And dark rooms, well, it's depressing to see the sun go down and know we'll still be plugging away at the same problem well into the night. We opt out of hierarchy because it's a distraction, and hierarchy can take a toll on the process of creating. The process of creating is the key to success.

A hierarchy imposed on ideas is a danger to the project. There are justifications - experience, access to better information, trust and a history of past success - hopefully that's how someone moved up the ladder. But it is always the wrong approach. Ideas don't exist inside a hierarchy, they stand alone, they don't even belong to the person who had the idea. An idea comes from our collection of experience, education, knowledge and collaboration. An idea needs to be inspected, evaluated and given a good going over independent of its creator and their position in the company.

Good ideas are the building blocks of software success. Look around at your team, are there a lot of ideas, are there a lot of questions, does everyone get a chance to debate (not argue). A good idea is tough and can take the abuse. If it fails to live up to its promise, let it go, but gently. All ideas are good, some just don't fit inside the constraints of a project. After all, you don't know if an idea will fit until you've kicked its tires. Respect and celebrate the creative act and the person who speaks out with an idea. If no one is contributing ideas, you've got nothing to work with. If you're in charge, that means it's all on you. I've been there, it's a scary place.

So, if you see something wrong especially early on - speak up quickly. If you're ignored, speak louder. Remember, you live and breathe software development - you likely know more about it than most - including those sponsoring the project. Your ideas are the most important asset in a software project - don't be shy about them.