Don’t Trust the Team

At a major government agency, I once asked the chief of their Agile methodology if one should “trust the team”. I was referring to software development teams. His answer was an unequivocal and unilateral yes: One should always trust a team to make the right decisions, and if they don’t, then they will learn from their mistakes, and that is the best way to learn.

I had heard this attitude many times before, and it has never sat well with me. While I have been a strong advocate of Agile methods since 2000, I have felt that Agile’s principles were guidelines - not absolute rules. The Agile Manifesto says,

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

But this does not say, “Trust the team in all things, and allow them to fail”.

Somehow, people have made a huge leap from what - I believe - the Manifesto intended. I believe that what was intended is that a team lead should not micromanage them - that you should get “out of their hair”. But that does not mean that you should just sit back and cross your fingers.

Teams often have very bad judgment. This can arise from inexperience. For example, one thing I have noticed is that programmers choose tools based on what makes their job easier - not what is best for the organization. Junior programmers generally prefer type-unsafe languages such as Python, Javascript, and Ruby. Their reasoning is that it generates fewer errors when they compile their code. That is true, but errors still lurk undetected. But to a youngish person in a hurry to get their story done, a typesafe compiler can try one’s patience, since it requires your code to be much more solid before it will complete a compilation.

The creator of the Python language, Guido van Rossum, was hired by Dropbox to help maintain things, since Dropbox had written their platform in Python. They found that “when the company grew, new engineers could not understand the clever but 'short and cryptic' code written by and for earlier developers.” This was because of the lax rules of the Python language. For example, to learn what kind of object a Python method returns, you have to dig into that method’s code - Python methods do not explicitly declare a return type.

Today Dropbox uses Go, Typescript, and Rust - all strongly typesafe languages. They learned a lesson - the hard way - that most experienced programmers know.

My point is that programmers don’t make the best choices about technologies. They pick what they know, and they pick what makes things easy, and they jump on fads: everyone seems to be using JSON? - then I’ll use that too. Young programmers also have not typically learned some hard lessons that experienced programmers have. There are many such lessons, since few of the practical aspects of programming are taught in a computer science curriculum.

I often summarize the situation as,

A team of smart programmers just out of school can rapidly create a huge mountain of unmaintainable code.

Programmers make bad design decisions too. A particularly horrendous example is described in an article about the incident in which an Uber self-driving car killed a bicyclist. According to the article,

• “...the system design did not include consideration for jaywalking pedestrians”
• “...the constantly switching classifications prevented Uber's software from accurately computing her trajectory and realizing she was on a collision course with the vehicle”
• “The system used an object's previously observed locations to help compute its speed and predict its future path. However, ‘if the perception system changes the classification of a detected object, the tracking history of that object is no longer considered when generating new trajectories’ ”

As an experienced programmer, these things all sound like horrendously bad design. It tells me that no one did a system-level walkthrough of “failure modes”. It reminds me of systems that I have seen that were “coded” without being designed.

It tells me that whoever was in charge just let the programmers code away, with no oversight.

In the Uber case someone died. If someone is working on banking software, the worst that can happen is that a million customers’ deposits are erroneously tabulated, or that their data is exposed to hackers - not so bad!

That’s why I don’t trust the team. I don’t even trust myself. That’s why when I lead teams, I talk through designs with them. I make sure that a suite of high coverage test cases is created for critical business transactions, and that those test cases are not written by the same people who wrote the application code - because if it is, then a misunderstanding of the requirements will be present in both the code and the tests. I make sure that design decisions are recorded in a wiki to journal the evolution of the design. I look at people’s code, and I look at the test cases, and I make sure that every kind of functional and non-function need has tests and that the test coverage (not just code coverage) is assessed in some way. And that is just for starters.

Pairing won’t fix this. When people pair, they are focused on solving a problem. They are not focused on quality. They want to get the story done. To maintain quality, one has to have explicit quality practices, such as design level review, inspection, and - most importantly - discussion.

I use discussion as a weapon against defects: I have people walk me through their approach, and I challenge it: How do you make sure that data is consistent? What if an eventually consistent change uncovers an inconsistency? Where are the places in the app where that might happen? What happens if an event arrives out of the expected sequence? What if that message is missing a field? What if these two transactions against overlapping data happen at the same time? What if that service is unavailable for five seconds - will it trigger a sequence of failures that could have been recovered from?

To be able to trust the team, either their reputation precedes them - that is, they are known a-priori to be highly experienced and competent - or, you have trained them yourself so that you know what their capabilities are: then you know when you can trust them, and when they need some oversight. If you have a team with junior members, you should not simply trust them outright.

Have faith in the team - yes. Have confidence in their ability to develop and grow - yes. Advocate for them - yes. Be their biggest fan - yes. But trust a team containing recently out of school members to build whatever comes along, without any mentoring, assistance, or oversight? Hardly.

A team lead who sits back and “lets the team fail” is not a team lead. A team lead’s role is to help the team to succeed - not by doing things for them, and not by micromanaging them, but by mentoring them. And if the team needs expertise that they don’t have and their team lead does not have either, then the lead should bring someone in who knows that area to mentor the team about that. Discuss, collaborate, check, question, suggest. Always in a nice way of course, and always with encouragement - but don’t just sit back!