Fixing Agile, for the People - A Communal Approach for a New Age

Somewhere along the way, we forgot that people are the core of being agile, meaning that our “doing Agile” mindset is not centred on those delivering the value and we’re thinking too much about the output. Maybe a little holistic thinking will cure us of that? And no, we don't have to stop bathing.

Introduction

Before we start this article, let’s remind ourselves of the first of the four parts of the Agile Manifesto: “Individuals and interactions over processes and tools”. People confuse the “over” part, thinking it means “eschew processes and tools”, which couldn’t be further from the truth. One main problem arises when you start scaling up agile teams as Conway’s law is an inevitable outcome. I've mentioned it before, but let's refresh our memory:

"organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations." – Melvin Conway

This article is just my train of thought, an internal experiment I’m running in my head. I might try to make some of it materialise beyond these pages if I feel like I can round off the corners. So, take what you read here with a pinch of salt. I’ve done my research but a lot of this is just opinion, so I’m afraid “no monies returned”.

There are three major and repeated problems I can see that are hurting projects, and they’re all to do with the people (shocked face):

• Communication – There’s too much of it and it’s often not helpful
• Comfort – The environment is not one that makes people feel comfortable working in
• Competency – We, as an industry, have somehow erected new siloes and this is really starting to bear bitter fruit

But I think it’ll be ok… Now is the time, as we’re in what must be the third “Agile Industrial Revolution”, to do something about it and help us deliver more effectively with more compassion to our real bread-winners, the technologists.

Let me put this out there, if it wasn’t for the people who are often the lowest on the food chain in many organisations, the engineers, we would be nowhere. The industry has somehow taken some of the dignity of this role away, now it’s time to take it back (this rant continues in another article).

In addition to the above, it’s worth noting that most, if not all, engineers want to do a good job; where that means delivering high-quality on time, using appropriate technologies. Notice how I use the word engineer, part of this train of thought is that treating data scientists, developers, testers, platform engineers and architects as separate beasts is, in my opinion, deeply unhelpful and is damaging our industry.

Communication Breakdown

The more onerous the communication plan, the more time is expended on it and the less value is imparted. Not all of that time is a waste though as it’s necessary to communicate.

There are three main types of communication that should be considered:

• Intra-team communication - which should be rapid and light with rich content
• Inter-team communication - which should be lighter for informational purposes and to enable cross-service/system interoperability
• Wider programme/organisational communication - lightest with least content

I've found that it's best not to attempt to conflate these or use the same communication methods for each. The shamelessly stolen diagram below broadly maps onto these types of communication.

Of course, as a blankey statement, I'd avoid lumbering MS Word documents that nobody will ever read. I hope that comes as no surprise to you.

Some companies create a double down on this communication problem by also doubling their Conway’s law by segregating by skill as well as by system. (see figure below), I can guarantee you that they're not doubling their pleasure.

This approach is sadly quite prevalent in more traditional organisations and is a massive blocker to true Digital Transformation. But we're not here to talk about that today, suffice to say if you see this structure, you should start asking questions (like "where's the door?").

You may see tensions in your own organizations where your structure and software are not in alignment. You may have seen for example the challenges involved where distributed teams try and work on the same monolithic codebase. Here, the communication pathways that Conway refers to are in contrast to the code itself, where single codebase requires fine-grained communication, but a distributed team is only capable of coarse-grained communication. Where such tensions emerge, looking for opportunities to split monolithic systems up around organizational boundaries will often yield significant advantages.

https://www.thoughtworks.com/insights/blog/demystifying-conways-law

A lot of the problems faced while working in distributed teams, stem from the fact that the entire structure of the programme of work has an impedance mismatch, which causes tensions, inaccuracy and inefficiency. This doesn’t mean one shouldn’t have an architecture, principles or design (which is a common misconception around GDS implementation), but it’s just that you should be able to structure your teams and systems to naturally follow this law by federating through communities rather than enormous teams.

All energy flows according to the whims of the great Magnet. What a fool I was to defy him. – Hunter S Thompson

The diagram below, which I’ve shamelessly stolen, shows how more people leads to an ever-growing communication problem.

You don't have to be a maths genius to spot there's a point at which increasing your team size by one creates a huge impact. This expansion has a threefold impact on a programme of work:

• It costs more as it takes longer to disseminate the information, and/or you end up doing it multiple times
• The information ends up becoming inaccurate or “watered down” in an attempt to mitigate the first point
• The turnaround time from communication to actions to consequences is increased

Luckily, as we move towards a more distributed (and often MicroServices) model, slicing your enterprise-scale system up into smaller pieces is now fashionable. This fits nicely with what we’re discussing today.

Ideally, you'd build communication through your components that matches your team, i.e. intentional Conway’s law and avoid getting in the guts of the code of another team, as it can be a real productivity drain.

Home Comforts

“History will be kind to me, for I will write it” – Winston Churchill

Modern offices are designed for extroverts, for the neurotypical, for the boss. They look cool, they’re open, they’re bustling, they’re loud, they’re many things that make neurodivergent people (of which technologists seem to be a statistical clustering of; see https://www.telegraph.co.uk/news/science/science-news/11973110/Scientists-and-mathematicians-test-higher-on-autism-spectrum-says-Cambridge-University.html) want to scoop out their eyes and eardrums with a rusty spoon.

Consider how noisy open plan environments can be distracting or lead to individuals feeling overwhelmed - https://workplaceinsight.net/neurodiversity-not-agenda-9-10-uk-organisations/

But the problem is actually much wider than that, I had a little insight on this, an epiphany. I believe that neurotypical people hate these offices too, they just have the ability to deal with it (or hide their feelings about it) slightly better. I've had many conversations to try to figure out if this is the case, and I think I might be onto something.

Survey respondents flagged noise (15 percent) as the primary cause of inefficiency during the working week, a number that has risen by four per cent in just 12 months

https://workplaceinsight.net/open-plan-offices-are-distracting-and-reduce-rather-than-improve-productivity-says-report/

Also, sprint and ticket-based working practices favour the lone worker or the micromanager, so they don’t necessarily foster close, thoughtful working in an environment that feels comfortable.

People often confuse modern and "fun" (think table football) for comfortable, and don't even get me started on the "forces hot-desking for all" policy some organisations have put in place to save a buck-o-five on desk space. Unless of course you'd like to have a discussion on the systemic dehumanisation of your workforce and their eventual migration to other companies. Also, the lack of actual, safe office furniture in some places puzzles me... Sure, it looks cool to be working on a bar stool in an office that's brighter than the surface of the sun, but it's rubbish for your posture and forget sensible power outlets or second monitors!

But it's worse than it just being distracting, this modern office might actually be making you sick!

A study at Queensland University of Technology's Institute of Health and Biomedical Innovation found that working in environments without offices "caus[es] high levels of stress, conflict, high blood pressure, and a high staff turnover."

Not surprisingly, employees in open-plan offices take more sick days. According to The New Yorker, companies with open-plan offices can expect employees to take a whopping 62 percent more sick leave.

https://www.inc.com/geoffrey-james/why-your-company-will-benefit-from-getting-rid-of-open-office-spaces-first-90.html

Maybe we should do something about all this? We're not going to go back to Dilbertesque grey cubicles, but there has to be a halfway house along the way. Well, I know there is, for I have seen it in action.

Competency, it’s Quickly Becoming a Four-Letter Word

Historically organisations built teams of similarly skilled individuals and put them in "departments" which represented a capability, which is absolutely not the right thing for an Agile team as the cross-department communication immediately kills productivity.

When I first joined the industry, the software developer was the BA (not necessarily the SME, which was usually the client), the developer, the networking engineer (the terms DevOps and Platform Engineer were a way off back in those days) and the tester, probably with some other stuff thrown in. This meant that when we first moved to Agile, we were pretty cross-functional by default.

One of the key concerns I’ve had of late is that modern Agile teams/organisations seem to encourage role siloes, which causes internal congestion when a particular skill is in demand and makes setting out a Kanban board fairly redundant (Kanban is more than column limits, it’s also about resource fungibility). This is almost certainly due to the increasing complexity of the industry, we have more technology than ever and it's impossible for one person to know it all (this was true even 20 years a go, but more evident now).

Let me be clear about my feelings on this matter; if you’re writing code, you should have standards and best-practice around the use of that language and its tooling. All work should be reviewed to the same standards, all code is production code. Allowing one capability to get away with quality compromises means that you’ve just proven “a chain is only as strong as its weakest link”, only you’re doing it with an enterprise-scale system and reputation, money and/or lives are at stake. Which I’d say is not what anybody really wants.

Also, some people value the idea of siloes, they can bring an artificial wall up, claim you don't understand and protect their precious knowledge/feifdom. The flip-side to this is that people take that as an excuse to shrug their shoulders and say "sorry, not my job"!

There are several other issues that crop up due to capability siloes:

• Cost of handoff between team members
• Single point of failure
• Lossy nature of knowledge transfer
• Lack of motivation to step outside of silo

I'm not against capabilities, quite the opposite in fact, however they should be considered as an activity that's outside of regular project delivery.

What Qualities are we Trying to Engender?

In setting up an Agile commune, you should be trying to encourage healthy and sustainable behaviours, including:

• Providing a deep sense of purpose, having a goal to work towards, knowing how important it is to reach the desired end-state
• Empowerment to speak up, with a feeling of buy-in to decisions
• Shared leadership; making it a team sport, not a dictatorship
• Familiarity and trust, which leads to people acting in a more egoless way
• Honesty, allowing people to speak freely and not feel afraid
• Empathy for your commune members, keeping them healthy and engaged
• A “no heroes” mentality, you succeed together, you all share the successes
• Management of your own turf including technical debt, quality, repositories and artefacts

The bottom line, is that if people are bought in and care about your programme of work, they'll make it their business to do a great job and go the extra mile with no extra coercion necessary.

Enabling Agile Communes

Hierarchies are a part of human nature, but so are communities, however in Agile we already have Communities of Practice, so that word is “reserved”, so are cells, teams, tribes etc. so, for the purposes of this article, I’m going to use the word “commune”. Because it has a meaning and also some 60’s hippy connotations that please me.

Structuring an Agile commune has some basic rules, which look very much like what we should have been doing anyway if we'd paid attention to the Agile Principles (https://www.agilealliance.org/agile101/12-principles-behind-the-agile-manifesto/):

• Keep your communities down to eight or fewer
• Give people an opportunity to self-organise
• Give them the tools to enable self-sufficiency - Genuine cross-functional teams should have all the competencies required to complete their goals without requiring external skills
• Let them run their own ceremonies, as long as they’re happening

But that doesn’t exempt them from the wider programme:

• Programme architecture and platform should be mature enough to be adopted without too much variance or impact on the team
• Transparency should be maintained. Everybody should use a level playing field when it comes to velocity, quality etc.
• Keep deliverables as concrete as possible, humans deal badly with abstraction and bigger-picture thinking

However, there are some “Agile Team Smells” to Look Out For:

• Unattractive or uncomfortable working spaces
• Lack of interaction
• Lack of transparency (information radiators, dashboards etc.)
• Focus on form of the office (looking cool) over function (usable by the team)
• Stale or incorrect communication on the walls
• Seating to suit facilities, not the team
• People creating communication barriers like constant headphone usage

The Qualities of a Happy and Productive Commune

I've tried to categorise the different qualities that being in a proper commune environment would bring. A kind of charter to stick to, as it's too easy to stray from the path. I'll add to this section when I feel inspired!

Unification

• Agree your availability internally i.e. core working hours, days working from home, from the office, community lunches
• The commune should speak as a commune, not as individuals. No one voice is louder than the other and messages outside the commune should be agreed and consistent
• Internal disagreements are adjudicated without external interference, unless it’s absolutely required (i.e. stuff that is intractable or requires HR)
• You should work collaboratively. Don’t do anything on your own, apart from maybe visiting the bathroom. Pair for analysis, design, implementation
• You should review collaboratively. Don’t do cold, online “code reviews”. Review your work with your team with the analysis, testing, development and preferably with a client user there too
• Avoid casting roles as individual people where possible. I’d sooner take a great technical who can wear multiple hats over a separate Scrum Master and Technical Architect

Equality

Communes are all about equality and if you remove that from them, they turn back into a heirarchy. There's some temptations along the way, don't say I didn't warn you!

• Everybody should be at the same “level”, there is no “boss”. Just roles that perform functions in the team that are shared anyway
• Nobody gets a shout out for finishing the most points
• The entire commune gets the plaudits for delivering something good/creative/helpful
• Show off your output as a unified team, no "I", just "we"/"us"
• Have a communication style that means everybody gets a voice, pass a "speaking" ball around if it's too hard to not talk over each other
• If somebody is having a hard time, the workload can be rebalanced with no further repercussions
• Feedback to colleagues should be honest and continuous, making an annual review almost a formality, there should be no surprises there!

Comfortable Environment

Address comfort at work on a regular basis through workspace preference questionnaires and broader employee satisfaction surveys - https://workplaceinsight.net/neurodiversity-not-agenda-9-10-uk-organisations/

Imagine if you could control your own office environment, you wouldn't need "workspace preference questionnaires". Give that power to the people, let them manage things locally to their commune.

• Provide an environment that the community can control, decorate, light, heat/cool. This would probably require moveable partition walls, but this is a good thing, it allows a commune to shape their environment
• You must all sit together, but who gets what desk, that’s up to your own internal system
• Manage your own team visual aids. Agree on rules for information radiators, what they’ll show, where the whiteboards will be etc.
• Agree if you want a radio on or not, shared playlist, how loud, of if the environment should just be quiet
• Ensure there are some wellbeing initiatives such as mindfulness, yoga etc. in the commune
• Keep a mood map of the team, what needs to change? Have a retrospective on your environment
• Ensure that proper workstation assessments are happening and that people have the equipment they require to be comfortable
• Provide a budget for decoration and simple furniture, want a bean bag? No problem, this is your key workers' happiness we're talking about here

Focus, Peace and Quiet

Focus, one of the keys to productivity. According to a study mentioned in Peopleware, it takes the average person 15 minutes to "get in the zone" but second to break that, then you've got to start all over again. A workplace with random distractions is one that has a focus problem.

“Our research shows that the vast majority of our time at work is based on the need to ‘focus’ — more than 60 percent of the working day.”

https://workplaceinsight.net/open-plan-offices-are-distracting-and-reduce-rather-than-improve-productivity-says-report/

A friend of mine once instigated “quiet time” on an embattled project I was on. Two hours a day of no phones, no internet, no email, no client communication, no talking unless it was to discuss a problem. I scoffed at it, but I was dead wrong. A few weeks in and I realised that I was more productive than I ever had been.

• The internet and electronics can often stifle concentration, creativity and collaboration
• Make your community monastic for a few hours a day, have “quiet time” where no unnecessary talking, internetting etc. goes on
• Switch your damn phone off, preferably for most of the day
• Keep your random distractions for break times
• Provide areas where people can go to be silent and still, maybe even shut their eyes for five minutes

This doesn’t mean you never talk, listen to music or have fun, but coordinating your quiet time means that focus is shared without constant distractions. Try it, trust me!

Family Atmosphere

I’m not expecting people to work together and then find that they’re magically all best friends. However, many modern offices would benefit from better team cohesion. I have been really guilty of being a lone-gun but have seen some great leaders who create the kind of atmosphere that means the cogs of interaction are kept greased without it feeling forced.

• Have a morning coffee, set the day off on the right foot
• Get into a tea/coffee cycle, try to make activities like this as inclusive a possible
• Lunch together on a regular basis, people who dine together form stronger bonds
• Organise evenings out (not just drinking!)
• Keep the atmosphere fun! Have your in jokes or favourite music etc.
• Washup at the end of the day

And finally, but importantly, try to encourage people to work sensibly and leave on time. However, if it’s crunch time, you all stay (where possible) and pitch in

Expressive Culture

By industrialising what was once a very creative career, we seem to have lost some of our spirit. How about putting some of that back? Most engineers like to feel individual, we are not interchangeable units!

• Make the walls of your community centre drawing spaces (not just for diagrams)
• Allow for creativity in design and delivery, within the framework of the programme
• Have some non-engineering creativity in your commune, making music, writing blog posts etc.
• Celebrate failure in your commune; especially if you fail fast. Failure is the elimination of a path of enquiry and also lessons must be learned and recorded

Shared Roles and Knowledge

• Eliminate knowledge hoarding and any barrier to entry for a particular competency
• Although you’ll have expert areas within your team, that doesn’t mean your platform expert does all the platform work. Quite the opposite, if anything, the platform expert should be the one reviewing, guiding the team on the rest of the work
• Many/most developers also make excellent analysts, they just need some analysis discipline
• Many platform engineers are also great developers
• Testers are coders too, why don’t they also contribute to the main codebase?
• This means you collectively need to make a commitment to teaching your team members to fish
• Your experts are there to set best-practice and help engender capability, not be a single point of failure
• Have lunch and learns or lightning talks and consider sharing them outside of your commune

Just this act of becoming genuinely T-shaped (sorry, I shuddered too when I typed it) alone should benefit your team greatly.

Values Alignment

It's really hard to build a cohesive team if your values aren't in some way aligned. I'm not expecting people to all convert to veganism or the same religion, but there should be some shared vision. Otherwise, when you're asked to compromise and you're not bought in, you'll just shrug your shoulders and say "ok".

• Ensure your principles and ethos are up on the wall. Draw them up together, agree them, post them for all to see
• Have your goals up on the wall, showing concrete progress towards them. People like to feel progress is being made, it maintains buy-in
• Make each community deliverable a “system” or a product. This means you have to put some serious design thinking into how you split up your larger system
• Everybody in your commune must be bought in to the way, which raises some questions around those who can't or won't adapt to this way of working

What are the Benefits of this Approach?

• Increase richness and accuracy of communication
• Reduce time taken for communication
• Reduce the number of people required and involved in your project
• Reduce and focus inter-team communication
• Encourage transparency through a “no blame” culture. More honest status reporting and estimates
• Your team should feel insulated against the kind of pressures that would result in compromises in quality
• Reduce/remove skills siloes and single points of failure
• Improve quality, morale and delivery pace

The upshot of this should be that you deliver on time, and that the victory needn't be a phyrrhic one.

Working with Other Teams/Communes

Unless your project is tiny, you’re going to have to put your head above the parapet at some point and share your work with the outside world or consume other teams’ work. An effective programme-led communication plan is required to ensure this goes as smoothly as possible.

One main point is to keep your build and other moving parts separate, as being brought to your knees by a central build being broken is a real sign of CI/CD immaturity. If you're in that world, guess what, you created a distributed monolith. Each commune should maintain control of their own Git repo, build and build artefacts at all times, taking only versioned artefacts from other teams, don’t just use the “pull from head and hope” method, it’s too messy. The other benefit you'll get from that is really well controlled release versioning and thus fewer bugs popping up further down the path-to-live.

Treat your boundaries as APIs where possible and ensure you map these boundaries out up-front, providing sandboxes and stubs for teams wishing to consume your services. You can then act collaboratively by implementing parts of your services as required for other teams through carefully mapping dependencies on the programme level. Having your interfaces mapped out with stubs allows for early integration and less rework.

Some Problems Not Solved by this Approach

This way of working could solve many woes, but would still leave many puzzles on the table, not all of which may be solvable. However, maybe that's for the best, we shouldn't try to boil the ocean as we fail when we try to do so.

The main question that remains for me, is how we deal with shared team members, services and components? They are inevitable in an enterprise-scale programme and some of the finer grained things around that include:

• Maintaining a core architecture and set of standards without putting too many restrictions on the team (although, I feel this one just needs some good "adulting" through light-weight agile governance)
• Dealing with location-diverse teams, attitudes, skills gaps, culture etc.
• Using communication effectively - telepresence/dealing with teams in different time-zones
• Planning dependencies to keep information and functionality flowing between teams while maintaining the minimum level of coupling
• Having a leadership team that can float between communes, sympathetic to the nature of each of them

The other main concern is around those who don't want to work collaboratively in this way, the ones who just cannot bear to connect with their team. How to reach them, do we even try? Or do we move the non-team-players into roles where they can do their thing how they want to?

Conclusion

Fixing agile for the people, is a complex issue as you have to fix if for all, neurotypical, neurodivergent, being inclusive as possible. Trying to do this by putting everybody in a single, uniform environment cannot ever really work, because people aren't cardboard cutouts.

But there's a danger to breaking your teams out into communes as being too parochial means you miss out on opportunities to share or reuse and possibly create a future code-ownership/knowledge transfer issue. However, having a huge programme of people all attempting to take in a blunderbuss of information is also not an efficient way to work. In many ways, your communication matters as much as your environment.

With some focus on team structure, working environment, practices, breaking down role barriers and reducing communication overheads, I believe we can considerably improve the happiness and thus the efficiency of our engineering delivery teams. We can also acheive all of this while being hugely inclusive, which should be a great selling point for any organisation.

We owe it to ourselves to continue to improve how we implement Agile without dogma, improving the lives of those we work with. Because we work to live, not live to work, but either way, we have to work, so we might as well make that experience as pleasant and productive as possible.