Neither 'Agile' nor Architecture are Going Anywhere

Many of you know I absolutely believe in what appears at first to be natural opposites, Autonomous Agile Teams and Large Architecture Practices. Not only do I believe both are essential, but my experience and today’s trends indicate that they cannot function long periods of time without each other. THey are both natural and extremely beneficial to each other, if certain conditions are met, though they will always create a certain tension. That tension, which I call healthy tension, is extremely valuable in the delivery of strategic business technology.

Shameless Plug: If you like these concepts, we teach them at Iasa, the world’s most inclusive business and technology architecture professional association. Next class is Feb 6th online: iasaglobal.org

Autonomous Teams

Im going on a bit of a history journey here so bear with me.

Team structures began being modified many years ago. Outside of truly large projects/transformations, I have not really seen the waterfall that others talk about in their career. Even in those 250+ million dollar projects we were breaking teams up functionally and into components (terms used at the time). There simply is no natural way to have 200 to 500 people do just about anything. As a chief architect I have had roughly that connected to project work including 50 internal and 150+ external personnel.

We didn’t have continuous delivery, agile or devops terms to use then but there was plenty of information on iterative development, spiral methologies and smaller team size. However, the huge walls of business to IT, analysis to development, and development to operations as well as the high cost of provisioning hardware necessitated a certain amount of disconnect between staff types. Oddly it was out of a strange situation in my employer (Dell) that I ended up becoming both truly agile and also a hard-core believer in architecture for agile.

Dell had two technical teams in Japan. The online team (which I was in) and the IT team. These were effectively bi-modal teams. Online moved quick because we were driven by retail demands, IT moved slower because they had to deal with manufacturing and systems of record. It was through the online team that I built out my first continous build environment and then continuous integration, continuous deploy was not even something we talked about, though we ended up with a somewhat automated version of that because I had operations on my team as well.

The autonomy we had allowed us to form cross-functional teams (business, operations, test, analysis, development) because we were small at first. Even as we grew our deep business integration (we sat across the room from sales and marketing and reported to the marketing VP) meant that we were able to setup teams to work directly with their business counterparts.

That situation also allowed me to write my first business case and get my own project funded. The massive success of that project won me the chief architect title. This is where I experienced all the pros/cons, successes/failures of both agility and architecture. I would later be able to see these in hundreds of other organizations.

It is important that you note that all of the below take always are related to how well a practice ages, renews itself and continuous to deliver excellence.

Agility and Autonomy in Reality

There are millions of examples of this in both natural and human systems. Pure autonomy, one where a small group defines its own objectives and direction is the death of strategy for a larger inclusive group. Unless those autonomous teams adhere to reasonably scoped objectives which are well orchestrated, they end up causing each other more problems than creating solutions. The decisions that are made by purely autonomous teams are often not aligned. We have seen this in all human social systems. Complete autonomy defies the ability to scale.

Autonomy is Needed for Tactical Execution and Innovation

However, some autonomy is needed because no strategist can understand a tactical situation completely. And very few executives can see where innovation can occur. Innovation happens close to the ‘customer’ or ‘stakeholder’. Thus autonomous, but strategically motivated teams work extremely well for highly unknown scenarios where speed and ability to move quickly are important. However, again these problem domains must be addressable by very small teams. When we are talking about transformation, that is NOT a small team.

Product Owners are neither Owners nor Digitally Aware

The core concept of the technical/business relationship has always been business stakeholders knows what they wanted and we assumed that was good for the overall business. In traditional terms this was implemented as requirements, in agility it is continuous feedback and product ownership.

But here’s the rub, technology doesn’t support a single business unit. It supports a business strategy which crosses business units. And business people do not really know what is good technology strategy for the business overall. Their requirements are optimized for the business THEY see. As a business strategist themselves a qualified architect often sees value in a further reaching and more balanced way.

And bushes units do not get along that well. They are broken out into groups which are semi-independent and hierarchical. They often dislike each other or consider their own goals more important. Think Finance and Legal. Think Sales and Marketing. And I don’t care how many html sites they setup for the family, they are (as a group) horrible technologists. And that is the death of agility, autonomy and most of all digital strategy.

I accept the real world, traditional business people have a lot of the power and will for some time to come, but tell me you aren’t sick of idiots telling us that Digital is a Priority, but then overriding qualified technology-based decisions for literally the dumbest shiny new thing? No we need digital product owners... oh duh, we call those architects.

We get Religious about Agile

Agile is good when it’s good and sucks when it’s not. It’s been 30 years we’ve been playing with it. And it is ultimately a project management, team size and component scoping methodology. It is neither inherently good or bad. But we get so obsessed with agile, we get lost in optimizing that instead of delivering. Microservices was a symptom of this obsession. Team size, velocity, time, time, time, agile, agile, agile. Lose the anger. Lose the religious obsession and agility works fine. With one caveat…

Agility Requires Skilled Architecture Teams

Yep, that’s right.

The elephant in the room is agility without architecture is going to continuously fail for all but very simple business models with technology at their core.

Amazon, Netflix, Uber, AirBNB, all shared one key aspect. They are were freakishly simple business models with a single technology platform at their core when they scaled. Oh some of them have gotten more complicated, and guess what started happening, Iasa started getting calls about training architects. Banks, Insurance, Defense, even Traditional Retail, all have complicated business models and ecosystems.

Agility and Autonomy at scale needs a robust nervous system to connect digital decisions and outcomes to a cohesive strategy and business or mission model (mission models are for government, military and non-profits).

Small Teams Form More Connections

As you increase team numbers and decrease their size they become quicker and better able to deliver. However their dependencies increase as well as the total communication needed to keep them functioning. Thus the ‘surface area’ for connections and coordination becomes seriously unmanageable without distractions. Of course each team member needs their own ‘guild’ but the overall technology strategy needs arbitration, facilitation and sometimes decision authority. Because we want small teams who act independently we need more architects to connect them.

Axiom: Great architects are PART of a team and NOT above them. I attend standups. I brainstorm designs with team members. I mentor. I deliver. This gives architecture a real seat in execution, not just strategy.

Technology Strategy Is Not Local but Execution Is

Features, quality attributes, functionality, stakeholders. Much of a single teams work is important but not strategic. This is sometimes not the case (Dell’s retail website for example). Much small team work is part of a working ecosystem that individually less important than as a whole. Dell’s would not have been possible without just in time manufacturing and supply chains. But without the ‘important stuff’ being executed and executed properly, the whole system falls down. Thus the penetration of architectural competencies is essential to determine and structure the chaos.

The scope and scale of the architecture practice plus its specializations allow it to reach throughout all aspects of the organization (business units, operations, data, even executive) allowing us to optimize technology value outcomes.

Digital Objectives Need Priority in Backlogs

Want to move the enterprise to little a or big A agile? Want to modernize the technology stack? Implement flex points in subsystems? Integration effectiveness? Harness information for outcomes? Deliver technology services? Event-Driven Architecture? Customer-Centric Design? Manage cross-system compatibility and quality attributes? Handle mergers and acquisitions well? Project/team thinking do not account for these outcomes. The product owner doesn’t understand them and the development lead is focused on speed, simplicity and delivery. They may not understand them either. Architecture connects big outcomes to little decisions. I have seen huge objectives brought low by simple development decisions.

Covering Technology Strategy is a BIG Job

From the board room to the basement. From idea to outcome. In between operating responsibilities. In between competing business objectives. With partners. With vendors. With an ever changing technology adoption cycle. From finance to legal to customer impacts, it takes a LOT of fascilitation, discussion, decision making and prioritization to deliver a balanced advantageous technology strategy. Even with a great architecture practice at the right size it is hard. When there are too few of us (as is the case in 90% of enterprises) it is nigh on impossible. The average ratio of architects to IT is 1% or less. Our research indicates that number needs to be closer to 6%. And not just 'titled' architects, but skilled and measured architects.

Traditional Roles Make Bad Architects (In General)

‘ I’m a great architect and I am a developer,

‘teams make better decisions than architects’,

‘my last architect didn’t even know what was going on’,

'you have to write code to be an architect',

'architecture is about the business',

I can hear the the fire building.

Ive made a career out of understanding what makes a great architect and what makes great architecture teams. And the rule is competencies make an architect, not a title. Read the competency article in the BTABoK... https://iasa-global.github.io/btabok/competency.html . Until a competency model dominates the profession, and people are measured against it, no generalisations can be made about what works, because they are being made without evidence.

Most Other Roles Don’t Enjoy True Architecture Work

The people problems, the spreadsheets, the decision trade offs, the uncertainty, the difficulty balance of money, outcomes, resources, opinions, and possibilities. How many of you technologists and engineers really love being in the heat of meetings? Or presenting to management? Or going and meeting with 2 other teams to hash out an identity management decision, or dealing with program management and procurement delays? Those are where an architect eats lunch. Making order out of chaos so our team(s) can deliver. Prototyping new ideas, jumping into production so our services get the data they need to function.

It Takes a Broad Number of Competencies to Be Great

There are 42 rigorously defined competencies in the BTABoK 5 pillars. Each of those pillars is essential to an architect (in one context or another). These competencies have multiple-sub skills. And all of that comes BEFORE specialization (business, software, information, infrastructure, physical systems, solution, and their derivatives). It takes a lifetime to become a great architect.

Engineers Don’t Think Like Architects

It boils down to this one. The engineering mindset is absolutely essential to the success of the enterprise. But it is in natural tension with the architectural mindset. I have held both lead engineer and solution architect roles at the same time numerous times in my career. And it makes my head hurt. It is not because I don’t like or understand deep technical decisions. It is because as an architect my outcomes are driven by value not quality, not well engineered products, not requirements. Value.

If you are interested in actually making architecture and agile work together, come talk to Iasa and schedule some time to discuss changing your paradigm.