TOGAF & Enterprise Architecture: the next iteration of Agile?

I am a product of the Agile generation. A series of attempts to improve project efficiency moved the IT world from big projects to iterative development: small teams and small, self-contained, self-defined assignments. The theory was the sum of the parts could self co-ordinate. We had come a long way with Agile, but really, where were the moon shots of lore? How do you co-ordinate the seemingly chaotic parts across the myriad of autonomous teams, units and divisions an enterprise had turned into?

It may seem ironic that "Boundaryless Information Flow" originates from the US Department of Defence. Indeed, in order to be more effective, and overcome the "fog of war" it is necessary that troops are coordinated and information flows freely.

These ideas, and others, gave birth to The Open Group's Architecture Framework of TOGAF. After over 20 years of Enterprise Architecture, of being a TOGAF sceptic, but standards advocate, I decided to take my own medicine and go beyond surface literature and engage in an in depth study of TOGAF. What better way to do this than to go through the rigours of getting certified?

I reflected on 20 years of enterprise work and drew a few conclusions, I hope will aid the discussion,

Around 2008, as an Enterprise Architect myself, in the middle of some major transformational work at the Commonwealth Bank of Australia, thanks to a visionary CIO, Michael Harte, I had the privilege of learning more about Enterprise Architecture (EA) from some of the all-time greats in the field, through MIT Sloan Professor Peter Weill (an Australian himself) and Jeanne Ross (in this video she explain EA) who claimed that Enterprise Architecture is not just a technology concern, but a business one, and good EA leads to more successful enterprises. In CBA 100s of executives heard this message direct from MIT, and Enterprise Architecture became a big deal from the CEO down. In the case of CBA, work at that time triggered real time payments in Australia, and really this is when we see the birth of Australia's number one digital app, and perhaps our most successful indigenous digital enterprise. In my mind there is a direct correlation between EA and lasting success.

Since then I have seen some major transformational work, from neo-banks, large corporates, to government initiatives and in all cases, I see a common theme: that good enterprise architecture is necessary for success, but not sufficient.

Why do I say that? Enterprise architecture, is commonly associated with a top-down, strategy first, control of the enterprise. From a social point of view, it sets a common goal, and lifts us out of the often animalistic instinctive self-centred behaviours, that while necessary for survival, does not get a community or enterprise together to reach a common goal. Without good communication and empowerment, this top-down approach can frustrate the projects on the ground, ultimately leading to dysfunction. Picture troops in a trench, without clear guidance, empowerment and communication, they don't know if they should fight or retreat, and every day in the trench kills morale and can only lead to decay and inefficiency of the war machine.

While standard operational process were born at the same time (ITIL late 1980s) they really only picked up later. Agile (early 2000s), Lean (also early 2000s), and wider adoption of ITIL were reactionary movements that re-energised the teams and troops. Agile's manifesto is a direct snub to the hierarchy approach, the musings of an angry mob wanting to uproot the establishment:

Agile Values:

Individuals and interactions?over processes and tools

Working software?over comprehensive documentation

Customer collaboration?over contract negotiation

Responding to change?over following a plan

Agile in hindsight was a necessary decadence. It included the business stakeholders in the execution, so they could see where their money was being spent and how difficult it is to get things done, when prioritising scope due to limited resources. The process has led to more tech-savvy business users, and this is a good thing. But too often I see without the top-down view, we miss the forest for the trees.

I feel other movements - microservices vs SoA, DevOps vs ITIL, even big data vs data architecture, are examples of bottom-up empowerment. All of theses are also necessary for success, but not sufficient.

Looking at this battle, between the top-down and bottom-up, putting our emotions aside, as the next generation of practitioners we need to acknowledge we need both,

There is clear empirical evidence that good EA leads to success. There is clear personal evidence that self empowerment makes us feel more useful and productive.

TOGAF, as a framework for Enterprise Architecture has evolved. The standard templates, views, terminology and processes help new people join an organisation and understand the process. They help partners integrate seamlessly. It has evolved to embrace lean, Agile, DevOps and ITIL.

Figure 2: TOGAF now facilitates a number of Agile-style iteration approaches.

You can always come up with a different way, perhaps a better way, and in a world of unique problems, the solutions need to be unique too, unique views can provide new insights or explain why a certain course of action is better than another. But ultimately, if as humans we want to grow at scale, we need to co-ordinate, the importance of standards should not be overlooked. That's why TOGAF is important. It is a common starting point, a common baseline, and there are few alternatives.

It is logical to conclude that enterprises seeking success should (re-)embrace enterprise architecture as a primary concern of their business because business that have good EA are more successful*. Second, we can all come up with a better mousetrap, but in a world of collaboration, why not build on a standard baseline, or improve it. TOGAF as a standard framework is an excellent diverse and tailorable framework that every organisation can use, so as a starting point, why not pick it up?

*https://scholar.google.com.au/scholar?q=good+enterprise+architecture+success&hl=en&as_sdt=0&as_vis=1&oi=scholart