The Agile Architecture Challenge

Over the past 25 years I have seen the pendulum of architecture swing back and forth from top-down, centralized architecture to federated, decentralized architecture and even all the way over to no architecture at the dawn of agile. There are benefits and issues with each of these organizational constructs for architecture, but I think as with most things the right answer is not on either of the extreme ends, but somewhere in the middle. The key is to have the right engagement model as part of a well disciplined architecture practice that has a consistent set of tools, techniques and adaptable processes. In the case of enterprise architecture at MassMutual we have a core set of dedicated enterprise architects across horizontal domains (i.e. strategic cross-cutting concerns) for data, security, infrastructure & cloud, DevOps, application and business architecture combined with a federated solution architecture (SA) community. This model enables the EA’s to concentrate on strategic deliverables such as architecture strategies, roadmaps, reference architectures and patterns while SA’s concentrate on solution, product and platform architecture. This keeps EA’s focused on strategy as they remain informed and engaged in concrete implementation and it keeps SA’s focused on concrete implementation as they are informed and engaged in strategy. In this carefully balanced model we strive for an 80/20 split with EA’s 80% focused on strategy and 20% on solution and the inverse for SA’s with them focused 80% on solutions and 20% on strategy. The two pillars that hold up this structure are community engagement and training. The SA community, which includes participation in governance meetings and communities of practice (CoP) as well as 1:1 coaching and mentoring with EA’s, is designed to facilitate two way communication and collaboration. The training program, which is composed of multiple certification levels ranging from foundational architecture skills to hands on architecture delivery, is designed to facilitate the application of standard architecture tools and techniques so that both EA’s and SA’s are operating together in a consistent architecture practice. I believe this model gives us the best of both worlds and positions us well as an architecture organization to take on digital transformation and agile.

While the organizational structure for enterprise architecture is important, how that organization executes through its engagement model is equally critical for success. In the past, architecture engagement had been aligned to a waterfall methodology with large architectural specifications that were produced and approved in the early lifecycle stages of a project. Such artifacts were often highly prescriptive, very rigid and rarely revisited as a project progressed. This often led to design defects, operational issues and significant disconnects between the original architecture and the actual implementation. As enterprises adopted agile development methodologies this past experience with architecture often lead to the conclusion that it was costly and largely irrelevant. Speed was king and lengthy architecture involvement slowed things down with little value. It was soon realized however that although multiple, independent agile teams could move very fast within their product silos, the lack of a big-picture strategic architecture or road maps to tie things together created redundancy, technical debt and organically growing systems whose components and services did not integrate well. Architecture was needed, but it needed to become agile. This meant that architecture had to engage at the right times, at the right level and have more frequent feedback loops to quickly course correct. This more agile approach requires a platform-centric architecture foundation that provides all the common services, capabilities, standards, technology stacks and patterns for development teams so they can compose their systems from reusable technology building blocks and deploy them onto reliable, scalable runtime environments. For a digital business these platforms include data (e.g. warehouse, analytics, database as a service, caching etc.), API Management (e.g. API gateway, domain models, registry, event streaming etc.), DevOps (e.g. CI/CD, Pipeline orchestration, Artifact registry etc.), Cloud (Infrastructure as a Service , Container as a Service, Function as a Service) and Security Services (IAM, Data Protection, Software Security.). With this foundation in place, solution architects and agile product teams can focus on the rapid design and construction of composable software that delivered business value, while enterprise architects could focus on strategies, roadmaps and patterns that guided longer term system evolution and integration. In this model developers are empowered to make decisions and work in an accelerated mode and enterprise architects only needed to be deeply engage when things moved outside of the agreed upon guardrails such as the introduction of new technologies and platforms or significant, complex architectural changes. Solution architects (properly trained in EA practices) are empowered to make the calls on when deep EA engagement should be triggered.

As part of the agile architecture process, enterprise architects routinely participate in agile ceremonies such as planning games, retrospectives and stand-ups (although not every day). EA’s also work with solution architects (wearing their product architect hats) to develop product roadmaps that look one to two quarters ahead of current time and recommend critical platform capabilities to be developed in alignment with strategic technology and business goals. Product architecture roadmaps then inform the shorter-term architecture runways that lay out the more immediate architectural prerequisites that an agile product team needs in order to be successful in the next set of sprints. These are architecture artifacts such as reference architectures, patterns, technical evaluations and key decisions that are delivered just in time ahead of the agile sprints where they are needed. This is a lot like laying down track ahead of a fast moving train and it takes a lot of engagement, collaboration and partnership between enterprise architects, solution architects and agile product teams to get it right. When we do get it right however it is formula for speed, agility, scalability and strategic alignment across the enterprise. Agile architecture does not have to be fragile architecture. With a well structured enterprise architecture team that includes a federated solution architecture community working together with standardized platforms, tools and agile practices we can move fast and build high quality applications that provide business value while remaining consistent with strategic architectural objectives. I think this is one of the best ways we can take on the agile architecture challenge and succeed.