Enterprise Architecture: Assessing IT organization maturity with ITMAF

Sometimes, when involved on digital transformation engagements and pre-sales activities, and especially during the discovery/advisory phase, enterprise architects must face the question about: how we are regarding maturity across all IT processes ?

This question can be quite complicate to be answered if we don't have a proper tool and methodology in place to assess the current IT organization's maturity.

The term "maturity" here can be quite open-ended and sometimes, there's no common understanding from customer stakeholders on what "maturity" can be referring to; in fact, depending on the stakeholder we talk to, maturity can be defined in different ways such as technology maturity, processes maturity, DevOps maturity, and so on.

This is the right moment where we need to esentially have a tool that can help us to precisely measure the existing IT maturity across the diverse areas that comprise an IT organization such as:

• Application Maturity: measures how well applications have been architected in terms of architectural quality attributes such as availability, scalability, resilienct, consistency, data integrity, performance and security, among others. Coding practices, including code review, adherance to standards such as 12-factor and SOLID.
• UX/UI: measures the product usability end to end, including the frameworks / patterns and tooling as well as the standard in place.
• Data Platform: evaluates data linneage/provenance, as well as overall data architecture considering data partitioning / multi-tenancy and data lifecycle management policies in place ( cold-data / hot-data ), as well as data infra aspects such as data redundancy, backup, integrity and replication.
• Infra and Shared Services: common services that are available to all architectural components, such as CDC, data warehousing, reporting, data fabric, as well as cross-cutting capabilities such as logging,auditing,caching, distributed tracing, configuration management/ externalized configuration, centralized log analytics , as well as services associated with operational intelligence
• QA: overall quality maturity measured from standards/ frameworks/tooling in place, manual and automation QA, CI/CD pipeline hardening including SCA and CAS, perf testing, and overall QA governance
• Product: tooling/standards/frameworks in place to ensure proper product backlog tracking and monitoring of progress. Adherance to Agile best practices, quality for backlog creation and documentation. Completness of product specs including user stories, visual designs and solution design.
• Release Management: practices and WoW established to ensure proper product releasing, including major/minor versioning and release cadence policies. Product documentation via technical guides and product release notes.
• DevSecOps: measuring multiple different aspects of Dev + Sec + Ops, including CI/CD and SCM setup, deployment automation, Environment Management, Capacity Planning, Incident Management, Monitoring and log collection, Inventory Management, Patch Management, Security Risk Management, Cloud Spend Monitoring and TCO, BC/DR, Observability pipeline and metrics collection, SAST & DAST, among other areas.
• Architecture Governance: Measures how well the architecture board takes care of overall governance and common vision and roadmap across the various IT areas including engineering, DevSecOps, QA, UX/UI and design, Solution architecture/ Technical Architecture. This one is a key one that if missed, can cause the whole IT organization to fail, and must be assessed properly to avoid risking a project.

Evaluating all above areas for sure represents itself a discovery process for which multiple meetings with various teams need to be set up in order to understand and collect the existing maturity. While meeting with teams and stakeholders, it's better to have a questions guide so that we can drive the conversation forward across multiple topics.

I'm sharing here an IT maturity assessment model i have created called IT Maturity Assessment Framework [ITMAF] - to collect the multiple points across all above areas. Think about that typically, such a discovery can take from weeks to months to complete - depending on the IT organization size and complexity - and requires deeper investigation with the right parties involved on the meetings in order to capture the information.

Once all information is available, then the collected maturity levels can be averaged, and then shown in what's a typical "radar diagram" that represents the average maturity for each of the above areas, highlighting which areas require further development - enumerating the gaps found during the discovery process - , and which areas are complaint with the expectations. This typically is included in a high-level executive report that summarizes the overall view regarding IT maturity along all the involved parties and IT organization areas. A report like this, helps the upper management to understand where the budget needs to be invested on priority, and why, contributing to a more accurate decision making regarding allocated budget and roadmap.

During my own career, i have gone through this exercise multiple times, by applying the IT organizaton maturity assessment method described here, with pretty good results - i remember being at Banco Familiar - a mid/large sized banking company headquartered in Paraguay - during a digital transformation discovery during a 3-week on-site discovery process that we conducted, and the team getting rewarded and claimed by the upper management due to the great vision we gave to them - in fact, the upper management was completely unaware of the amount of gaps we found, and were happy to move forward in closing those gaps as part of a multi-year work agreement.

Usually, after the IT assessment report is written and shared with the customer, a follow-up meeting takes place, where the gaps found can be planned over time and shared for agreement - especially if you're driving this discovery from a consulting firm that has execution capacity. But most of the times, it's not the case, and the work concludes with the report which may be good enough for the majority of the customers.

Below i'm sharing the excel sheet i use to capture the maturity as part of the ITMAF and that can be used to drive forward this kind of engagements while we are on the initial phase regarding IT maturity assessment and data harvesting.

Of course, ITMAF is not only comprised by an excel file only, but also other supporting documents that describe the ways of work related to interviewing, collecting data points, weighing data points, and also a report format and report scope to cover. All these documents ultimately constitute the ITMAF framework, however the excel file constitutes one of the cornerstones of the methodology.