How do you evaluate the success of your solution architecture?

Measuring the success of a solution architecture is like baking a cake. It's not just about having the right ingredients but it's also about achieving the perfect form and taste to meet client needs and expectations.

Totally agree, success criteria is one of the critical indicators to understand if an architecture meets the requirements and goals of an organization. In some scenarios cost optimization is the top goal, in others high availability or security are the top goals regardeless of cost so defining those high-level goals at the start is very critical.

Defining success criteria is crucial before embarking on a solution architecture project. These criteria are measurable outcomes that signify the project's success, including factors like functionality, performance, security, cost, and more. Collaborate with stakeholders to identify and prioritize these criteria, ensuring they align with the project's goals and requirements. Document them clearly to guide the project effectively.

As others have stated an overall Solution architecture purpose is to meet the stated business and technical requirements. As well as these stated requirements there are also unvoiced expectations for an organisation for any system or architecture that they use. It is important to be mindful of these and can be grouped into: - How well a system or architecture can cope with change? This includes how easy it is to make changes - Whether your architecture can satisfy the initial requirements without significant change? - The benefits expected to be delivered by the architecture or system, whether these can be met by it or even succeeded. This is important as it is a tangible measure and will be part of project justification.

To ensure the success of your solution architecture project, it is important to define what success means for that specific context and scope. Success criteria are measurable and verifiable outcomes that indicate that your solution architecture has achieved its objectives and satisfied the requirements. Success criteria can be based on various factors, such as functionality, performance, quality, cost, time, security, usability, scalability, and alignment with business goals and strategies.

Involving business stakeholders in defining the metrics is crucial to make sure the architecture is a success. Sharing those metrics at the early stage of envisioning this architecture all the way through implementation to operations is important to make sure that the proper architecture and controls are implemented at every stage. For example; if security is one of the main goals, measuring the security posture of the architecture needs to happen at every stage of the implementation across the different stakeholders.

Defining exit criteria at each stage and by keeping a metrics and indicators to measure them. Also, it is important to implement the feedbacks from business stakeholders so that any gaps are identified at the early stage. Solution architecture can be deemed successful once the business requirements are seamlessly converted into working solutions.

Once you have defined your success criteria, establish how you will measure and monitor them. Metrics and indicators are quantitative and qualitative data that show the progress and results of your solution architecture. They can be derived from surveys, feedback, tests, audits, benchmarks, reports, dashboards, and analytics. It is important to select relevant metrics and indicators that can be easily measured and monitored. Track your metrics and indicators over time to identify areas that need improvement and make necessary adjustments.

Two core types of metrics: 1. "It was not there before, now there is something" 2. "There was something before, but it was not working - now we fixed it"

* Surveys: You can send surveys to your clients and stakeholders to get their feedback on your solution architecture. * Interviews: You can conduct interviews with your clients and stakeholders to get their feedback on your solution architecture. * User testing: You can test your solution architecture with users to get their feedback. * Metrics and indicators: You can track metrics and indicators to get feedback on the performance of your solution architecture. By using feedback loops, you can get valuable insights from your clients and stakeholders that can help you to improve your solution architecture.

* Meets customer requirements * Satisfied customer * minimal costly mistakes * Ease of deployment * ease of support * Flexibility for future functionality

The only thing to add here, is to implement feedback loops at every stage, starting from envisioning to implementation and operations. It should be an ongoing process not a one-time activity.

Never forget that you enable the business through your solutions. If the business is not enabled, your solutions are irrelevant and not worth money spent

Yes, I agree. On a periodic basis, talking to your business customers, product end users, your engineering delivery and operations teams, collecting metrics from telemetry tools, are some of the best ways to seek feedback for rating the success of your solution architecture.

Often we don't revisit the original architecture which was set up years ago and is able to cater to needs of the hosted application. In my view, an annual review of existing architecture should be done keeping in mind what has changed from original functional/feature perspective and usage patterns. What was originally designed (keeping in mind the usage forecast) may need minor tweaks with respect to Infra components sizing based on usage pattern after going live, would significantly save costs and also provide project managers the scope to reinvest the savings elsewhere.

In my opinion, it is also vital to revisit the objectives from time to time. IT infrastructures change very quickly and companies need to react to adaptations in the IT landscape and upcoming trends. The metrics which are tracked change a lot, hence keeping track of them is crucial to ensure a state-of-the-art architecture.

Data ingestion is fully automatic and storage is fully automatic. You have created AI processes that adapt to changing data flows and reorganize data models to suite the analysis metrics you are indicated to data mine to get results outlined in your metrics objectives. This is our future!

Measure your architecture. You do it tipically at least twice: at pre-Prod stage and after Prod deployment. At pre-Prod you run NFR tests (load/stress) against your defined criteria (number of concurrent users, response time, uptime, recovery time, etc). Make adjustments and move forward. After Prod, make sure you have observability mechanisms in place. Rate your architecture according to the number of perceived incidents in a given time range (0=A; 1-3=B; 4-7=C, etc), this will give you a prioritization criteria (if needed) to work on improvements. Also, ideally, your NFR tests will be emded in a pipeline so you can run them on a regular basis and check if added features or environment changes caused degradation of your baseline.

Adding another validation mechanism… How about a Capabilities Model where you lay out what your client will be able to “do” with functional capabilities, when… juxtaposed against what is needed (rqmts, evolutionary stages in transformation)

A robust solution architecture should clearly define the solution in a manner that allows for early verification. If the solution is flawed, it should fail quickly, enabling prompt redefinition and improvement. Therefore, a solution architecture should be designed to be implemented incrementally, allowing for step-by-step development and refinement.

The art of solution architecture is hearing what a customer asks for, and giving them what they desperately need without even seeing that. The elegance of that gift comes with experience, through continuous refinement of deliverable - making artifacts crisp, clear, and efficient.