How can you design agile application architecture for easy debugging and troubleshooting?

- Assign?each?of?your?program's?primary?features?and?functionalities?to?its?own?module. - Give?your?modules?well-defined,?uniform?interfaces?and?contracts?that?clearly?identify?their?inputs,?outputs,?and?dependencies. - Capture?the?inner?workings?and?how?your?modules?are?implemented,?letting?other?modules?just?see?the?information?they?need?to?know. - Optimise?the?number?of?couplings?between?your?modules?and?boost?their?cohesiveness?to?guarantee?low?interdependence?and?strong?internal?consistency. - Implement?the?separation?of?concerns?principle,?which?states?that?every?module?has?a?distinct?duty?and?goal it in an agile way

If you are looking at an application architecture for its agility and effective debugging, a common logging and error handling reusable component can be used across your application landscape in your design pattern be it SOA or ESB, route the monitoring notification message to a messaging queue and handle it however you want from a standalone queue listener process and generate reporting or much more with the data..

I think of 'agile' as a method of navigation, like sailing. In sailing, the navigator must constantly adjust the sails and rudder in response to changing winds and currents. This represents the agile methodology's adaptability and responsiveness to change in the development process. Architecture, on the other hand, can be likened to the design of the ship itself. The ship's design must be thought out and stable before it's built. While minor adjustments can be made while at sea, major changes to the ship's structure are not feasible without returning to dry dock for significant reconstruction.

In agile work, the crucial part is to have a clear feature model and an even clearer breakdown. This model helps us create precise modularization in the architecture, pinpointing the exact system parts to debug.

While testing tools and frameworks will help you to automate regressing tests, and techniques like fuzz testing can help fine certain type of bugs or security issues, good hand crafted test data by your users is invaluable. Of course creating test data by developers or via some automated method is also useful, but since ultimately the users are the ones who actual use your system, they can provide mocked real world data that can help ensure your code functions as expected in both positive and negative test cases. Not to mention unveil missing requirements such as additional validation logic on the inputs as well as potential model and algorithmic issues given certain combinations of data or even the ordering how how they enter that data.

Again, testing can be overkill and is the result of poor planning. Why do you want to maintain three sets of code? One functional, one written by the developer to test, and one written by QA to test? Engineer the software, trust the process, and focus on the functional logic to be implemented. If you have a flowchart, a simple flowchart, you can focus on the logic and the QA department can focus on creating proper testing based on the logic.

Logging is good for debugging, auditing, and tracing, but is not the be-all-end-all. Concepts like Open Telemetry are not unique to Microservices or Cloud Architecture. A company I was a Middleware Lead Software Engineer in 2005, the Chief Information Officer mandated all system implement "Synthetic Transactions"; a transaction that could flow through every component regardless of tier in production. These mimic real user and system generated transactions but the system knows it as "test" data and while allowing it to flow through the real process, otherwise treat it as test data only. This allows one to observe the system and take performance and stability metrics in real time in production at a level that a real user would experience.

Besides using tools to monitor app behavior, I think pairing up team members for periodic programming sessions with senior engineers is key to maintaining agility. It enhances everyone's coding skills and gives senior engineers a broader perspective, ultimately reducing issues detected by those tools.

I agree, in distributed system we must need tools for log aggregation, monitoring, dashboard and alerts. AIOps provide preventive measures.

Stick with open standards like OpenTelemetry. All the big vendors support it and when you run into the inevitable issues of cost and having too much telemetry you will be able to easily remediate them and make sure that the data you are surfacing is actionable and useful.

Don't overdo it. In a million lines of code there might be a dozen procedures you really need to observe to this level of detail, and there are other ways to evaluate those procedures that are far more efficient. This is why we have development, QA and UAT environments. We don't want our application logging, tracing and monitoring when it needs to be processing. Instead, take the time to create flowcharts of the logic and identify issues in the flowchart. Take the time to create process diagrams and sequence diagrams so you have a roadmap of where the software goes to perform a function. Create ER diagrams to create efficient queries. Engineer first, software second.

To kick off and sustain agility in a project, the key is having a domain expert as the source of truth. Other aspects can adapt with various tech solutions.

It is important to understand that your architecture is supposed to evolve, start with a minimal viable architecture /w legroom. (MVAL) This is to suggest that diving straight into microservice architecture often doesn't need to be the first step. 1. Focus on clear boundaries at the code level, this mimics a microservice at the code level. 2. Assure that your features are working as intended 3. Assess the need for evolving the architecture, and consider the holistic picture; its current form might be the Apex or it might be time to evolve to address the next set of business needs. By doing this you deliver value early - find the MVAL

If you create the right diagrams before you code anything you'll be able to follow the logic in the code along with the logic in the diagrams to troubleshoot things more easily AND verify bugs with the documentation. It's hard enough finding errors and troubleshooting without a roadmap and that's why flowcharts, ER diagrams, sequence diagrams and process diagrams are as important to debugging and troubleshooting as any logging can be. Be verbose with your logging but don't log things that aren't helpful, and don't log things that don't need to be logged. Logging for the sake of logging is a waste of so many things. Show the error, show the stack trace. Keep it simple.

Don't be overly verbose. If you have a large application, limit the output to meaningful errors and avoid logging everything that happens. Even Kibana gets overwhelmed. Log what you would want to see if you had no prior experience with the code and limited time to figure out the problem. Front load the software design process to fully document the logical flow of the code and create pseudocode prior to coding. These two documents allow someone to understand the code quickly and helps to identify where the logical errors are most likely happening. It also significantly lowers the chance that an error will occur to begin with.