APM is dying ....but now's not the time to grieve

MELT. That's an acronym to remember if you are into monitoring and observing the health of your enterprise's application landscape. These telemetry data components are known as the "three pillars of observability".?

Once upon a time, monitoring was a passion of mine, and a recent conversation made me reflect on this. I am still fascinated by technological advancements, and the architectural landscapes of enterprises have continued to evolve, as have technologies to ensure their availability and the terminologies used to describe them. I was once responsible for implementing the most effective solutions for monitoring service health, availability, and performance of enterprise systems at a tier-1 bank. Coupled with my responsibility for architectural dependency mapping, it was one of the most rewarding and impactful times in my career.?

I try to remain abreast of the evolution of Application Performance Monitoring (APM), and it is clear that with the prevalence of cloud technologies, DevOps, microservices, and the enormous surge in data volumes - a faster way to monitor and assess highly-complex environments is needed.?

Back to MELT. This acronym has stuck in my brain for many years. Understanding MELT (MEtrics, Logs and Traces) will help you understand the basics of monitoring and observability.

• Metrics: A numerical assessment of application performance and resource utilisation. i.e. What is happening?
• Logs: A machine (system-created) or human (developer-created) record to aid issue identification. i.e. Why is it happening?
• Traces: How operations move throughout a system, from one node to another. i.e. Where is it happening?

I've stated the word "observability" twice. It is one of the terminologies that has evolved with the technologies impacting APM. Frankly, a difference exists between?Monitoring?and?Observability, and while I'm at it,?AIOps.

• Monitoring:?Reactive. Getting an understanding of whether or not application behaviour and performance are working as expected.
• Observability: Proactive. Getting complete, deeper visibility across your systems and tying business metrics with technical data to identify the problem as soon as, and sometimes even before, an incident happens.
• AIOps: Getting meaning from the visibility that observability provides (technically, a part of observability)

The differences hinge upon identifying problems you know will happen and maintaining a way to anticipate potential ones that may. Just to let you know, some vendors use the terms interchangeably, which can cause understandable confusion.?

Collaboratively, monitoring platforms can feed valuable data to observability platforms, which can output to an AIOps platform. The AIOps platform can correlate events and identify problems using AI/ML (Artificial Intelligence/Machine Learning), thereby avoiding manual intervention, which is crucial when dealing with vast datasets. Besides the suppression of incident noise, AIOps can help ensure only 'real' incidents are alerted, their location, root cause, and (amazingly) what to do to put things right.?

Before you are completely sold on AIOps, it comes with a forewarning - it is not the utopia it claims to be. There are things AI (and AIOps) can and should address and vice-versa. Systems and applications that frequently update throughout the day and which change the software that they are based upon multiple times a day (such as a CI/CD process with thousands of code pushes that may contain new topologies and dependencies) make learning a "normal" state impossible. In their?2022 Global CIO Report, Dynatrace reports that 77% of CIOs state that their systems change every minute or less - this does not lend itself to AI approaches. A good AIOps solution should be able to learn which systems and applications they should target to avoid decision-making by a human.

A Gartner Magic Quadrant for APM and Observability shows the front-runners in the space.

With the rise of cloud-native architectures, application components have shrunk and become ephemeral. With distributed applications, containers will spread across multiple Kubernetes clusters and locations (Cloud and On-Premise). Traditional APM solutions can no longer point out the problems as simply as they once could. The data explosion means humans can no longer scroll through monitoring output effectively. The glorious days of the APM solutions I remember are almost over, and the USPs that vendors used to entice me to recommend them to the business have lost some of their sparkles.

But before you consider sending your APM solutions to the undertaker, discuss observability with your APM vendor, and investigate if this is part of their product roadmap. If your organisation is unlikely to require observability, your current APM solution may be good enough, so keep it!

Before embarking upon an implementation of an Observability platform, make sure you follow careful consideration and due diligence. Understand your needs, and avoid making things too complicated. Be mindful of costs, and ensure your solution can scale with your business and handle telemetry in a cost-effective manner to return a positive ROI.

4 Implementation Steps towards Observability

1. Future-proof your choice of observability platform. What fits you today may not fit your environment in 3-5 years.
2. Implement value-based monitoring. Irrelevant metrics offer no value, so aim to obtain visibility into big-picture issues.
3. Make observability an integral part of your incident management processes. Track successes, continuously improve, and document lessons learned.
4. Develop observability into your organisational culture. Develop and train staff, promote cross-functional embedded observability champions, and continue investment in observability.

Did you like this post? Please like, say why you liked it, and share it with your network. Follow me to read my posts and feel free to share your thoughts, ideas, and experiences with researching, implementing and using APM, Observability, and AIOps.

I'm now on Instagram! Follow me at??https://www.instagram.com/dean_a_consulting_ltd/

Useful follow-up reading:

5 Tips for Observability Success ?

https://www.datamation.com/applications/5-tips-for-observability-success/

How to evaluate modern APM solutions ?