A look back at Observability predictions of the past and where they landed

The rapidly evolving observability space has been extremely interesting over the past 5 years and I have heard pundits talk about many trends and predictions in the space.? There are trends that have been clear throughout the lifespan of observability but other predictions I’ve heard have never really taken shape.??

Here are three observability predictions that were prevalent in the late “twenty-teens,” and some thoughts about their development about 5 years later.?

1. The Big 3 cloud providers will be the biggest threat to observability vendors.?

While AWS, GCP and Azure are obviously key partners in the observability of cloud native technology, they also offer their own native tooling to monitor their customers’ environments.? They are in a unique position to provide insights because the data is already within their walls and additional capabilities are so easy to use in the platforms rather than engaging another 3rd party.? It would only make sense that observability becomes a first class offering from these providers.??

Well, this hasn’t come to pass.? While the basic metric tools [think Cloudwatch and Stackdriver] seem sufficient for basic functionality, these tools are rarely used by highly cloud-adopted companies.? Metrics are also a simpler component of observability and when it comes to the more complex distributed tracing, cloud providers’ native tools are even less prevalent.? This prediction may just need more time but so far, that crystal ball was wrong.

1. The single pane of glass [Metrics, Logs and Traces] is observability Nirvana.?

This particular topic was SO exciting around 2018.? All customers wanted a “single pane of glass,” all vendors were promising it and this prediction came true in spades.? It seems every vendor in the space now has a competitive offering for the single pane of glass either by acquisition or building in house.? In fact there are very few “point solutions” left? in the market as they have mostly been incorporated into other platforms to complement the other capabilities.??

Ironically, after every customer wanted the single pane of glass and every vendor developed it, many Observability teams still use multiple tools for metrics/traces/logs. They suffer through a suboptimal troubleshooting experience trying to understand incidents by using one tool for alerts, another for logs and maybe nothing for traces.??

1. OpenSource will be a key component of observability.??

Observability and the cloud native space has had many open source projects along the way [Zipkin, Jaeger and Prometheus come to mind].? Many of these were cutting edge technologies, foraging capabilities that had never been developed before.? Today the OpenTelemetry project seems to be the most widely adopted of any open source spec in observability and it does not show any signs of slowing.??

Opentelemetry.io is essentially an open source standard for instrumenting applications and collecting telemetry data [including logs].? It can send to any backend vendor or homegrown tool and avoids the lock-in of any vendor’s proprietary agent.? This flexibility is very appealing to observability teams and most are either using OTEL or have plans to incorporate in their next phase.

Some of these trends will continue to develop along with other emerging trends and I am excited to see where it goes.??

Thanks for reading.