All Observability Measurements Are Events
Logs, traces, and metrics all represent something happening at a specific time.
Technically, they're all events, and at my company, we treat them as such.
I'll explain why.
My Observability Experience
As the head of a logging product in one of the large observability companies, I witnessed firsthand the challenges our customers faced with the sheer volume of data they sent us.
They needed more control, but every time I brought ideas to leadership that gave customers more control over their data (and, in turn, more control over their spend), I was shut down.
Observability is now the second most expensive part of running a software business, growing at a staggering 30% year-over-year.
"Observability is now the second most expensive part of running a software business, growing at a staggering 30% year-over-year."
The problem is: The value companies get from observability isn't growing at the same rate as the cost of observability.?
It’s nowhere close to what they’re charging. Here’s how we solved that at my company.
All Observability Measurements are Events
Log data, trace data, metrics—we treat everything as an event.?
As CEO, I believe all data is a representation of something that happened at a specific moment in time. An event.
At Datable, we take data and normalize events into the OpenTelemetry standard. This creates a single, unified playing field for all of our data, regardless of origin.
Since data comes from various sources, we built our software to support a wide range of formats and protocols. Datable supports Syslog, Json, and Fluent, but we also support open source vendor protocols (from the big vendors).
By normalizing our data, we ensure consistency across the board.
No more dealing with "user_ID" in one place and "userID" in another.
领英推荐
"No more dealing with "user_ID" in one place and "userID" in another."
This also makes it really easy for us to go and send the data out to third party vendors. If we need to send something out in the Splunk or New Relic format, it’s not a problem.?
We take it from that OTel format, transform it into their API, and send it on its way.
How I Addressed It at My Company
Everyone should have the opportunity to create a solid data pipeline in the company. And it shouldn't disrupt anyone else's work.
PMs, SREs, and BI professionals can all use Datable's no-code pipeline builder to transform, enrich, and route data with just a few clicks.
When SREs build pipelines with Datable, they can trust that their data is secure and will always end up where it needs to go.
When Product Managers create a pipeline, role-based access ensures that no one will modify their pipeline without permission.
Each person can build the data pipeline they need to succeed.
That was the most straightforward, first-principles way of handling it.
In the future, everything is an event
By treating logs, traces, and metrics as events and normalizing them into a single standard, Datable gives companies the power to control their data and their costs.
We help teams create data pipelines that are actually useful to them, and any team member can create one.
Peel off the data you want, and route it anywhere.
You can even use Datable to have a live bake-off, comparing two services simultaneously without interruption.
New Relic and Datadog users save an average of 35% on observability when using Datable.
Imagine the impact cutting a third of your observability budget would have.
Want to pull this off in FY25?
Reach out to me. I'll personally get you rolling.
Co-Founder @ CtrlB / Data engine for Observability
1 个月That's a good way to look at the telemetry data. The only exception is that Logs also fall in the category of full-text search.
Growth Engineering | Enabling Tech Leaders & Innovators Around The Globe To Achieve Exceptional Results
1 个月Great article Julian