The OTel Problem No One's Discussing

This notion of a collector has existed in the logging space for decades now. Logstash, Filebeat—these aren't revolutionary concepts.

Now the focus is on OTel.

But people aren't talking about what happens when you actually try to use these tools in a real-world environment.

Inevitably, your files get extremely difficult to manage. These programs are excellent at collecting data and sending data. But the more responsibility you put into them, the more complex and slower they become.

"These programs are excellent at collecting data and sending data. But the more responsibility you put into them, the more complex and slower they become."

As you scale, you erode OTel's core competency of collecting and sending data.

Start with one team: a simple transformation.

"Hey, I'm taking Apache data and converting it to structured JSON."

Great.

But the moment you introduce multiple inputs, different teams, complex transformations— the whole system falls apart. One modification can break everything.

So, when these requests come in from a B.I. team saying, “The CEO wants to report, and they need access to this data,” we’re going to ignore it.?

Nobody wants to touch that. —Too much risk.

So what do SRE teams do?

They lock down the configuration. And who can blame them?

Developers, business teams—they're completely shut out. It's a horrible feedback loop where the people who need the data most, can't touch it.

The real innovation isn't just in collecting data; it's in making that data actionable.

We handle this at Datable in 4 core ways:?

? Visualization that shows real impact

? Isolated branches for different team needs

? Robust state management

? Bringing business logic closer to the data

OTel and traditional collectors are excellent at sending data. We’re more focused on the insights that our data provides.?

Data collection is a means, not an end.

Observability should focus on how intelligently you can transform data into actionable intelligence. That’s the end goal.