Cribl, Explained

Last week, my team at IVP announced our investment in Cribl as part of a $200M venture capital funding round. Many people asked me what exactly Cribl does. As I started writing a post about “observability pipelines'' I heard Winnie-the-Pooh in my ear in discussion with Owl, having read the book ten times in about ten weeks to my two daughters:?

‘Well,’ said Owl, ‘the customary procedure in such cases is as follows.’

‘What does Crustimony Proceedcake mean?’ said Pooh. ‘For I am a Bear of Very Little Brain, and long words Bother me.’

‘It means the Thing to Do.’

“As long as it means that, I don’t mind,” said Pooh humbly.?

So, to take a cue from Winnie-the-Pooh, rather than speak of Observability Pipelines and Event Stream Processors, I will explain what I mean, and then I will elaborate on Why We Care.?

There are many different types of data. Transactional data is arguably the most common, and represents atomic actions, like purchasing a plane ticket. The action of buying a ticket generates lots of other data sources, often in the form of events and logs. You can think of those events and logs as side effects of the transaction. These are things like, which link a user clicked, which server completed their request, and so on. The quantity of the generated events and logs often dwarfs the amount of transactional data, and collectively the event/log data is often referred to as “telemetry” or “machine data”. This is the data that Cribl helps to manage.?

Why bother with these side effects, or this digital exhaust? Unlike transactional data, which represents the end of a business process, these metrics, events, logs, and traces represent how that process happened. As more companies realize revenues from digital experiences ranging from online doctor’s appointments, to ordering food, to streaming the latest Hollywood blockbuster, understanding the customer’s experience in these channels becomes critical. Gaining this understanding requires the ability to ask questions across these mountains of data and uncover new insights. In IT circles, this ability is called “observability”.

When taken in aggregate, observability data is as important as transactional data. But, as I’ve mentioned, the volumes are vast and the sources are diverse. Companies wanting more value from their observability data must grapple with both the volume and diversity challenges. Storing more data drives up infrastructure costs and degrades performance. Ingesting a diverse array of data formats and types complicates analysis and makes sharing data across systems difficult, if not impossible.. Every practitioner we talked to is up against a price barrier, but every single one pointed out that if they could afford to store more data in the right places, they would. The reasons for doing so were many: compliance, longitudinal analysis, faster and better security responses, among others.?

So, back to what Cribl does: Cribl is not trying to compete with the existing vendors that are solving the aggregation problem of bringing data logs together. What Cribl does better than anyone is management of in-flight observability data. Cribl does this intelligently: think Grand Central Station, not Rose Parade. Cribl can transform the data efficiently, and perform actions like delete, create, update, and more. The platform can also receive push or pull log data, perform an action, and then route to various downstream data stores.???

This explains why customers buy it. If you can apply logic to important data before it hits the downstream data store, you can apply various functions that result in reducing cost – storing streamlined logs, storing only meaningful logs – or increasing flexibility, like sending logs to their ultimate destination without an intermediate stop, enabling a cutover from one data store to another. Incumbent observability software tends to carry a high total cost of ownership, so cost reduction is always welcome.?

Which brings us to Why We Care. We care because of where Cribl can go from here. If there was a single investible nugget as I talked with customers, it was captured in this exchange that went something like this:?

[Cribl Customer]: All of the data we are getting from Splunk forwarders (aka data senders) we send through Cribl pre-built pipelines so that we can do transformations on it. Cribl grabs all of that data.?

[Cack]: Before it lands in Splunk??

[Cribl Customer]: Yes. before it lands in the Splunk data store. If we send it to Cribl first, we can route it off to where it optimally resides.??

This may read like a mundane exchange, but buried are two key ideas:?

1. “...we can route it off to where it optimally lives.” Cribl is a neutral zone, like Switzerland. Cribl need not compete with the existing data stores and instead can accommodate all of them, send data to all of them, and not ruffle feathers. (*So far.. See Point #2).??
2. “Yes, before it lands in the Splunk data store.” Cribl is upstream from the current de facto data stores and given their position it is reasonable to imagine that Cribl could one day become the data store. It behooves Cribl to offer a future destination for data at rest given what I view as one of the established Rules of Venture Capital: that data has gravity and once it resides somewhere it is hard to move. But, what makes Cribl different is their values and their open orientation. They believe in being customer first and vendor neutral, such that a Cribl data store will be an option for customers but not the opinionated answer.?

We all choose different investments, for different reasons. Here is a window into why we chose Cribl, not to mention the enabling trends in favor, like the shift to the cloud and the shift to microservice architectures (without which, the data volumes of this machine-generated data would not be as large.) Given the data volumes, coupled with the strategic position of Cribl, we think Winnie-the-Pooh, a Bear of Very Little Brain, might too agree.