From Good To Great: How Operational Analytics Gives Businesses A Real-Time Edge

Originally published in Forbes

All businesses today are a series of real-time events. But what separates the good from the great is how they capture and operationalize that data.

Companies like Uber have talked in-depth about how they use real-time analytics to create seamless trip experiences, from determining the most convenient rider pick-up points to predicting the fastest routes. For the last decade, the big data movement has been about capturing a lot of data and crunching it to identify problems and make better decisions. What's refreshing about Uber's approach is that it doesn't collect and store data hoping to find insights -- instead, it has operationalized event data to take automated actions in the Uber app in real time.

It's not just modern companies like Uber that have real-time data that can drive intelligent actions on the fly. A toothpaste company can use point-of-sale data to manage inventory and ship more units to local stores that are running promotions. A medical devices company can deliver more insulin through a smart pump based on a patient's fluctuating glucose levels. In fact, IDC predicts that by 2025, nearly 30% of all data created will be real-time (compared to 15% in 2017).

Unfortunately, many businesses are still stuck in the old world of data where they had to choose between transactional and analytical data systems. Typically, transactional systems are online databases that are best suited for order entry, financial transactions, customer relationship management and retail sales, but they're not ideal for complex queries like determining how much of a particular product the business sold in a certain region this week and how that compares to last week.

For such complex queries, analytical systems like data warehouses have been the go-to solution, but they tend to be too slow because they need new data to be prepared, loaded and analyzed in batches. Meanwhile, modern companies have quietly embraced a whole new world of operational analytics, which completely transforms the way data is collected and consumed by the business.

Operational Analytics Feeds Real-Time, Data-Driven Automation

The primary focus of businesses using operational analytics is to increase revenue and margins through extreme operational efficiency using real-time data. This is not an isolated initiative in one corner of the business. It translates into the marketing team harnessing user data to make more personalized offers to customers while they are in the store. It means better manufacturing productivity using sensor data for predictive maintenance. It provides a unified view of the business so that inventory can be proactively managed in advance of new sales promotions going live.

What's common across all these scenarios is the fact that data is not being used for insights after the fact. Instead, new data is being processed immediately and is consumed by live software applications to take actions automatically.

Entirely New Data Stack For Operational Analytics

One of the biggest challenges in the move toward operational analytics is that the existing data stack is simply not able to handle the pace at which new data comes and is not set up to process the new types of data being generated. It is incredibly tempting to make small incremental changes to modernize the current data infrastructure, but the reality is that the most successful new projects embrace a whole new cloud-native stack that allows them to move fast and show real value quickly.

Today, technologies to capture streaming data, such as Apache Kafka backed by Confluent Inc. and Amazon Kinesis, have made it easy to capture and store event data, but processing that data is a whole different challenge. Rather than traditional warehouses, streaming data can be better processed by modern search and analytics systems like Rockset. And lastly, instead of visualizing insights in static dashboards, aim to operationalize them in the form of live dashboards or data-driven software applications.

A Grand Challenge And A Grand Opportunity

Not surprisingly, Gartner has named continuous intelligence using operational analytics as one of the top tech trends for 2019.

“Continuous intelligence represents a major change in the job of the data and analytics team,” said Ms. Rita Sallam, research vice president at Gartner. “It’s a grand challenge -- and a grand opportunity -- for analytics and BI (business intelligence) teams to help businesses make smarter real-time decisions in 2019. It could be seen as the ultimate in operational BI.”

Good businesses use data to make informed decisions over time. Great business operationalize data to automatically take actions in real time.