Big Data Dashboards…

Originally posted on January 24, 2012

Given my Business Intelligence (BI) background (3 years at Business Objects as the head of their Analytic Applications business unit), I’m naturally drawn to conversations about how the world of big data is going to impact the BI world. Now, the BI world has been taking quite a beating lately as the excitement surrounding the new data scientist role sweeps the information management industry. But it would be foolish, if not na?ve, to sweep aside all the work that the BI specialists have put into codifying organizations’ key performance indicators into interactive dashboards and reporting environments. So instead of talking about replacing yesterday’s BI efforts, let’s brainstorm how we can leverage these new big data trends to super-charge our BI investments.

Today’s Management Dashboards

The BI professionals have done an outstanding job of working with their business constituents to define the metrics and key performance indicators against which the business will monitor and manage the business. They’ve invested the time and the effort to create the reports, dashboards and scorecards to help the business better manage their businesses.

To quote my friend Dr. Pedro DeSouza, we can integrate advanced analytics with our existing dashboards and reports to give context to an event. We can use descriptive statistics to give simple, direct context to an event by positioning the event against the average and standard deviations calculated from history. Then we can use predictive statistics to forecast the likelihood of situations and other events for a given context. For example, Mondays after the Super Bowls (which won’t include the San Francisco 49’ers this year) are the days with the highest incidence of absenteeism: lots of “sick calls.” We can predict how many people will call sick and even predict who will call sick given the person’s attributes.

Let’s take a look at a sample dashboard. For this example, I have developed a simple dashboard that could be used by a package delivery territory manager to better manage their daily package delivery operations.

In this sample dashboard, the Delivery Operations Manager is interested in monitoring the following 4 key performance indicators:

• District Daily Deliveries Performance –used by the Ops Manager to compare the performance of their district to the previous periods and district averages.  The Ops Manager will use this to identify unfavorable trends that might require further investigation.
• Average Minutes Spent per Driver –is used to measure the effectiveness of the drivers and the route planning algorithms and to see if there might need to be some changes to improve individual driver delivery performance.
• Driver Absenteeism –used to identify and track trends and patterns in driver absenteeism, which is critical to proactively identify individual driver performance and absenteeism trends.
• Successful versus Unsuccessful Deliveries –is used to track the percentage and cost of missed deliveries, and costs associated with missed deliveries. Missed deliveries are probably the most important preventable cost item under the control of the Ops Manager.

This dashboard helps the Ops Manager monitor the key performance indicators that dictate territory performance and identify areas of the business that might require further investigation and investments. And each of the graphics in the dashboard would be interactive in that it would allow the Ops Manager (and their staff) to drill into more detail across multiple dimensions (e.g., days, drivers, routes, trucks).

Tomorrow’s Real-time, Predictive Dashboards

So how do we expand upon this current BI and dashboard investment? We’ll explore adding two new characteristics or dimensions to the dashboards:

1. Make the dashboard more real-time (or lower-latency) with respect to shrinking the time between when the data event occurs and when the data is available for analysis by the Ops Manager
2. Add more predictive capabilities (especially with respect to treating the existing KPIs as dependent variables) to provider finer-fidelity reporting, insights and possibly recommendations.

Let’s look how we could modify the existing dashboard KPIs and graphics (see graphic below):

• In the Daily Deliveries Performance analysis, we could move from a daily update of the delivery performance metrics to a minute-by-minute data feed. The benefit here is that problems could be identified and resolved within the same day that the problem was identified (where additional costs could be avoided), instead of having to wait until the next day when it is too late to fix that problem.
• In the Average Minutes per Delivery analysis, we could add more variables to the analysis, including detailed weather conditions and traffic data, in order to create finer-fidelity models that can predict average delivery times more accurately. And these models could be refined throughout the day as weather and traffic conditions change so that the average delivery times could be updated and communicated more frequently throughout the day.
• In the Driver Absenteeism analysis, we could again add more variables to create a more accurate Driver Absenteeism model. Factors such as hunting season and NFL football games could be taken into consideration, as well as unstructured social media data that might provide insights into other driver “distractions” that might impact absenteeism. Individual models could be built at the driver level that incorporate all of these new data feeds, yielding even more accurate absentee forecasts and predictions.
• Finally, the coup de grace, predicting and minimizing costs associated with Successful vs. Unsuccessful Deliveries analysis. Here, we might want to build a more comprehensive predictive model based on the improvements in the other KPIs (average minutes to delivery, absenteeism score, real-time delivery performance) to create intra-day predictions of unsuccessful deliveries. This would arm the Ops Manager with the insights necessary to take intra-day actions to more quickly identify and resolve unsuccessful delivery problems (and save the company substantial money).

Let’s take a more detailed look at how we could use a combination of predictive analytics and low-latency data feedback to more closely monitor intra-day delivery performance.

The chart above is an updated version of the Daily Deliveries Performance analysis. I included a real-time chart showing the delivery performance of a day against the average plus and minus one sigma (standard deviation). In this example, the Ops Manager can see right away that the day didn’t start very well, with the aggregated performance slightly above “Average –1 Sigma”, which is still within tolerance. However, the situation deteriorates during the morning and, by lunch, the aggregated performance of the day is below one sigma of the average. The manager could then take action at that moment to root cause the problem and remediate the situation.

Summary

I find the potential for super-charging companies BI investments very exciting, maybe because I was one who over the past several years helped companies create these analytic BI and dashboard environments. Now with many of these big data innovations (e.g., social media data, high velocity data feeds, MPP architectures, in-database analytics, in-memory computing), we have the opportunity to build upon that BI investment and create a business environment that is both more real-time and more predictive.