Operational business intelligence: Improving BI to meet the needs of operations teams

Introduction

In today's digital age, businesses are generating a huge amount of operational data, making data an essential element in operations. This is especially true for operationally heavy industries, such as marketplaces, ecommerce, logistics or fintech companies. However, most operations teams still face challenges in getting the most value out of their data, which can hinder their ability to achieve operational efficiency and higher margins. In this article, we'll explore ways to overcome these challenges and implement true operational business intelligence.

Key Challenges that Operations Teams Face with Data Usage

Data Readiness: The first major challenge is that the data generated by businesses is not readily usable. The data is either not transformed to the right format or not reliably updated, and therefore lacks certain data points. For data to be useful, it must be clean, reliable, and ready for business use. Imagine that you want to be aware of certain operational metrics, such as fail or issue rates, but the result is skewed due to missing data points. One way to mitigate this challenge is to use data integration tools that can automatically transform data into the required format and ensure that it is always up-to-date, however data teams often don’t have the required capacity to maintain these pipelines due to all the support requests they’re receiving from operations teams.?

Data Update Frequency: Many companies already have a modern data stack that includes centralized data warehouses with transformed and cleaned data, monitored with data observability tools for reliability. However, a common challenge is that these data warehouses are not updated frequently enough for operational use cases. Operational incidents (e.g., a delivery delay or an out of stock situation) require immediate reactions, however these warehouses are updated only a few times a day due to mainly cost reasons. The solution could be to optimize the warehouses to lower the computation need, or segment them to update only those tables real-time that require it for the relevant use cases.

Data Source of Truth: Another issue that operations teams face is that there are often several results for the same metrics from different data tables. This means that there is no source of truth for the same operational events or metrics, which makes it difficult to make important data driven business decisions. A common example is when high-level revenue metrics (e.g., average commission or GMV) can be calculated via various ways and sources and therefore confusion is created in the organisation.To mitigate this challenge, companies need to establish a single source of truth for metrics (e.g. in so-called metric stores) and ensure that all teams are using the same data.

Slow Interaction between BI and Operations Teams: Operations teams rely on BI teams to utilize data, and the way they interact and work together has a huge impact on the overall business results. Given that operations are a fast-moving function, these teams require new datasets or updates on existing queries even on a daily basis. This is something that most data teams currently cannot meet due to being overwhelmed with all the work they have.

Insights, but No Actionability: Even if the right data is received by operations teams at the right time, they often use it via static dashboards or reports that generate insights but do not drive actionability. Furthermore, dashboards require a lot of time to be regularly checked, not to mention the errors that can result from human nature. A good rule of thumb is? the following: if your dashboard has more than 5 rows, then most probably it should rather be an actionable monitor that alerts you if something requires attention.

How To Get To Operational Business Intelligence?

To move away from the current challenges and finally unleash the true power of data in operations, companies require a new mindset, new processes, and new tools to arrive at 'operational business intelligence.' Here are some best practices that companies can follow:

Rethink Roles between BI and Operations Teams: The most important principle is that operations teams should be empowered to use the data themselves. They should be able to create their own reports, set up their own alerts, and define the actions that should be taken based on data triggers. This requires new tools because operations stakeholders won’t be able to create complex queries with SQL or code integrations to their tools to trigger automations. Luckily, there are more and more no-code tools in the market for this - like Metabase or Looker for no-code dashboard creation or Flawless for no-code monitoring & orchestration.

Data Teams Shall Focus on Improving the Data Stack: Given that data teams today are already over-utilized, this can work only in conjunction with the points above - if operations teams are empowered to use data themselves. In this case, data teams can stop working with repetitive, simple support requests and focus on more strategic projects, such as creating more readily usable, transformed data tables for operations teams or optimizing the data warehouse to increase the update frequency at lower costs. The most commonly used tools by data teams for this are ETL tools like Fivetran or Airbyte, transformation software like dbt or data observability solutions like Monte Carlo.

Differentiate between Insights and Data Activation: Operations teams should stop using data only to generate insights for data driven business decisions; they should also activate that data to improve operational efficiency. For instance, they can detect operational incidents in real-time and respond to them before they generate larger issues. They can also use dashboarding tools to create visual charts on the most important operational KPIs (e.g., cost per order, NPS) or conduct analyses to find correlations (e.g., between certain issue types and NPS). However, they should also generate automated actions for repetitive processes with no-code automations tools like Nextmatter or Make.com. Furthermore, for anything that is not so repetitive or cannot be automated easily, they should use monitoring & orchestration tools like Flawless to detect issues automatically and push them to the right operations stakeholders so that they could solve them real-time.

Find Stakeholders Where They Are: Operations and data teams should also switch from using dashboards and reports for everything and meet stakeholders where they are to get the most value out of their data. Operations executives will still keep using dashboards to monitor the trends of the most important KPIs. However middle managers will rather use monitoring & orchestration tools to detect incidents that drive those high-level KPIs because that’s the only way for them to influence and improve these metrics. While lower-level operations colleagues, such as customer service agents or warehouse pickers, will need to be contacted in completely different channels (e.g., via SMS or in their ticketing system), as they don’t have time to look at dashboards or alerts on 3rd party tools.?

Develop Data-driven Culture – Lastly, companies need to develop a data-driven culture where data is used to inform decision-making at all levels of the organization. This requires a commitment from leadership to invest in the necessary tools and resources and to ensure that all teams are trained on how to use data effectively. By developing a data-driven culture, companies can unlock the full potential of their data and use it to drive operational excellence and higher margins.

How Can Organizations Start to Implement Operational Business Intelligence?

Companies should start by rethinking the role of operations and data teams and implementing the necessary processes and tools to do so. This requires empowering operations teams to use data themselves, enabling data teams to focus on improving the data stack, differentiating between insights and data activation, and finding stakeholders where they are.

Once this is in place, companies can start thinking about machine learning-based analyses in operations - we don’t recommend doing so before the steps above are in order.

If you already have some of your operational data in centralized, cloud-based databases or data warehouses, you should start empowering your operations teams. Use Flawless to activate your data and create real-time, automated monitors to detect operational incidents and orchestrate them to the channels that your teams use. If you do this, you will not only start saving significant time for your data teams but also increase efficiency in your operations by saving time for your operations teams and by improving customer experience and with that the retention of your customers.