Operational & Analytics Workloads - Part #1 Convergence

When you look at the world from a customer perspective, it often offers an opportunity to transcend Conway's law and all the variations of it.

Take Omnichannel. Consumers don't think in terms of channels, their unarticulated needs are not expressed in terms of "channels" but more likely in terms of "experiences". It is not just a pedantic distinction at play here. If you filter your thinking through the omnichannel lens, you will end up solidifying more silos than you wanted to avoid in the first place!

Operational and analytical workloads are another distinction that we, as tech vendors are happy to linger on. However, customers big data problems are rarely reducible to one or the other, and more importantly thinking in terms of product features massively limit the potential of innovation we can unleash.

These new technologies that have arisen in response to Big Data handle data creation and storage, retrieving and analysing data. Operational systems provide operational features to run real-time, interactive workloads that ingest and store data. Azure CosmosDB is a top technology for mission critical applications widely adopted in many organizations that require elastic scale, turnkey global distribution, multi-master replication for low latency and high availability of both reads & writes in their transactional workloads. Analytical Big Data technologies, on the other hand, are useful for sophisticated analytics of your data. Azure Synapse is a limitless analytics service that bring together enterprise data warehousing and big data analytics.

But picking an operational vs. analytical big data solution isn’t the right way to think about the challenge. They are complementary technologies and you likely need both to develop a complete Big Data solution. In fact, many of our customers have built amazing applications never before possible as a result of combining operational and analytical technologies.

In this three-part series, we will make a strong case for this convergence. We will start here illustrating some meaningful use cases adopted by many Microsoft customers.

In the next two instalments, we will then move to two ways to realize this convergence, more specifically we will go through how Azure CosmosDB works very well with Azure Data Explorer and Azure Synapse.

SCENARIOS

There are numerous potential benefits of this pattern from business growth perspective. Let's take a look at a few example to illustrate the value proposition which is applicable to most organizations across diverse industries.

Cross Industry: Dynamic Customer Profile

Last year I had the fortune to be part of a team developing a new way to conceive of Intranets (admittedly, a quite obsolete concept). The crux of the whole endeavour was a a single, actionable view of employees built from a combination of first and third-party data e.g. email, work habits, content created, projects, social activities on social platforms, etc. All the data point related to "behaviour" enabled the creation of a dynamic profile, unique for each employee, not structured, and organically representative of who the employees really are. There was no siloed thinking at play. From the outset, the Employee and the related insights generated were one and only one construct. Insights were either at runtime and conveyed through a conversational platform, or driving the HR to provide to deliver consistent, personalized experiences and measure the impact of those experiences.

This case can be easily extended to customers* to create a personalized experience based on a visitor’s browsing behaviour during a session, as well as frequent flyer account information, favourite airports, and average purchase amount. 

?Retail: Personalization

Retailers must build secure and scalable e-commerce solutions that meet the demands of both customers and business. These e-commerce solutions need to engage customers through customized products and offers, process transactions quickly and securely, and focus on fulfilment and customer service. Generating personalized recommendations for customers in real time is paramount to simultaneously look at who the specific customer is, his/her behaviours, dynamic profile, purchasing history, and process immediate recommendations or next best action suggestions. "Immediate" means milliseconds.

Manufacturing: Supply Chain

Manufacturers are onboarding to cloud-native technologies to break out of constraints of legacy Enterprise Resource Planning (ERP) and Supply Chain Management (SCM) systems. With supply chains generating increasing volumes of operational data every minute (order, shipment, transaction data), manufacturers need an operational database. This operational database should scale to handle the data volumes as well as an analytical platform to get to a level of real-time contextual intelligence to stay ahead of the curve. For instance, generate insights over the operational data across the supply chain using machine learning allow to lower inventory, operations costs, and reduce the order-to-delivery times for customers.

Looking forward to your thoughts and experiences.

* For the record, I used to say that employees are customers - but the upshot of this paradigm is broadly and factually under-appreciated.