Data... the new Oil... Are we extracting its true value?

In our day-to-day work there is a lot of data that we capture on various situations, and at various levels. But how much of this data is used productively for arriving at the right business interpretations and for helping management make the most informed business decisions?

Below are some interesting facts about such data across most organizations.

1. Less than 50% of organized data available in organizations is actively used in making decisions
2. Lot of data is available with people who have no idea of what to do with it and who shouldn’t be having access to such data…Industry average is pegged at about 70%
3. There is a lot of unstructured data that is captured at various levels. Less than 1% is analyzed and considered for making business decisions
4. 80% of analysts spend time simply discovering and preparing data and in creating colorful dashboards

Before we go further let’s understand the difference between information and data.

Raw data such as sales figures, customer retention rates, distribution costs are of very less value from a decision-making perspective. Once this data gets integrated with other data it gets transformed into information which can then be used for effective decision making.

Sales figures once combined with competitive data or market size or against target or against previous history gives a wider perspective than when they are considered alone. Like we always say once combined with other relevant data it brings out the actual story which lies behind the data.

What is data architecture and information architecture?

Data architecture describes how data is collected, stored, transformed, distributed, and consumed. It includes the rules governing structured formats such as data bases and file systems and the system for connecting data with the business processes that consume it.

For example, when a sales representative visits a store and enters the product wise stock into his handheld device the rules that govern what data to be entered in what sequence and where should it get stored etc., are what forms part of data architecture.

Similarly, when a customer relationship executive calls for a service call or a sales call there are a sequence of data that they ask for which is then updated in the system as per the data architecture that is defined

Information architecture governs the processes and rules that convert data into useful information.

So, after the sales representative enters the data the stock report is compared with earlier data to arrive at SKU wise sale and put into a dashboard as per the information architecture which is then auto circulated to all key decision makers to ease in decision making

The same is the case when the call center customer executive attends a service call to resolve an issue raised by any customer. Once the customer finishes the calls all the data about the call if the calling is done through a calling software are populated as per the information architecture and compared with the standard designed parameters and escalated if there is any aberration or gets added to the dashboard which is send out at the end of the day or as designed for that organization.

While designing your data management system organizations have to considering operating between two strategies:

Defensive strategy

Defensive data strategy is about minimizing downside risk. Systems designed for ensuring compliance to regulations, methods to limit fraud and systems which are incorporated to prevent theft, systems to protect systems and so on. Defensive efforts also ensure integrity of data flowing through a company’s internal systems by identifying, standardizing, and governing data sources. From a department wise perspective in an organization defensive strategy is driven by IT, Legal, finance and compliance.

The more uniform the data the easier it is to implement a defensive strategy

Offensive strategy

Offensive strategy comes into play in customer focused organizations and is usually driven by customer facing departments within an organization like sales and marketing. Offensive strategy will involve more real time system updating.

In an offensive strategy more flexibility will have to be provided for the data captured so that the data can be quickly collected and transformed into various ways in which they can be analyzed and interpreted.

It is very difficult for organizations to adopt only one kind of strategy and very often organizations will have to flip flop between an offensive and defensive approach depending on the situations and the organization needs.

Normally when as a team we identify a parameter or data that we want to capture for business purposes we all follow a simple process.

1. Identify the parameters to be captured through a brainstorming session and freeze the parameters and formats
2. Depending on the IT accessibility or how easy are adding thee new data collection points data capture starts happening
3. This data very often stays within the said department
4. One major challenge when such data collection happens is the primary field that is used for data capture. By primary field I mean how this data can be retrieved later. For example, if someone is considering name as the basic field for a customer data collection while updating the first person would enter my name as Rejo, the second person else as Reji and a third person and Rego and so on.?

The above example would be easily implemented within the organizations systems if the organization is following a offensive strategy. If it’s following a defensive strategy the said data would most likely continue to exist in some excel sheet without even being captured and updated in the IT systems of the organization.

Considering the fluctuating business needs and the needs of the various businesses there is a need for two types of approaches to data management

1.??????Single Source of Truth (SSOT)

2.??????Multiple Versions of the truth (MVOT)

In the Single Source of Truth one invaluable primary source of all crucial data like revenue data, supplier data, consumer data and various product details is fully grasped and defined and accepted by IT and across the business and everyone in the organization

From the SSOT which acts as the source several MVOT can be developed from business specific requirements for transformation of data into information. Thus, based on the needs of various groups or functions or unit’s distinct controllable versions of the truth are created that when queried yield customized responses according to the concerned groups predetermined requirements.

Earlier technological limitations made it difficult for implementing SSOT and MVOT simultaneously.

Traditionally organizations have used traditional Data Warehouses which stored structured enterprise data in hierarchical files and folders, but these were not very suitable for managing the vast amounts of unstructured data that are created and which needs to be used effectively for converting the data to information as per the needs of the organization.

In 2000, an architecture which came up is Data Lakes which can store virtually unlimited amounts of structured and unstructured data from databases to spread sheets to free texts and image files. A data lake can house the SSOT extracting, storing, and providing access to the organizations most granular data down to the level of individual transactions. It can also support the usage of this SSOT data in almost infinite ways in the MVOT to provide infinite options as required by the various departments and teams within the organization.

Today, a new data-centric storage architecture that leverages a combination of different technologies including Data Warehouse and Data lakes has emerged called Data Hubs.?Data flows into and out of the hub through endpoints and the hub allows businesses to see these data flows in real time. Data hubs allow for huge quantities of data, unstructured and structured, to be processed quickly and standardized.

In most originations these data related decisions are decided by the CIO and CDO. Data requirements and data capturing are being done at all levels of the organization as per the needs of individual managers and teams. Technology can help ease the processing of the data into information for taking more informed decisions.

But it is building the right awareness that will help all kinds of data that are captured at all levels within the organization to be captured within the SSOT leading to these data not being available in silos but being used at all levels of the organization for better decision making...

Hope I have given you enough information for thought...

Keep watching this space for more … and do subscribe to my new LinkedIn newsletter?Rejo’s Biz Bytes?and visit my website?www.rejofrancis.com?for more such articles