The Data's Law

Assimilating data to derive information is a constant struggle for every organization. No matter how many analytical tools you deploy, every time you have a crucial review meeting most important data point would need out-of-system intervention.?All would agree data assimilation is the toughest part; while extracting information is a relatively easier part of the equation. Let’s examine what makes data assimilation the most energy-draining exercise and what can be done to make it less painful.

Like Murphy’s law on various matters, there is this Data's Law, and they are

1.??????Your superiors will always ask for data in a form and shape that is different from the way it is maintained. ??

Just recall how many times you have come across this issue. It could be due to all of us being under pressure to present data in a new format every time. The pressure to present oneself as a creative, innovative, and deep thinker is so intense that for a very trivial info a large team ends up spending lots of time. It is also not uncommon that in the end such data point is either not put in the presentation or not discussed at all during the meeting.

If we dig deep, we realize almost all the time departmental heads never spend enough time and energy on data structure and data points that are captured in systems. A rough estimate of the time required on the above issue is 25 man-days per quarter for a company of 3000+ manpower. Mind you, these 25 man-days are that of middle to senior management personnel i.e. it is not cheap.

2.??????Data that is considered comprehensive today will be inadequate tomorrow.

The comprehensiveness of the data is like climate, it changes on variables such as the presenter’s experience with stakeholders in the meeting, Stakeholders’ preferences, and the purpose of the meeting. ?As and when a new angle to the subject is added, existing data for a specific or few specific points become obsolete. These changes will happen every time you edit the definition or introduce a new segment to the existing data set. Example of an edit of definition – what is considered as revenue, what is considered as the level of manpower e.g. top, senior, middle, and junior management, what shall be considered as cost while computing productivity etc. ?Examples of introducing a new segment to the existing data set e.g. new segments of looking at manpower types such as Individual contributors and people managers, redefining/segmenting business groups, On roll and contract employees, Segmentation on performance levels etc.

It almost always feels like one will never have data that is adequate for instant availability for any given meeting. There will always be a new angle, the new awakening that will give shock your current data structure and sense of data comprehensiveness. ?

3.??????There will never be enough time to present the most accurate data.

Unless the most comprehensive data is managed year after year in a structured manner with no change to definitions there will always be gaps in data. Unfortunately, most companies can’t engage a dedicated data resource with the right kind of professional training in managing data for the organization.

In this common situation, an employee with his/her full time day job enters the field of data analysis. Particularly, when data is stored in multiple systems including individual employee’s laptops, it becomes a herculean task to put together sanitized data in the required format. Mind you it is energy guzzling and time taking task. It is highly engaging but unproductive. No amount of time is adequate to arrive at information without a tinge of doubt.

Upon entering a new month, we realize last month was super busy but there is so little to report, one can bet most of last month has gone into data churning. These are the productivity blackhole periods. ??

4.??????The curse of trend analysis - Data sets of the past will always have the most critical information missing.

This is the most important law of data. Like retrospective tax, this is the worst enemy of rightful reporting. Data as a stand-alone set of information is useless unless it is compared to how was it in the past and based on the trends, how it would be in the future. The inherent problem for trend analysis is the current data set would always contain more data fields than what was stored in the past and it happens due to ever-evolving new angles to look at data and new awakening among stakeholders. So, what it leads to is a unique challenge of guesstimates. The struggle is due to the present sense and sensibility imposed on the past to identify a trend; it is like judging George Washington on human rights abuse because he had slaves.

Data collators pull data from old records (if such data is inadequate, which is the case most of the time) and fill in the gaps with smart guesses to determine the content of the new data field to match the requirements. Hence, one would find data collators always venerable. Their professional life is most affected by the third law of data i.e “There will never have enough time to present the most accurate data.” ?If proper records and backup notes of such estimates/guesses are not maintained professionally, it will not be a surprise that the same individual will present slightly different numbers for the same report after some time.

There is no easy way out of this problem unless an organization invests heavily in systems and resources, which may not be economically viable options for most organizations. However, the following steps can reduce pain in this process. First, at the highest level in the organization time shall be spent to determine data points and definitions that will stand the test of time (at least 2 Years). Second, systems shall be capable to capture the required data. Third, employees responsible for entering data into the system shall be measured for data quality and held responsible for gaps. Forth, an organization shall train select employees in data science and analytics. Many disregard the fact that data management is a science with specific rules and processes. Fifth, Reviews shall maintain a predetermined standard of reporting, thereby we can rein in people managers’ imaginary horses.