Sherlock Holmes and Data analytics

Recently I had opportunity to read “The complete Sherlock Holmes” by Sir Arthur Conan Doyle. The stories are of the time when data points were limited so were the tools available to analyze but while going through the book I realized that the author had such extra-ordinary sense of analytics that it could be guidebook for starters even today.

The term “Data analytics” is quite often explained as “the process of examining datasets to draw conclusions about the information they contain”. Obviously, data points by themselves do not give us information or allow us gain any insights /conclusions and various tools help us to get deeper into the patterns they reflect, allow us get into possible “cause – effect scenario”, allow us to establish connection between to apparently unconnected events, give us useful predictions on what we can see in future…The list is long.

Surprisingly, we can see similar results in his books with such abundant clarity of thoughts and they display various tools he employs to solve cases.

Some of the points to highlight here are:

1. Stay focused on what we want / is relevant to the case : This is reflected not only in the cases that he handles but even in terms of his interests in general where all data related to other aspects in the world are simply ignored as of “No interest”.
2. Observe, observe and observe : There is no substitute to keep eyes and ears open to all inputs. Today, means have changed, reach has also grown multifold and developments at one end of the world can impact something at the other end in no time…So we keep on looking for signals in all directions.
3. Do not postulate till you have data : That is a very clear recommendation of approaching the situation with open mind. When we try to fit data in existing pattern, that we may have in our mind, we obviously start looking for data which suits that pattern and ignore other data points staring at us. We see this happen in social media all the time today.
4. Beware of red herring : While data is important it is equally important to notice and separate all that can mislead us in wrong direction…This is similar to identifying outliers to ensure that ability to see patterns does not get distorted.
5. Get down to minutest details : Things which may look irrelevant also need to be put on table. When we don’t know the patter there is no question of deciding relevance in advance. This is very relevant when we operate in real time scenario and there is great risk of losing that data for ever if not captured at that point of time.
6. Create and update database all the time: It is obvious that we may not be the first ones to work in similar situation and also experience always improves our diagnostic abilities based on earlier data sets /patterns. So he keeps database of many criminal cases which he may or may not have handled, even an example of keeping data of ash of variety of cigars is a classic example of using data base to quicken the pace of analysis or improve its accuracy based on historical information. Obviously, in today’s world where speed is critical in all such analysis, this aspect becomes very important.
7. Behavioral analysis : Almost all cases involve having a judgement of how people may behave depending on their own background as well as situation presented to them. I think this aspect is most relevant when we are trying to understand how our customers behave. Amazing to this getting used to set a bet (something similar to decide how effective promotional schemes).
8. Predictive analytics : There is nothing more important than ability to predict future events. It is a heady combination in many cases of inducing a particular behavior, trapping of criminal by predicting the next step he may take etc.?Such skills are crucial today not only to make quick decisions but sometime can be very helpful in avoiding costly error

As a summary, it appeared to me that there is a lot to learn, adopt and follow even from something written a long time back irrespective of whether you are a novice or an expert.