Opposite Proverbs problem in analytics

Just like the proverbs "pen is mightier than sword" and "action speaks louder than words" we have many similar analogy in world of analytics. One of this contradiction is in:

  • Six Sigma
  • Fisher's Theorem

Now, very simply put the idea behind Six Sigma is to reduce variation; Where as Fisher's theorem states more the variation, higher is the rate of adaptation; i.e. as per Fisher encouraging variation in environment works better.

Wait, whats going on? We have a rub here. So we have this problem of opposite proverbs. We need to remember there is the No Free Lunch Theorem (and it states that for certain types of mathematical problems, the computational cost of finding a solution, averaged over all problems in the class, is the same for any solution method). Thus we should not believe in one algorithm, or one theorem that will work on all problems. So we need to back up and think about "which settings", "on which landscape" these two theorems going to work?

And there you have it, landscapes where you have spent a lot of time to find the peak (like medical procedures, or production line, or reporting & MIS) Six Sigma is the best fit as you would like to hang out on that peak. But if the environment is churning, the world is changing then you would like to have more variation and Fisher's Theorem comes to rescue.

Thus on a dancing landscape, where things are constantly evolving your Marketing Strategies got to change to match customer preferences. The landscape is advancing you can't rely on a checklist. So you have to encourage new ideas so that you can learn and adapt from them. We need to have multiple models and then adjunct models based on their assumptions and settings where they can work.

When you are not at the peak, you want to move towards it. In Six Sigma setting you presume you are.

Thanks,

Abir Mukherjee.

Koushik Kadidal

Chief Data Officer and Head of Insights Business

6 年

Very well written. On related lines the problem many times in the real world is that you assume you are operating at the peak but you aren’t. Analytically this results in data blind spots which limits your visibility outside your local optimal making it cost prohibitive to operate outside. That’s why good companies always ensure they have randomized test beds. Unfortunately, too many do not and become obsolete over time.

要查看或添加评论,请登录

Abir Mukherjee的更多文章

  • Arch supporter

    Arch supporter

    Today I woke up to find myself transformed into my own arch supporter. By 7:30 I had selected my beverage for the…

    2 条评论
  • Daylight to me

    Daylight to me

    When I give you the task I take it you have a clear understanding of the task, and I won't interfere with the way you…

    7 条评论
  • Predictive & Causal Relationships

    Predictive & Causal Relationships

    With the rise of Big Data predictive modeling has grown explosively in the last decade, while causal models have taken…

    1 条评论
  • Seduction of the Underdog

    Seduction of the Underdog

    Tactics, Innovation & Evolution The phenomenon to switch our allegiances for a singular contest just based on who's up…

    2 条评论
  • p-value = Indian Head Nod ?

    p-value = Indian Head Nod ?

    I like to think of p-value in statistics similar to an Indian head nod. I might expect to answer yes or no in relation…

    1 条评论
  • Teach Your MODELs Not To Memorise NOISE.

    Teach Your MODELs Not To Memorise NOISE.

    Design, train and tune a Machine Learning Model is very much like raising a kid through infancy, childhood and then…

    3 条评论
  • Empiricism vs. Rationalism : What sort of Analytical thinker are you ?

    Empiricism vs. Rationalism : What sort of Analytical thinker are you ?

    There are two broad camps of philosophies behind analytics that people fall into. And that defines how do you like to…

    3 条评论
  • Why Second Price Bidding works for Google?

    Why Second Price Bidding works for Google?

    Let me begin by clarifying my intention to write this article is solely to depict some peculiarity of this subject and…

社区洞察

其他会员也浏览了