Opposite Proverbs problem in analytics
Just like the proverbs "pen is mightier than sword" and "action speaks louder than words" we have many similar analogy in world of analytics. One of this contradiction is in:
- Six Sigma
- Fisher's Theorem
Now, very simply put the idea behind Six Sigma is to reduce variation; Where as Fisher's theorem states more the variation, higher is the rate of adaptation; i.e. as per Fisher encouraging variation in environment works better.
Wait, whats going on? We have a rub here. So we have this problem of opposite proverbs. We need to remember there is the No Free Lunch Theorem (and it states that for certain types of mathematical problems, the computational cost of finding a solution, averaged over all problems in the class, is the same for any solution method). Thus we should not believe in one algorithm, or one theorem that will work on all problems. So we need to back up and think about "which settings", "on which landscape" these two theorems going to work?
And there you have it, landscapes where you have spent a lot of time to find the peak (like medical procedures, or production line, or reporting & MIS) Six Sigma is the best fit as you would like to hang out on that peak. But if the environment is churning, the world is changing then you would like to have more variation and Fisher's Theorem comes to rescue.
Thus on a dancing landscape, where things are constantly evolving your Marketing Strategies got to change to match customer preferences. The landscape is advancing you can't rely on a checklist. So you have to encourage new ideas so that you can learn and adapt from them. We need to have multiple models and then adjunct models based on their assumptions and settings where they can work.
When you are not at the peak, you want to move towards it. In Six Sigma setting you presume you are.
Thanks,
Abir Mukherjee.
Chief Data Officer and Head of Insights Business
6 年Very well written. On related lines the problem many times in the real world is that you assume you are operating at the peak but you aren’t. Analytically this results in data blind spots which limits your visibility outside your local optimal making it cost prohibitive to operate outside. That’s why good companies always ensure they have randomized test beds. Unfortunately, too many do not and become obsolete over time.