Why we suck at problem solving

The immediate trigger to this blog is the less than expected performance by India’s ruling party of last decade i.e. BJP and what transpired afterwards in media, on social media & various WhatsApp groups. What one got to see was hysteria & then crazy speculation about why it happened..what were the root causes. Nearly 1000 reasons must have been thrown around.

Something similar happened during Covid. From the layman to the experts to the heads of states everyone had some solution or other everday. Often a contradictory solution to what they suggested just a few days back.

In both cases & through multiple other experiences I have come to realize that..

People are very poor at finding out root cause(s) of a problem.

When presented with a problem, coming up with a solution, implementing the solution & ensuring that it is sustainable is part of what I do , day in and day out. Having done this for several years, across several industries & with several organizations I reached the same conclusion as above.

Be it a technical problem like a memory leak, a behavioral issue or problems in process etc. It does not matter, what the nature of the problem is. When I look back, I can see some recurring issues.

“Co-relation does not mean causation” is an idea most people struggle with. Just because event B happens when event A also happens, just because when you increase input x, output y increases, they automatically jump to establish cause & effect relationship.

When multiple variables affect an outcome, two contradictory aspects need to be considered simultaneously. We need to isolate impact of each variable. Also we need to understand their combined impact & sometimes the interrelations between the variables themselves.

What happens is that one of the variables is identified (sometimes just co-relation, sometimes causation). It gets fine tuned & we announce victory only to realize that the improvements we thought would materialize haven’t.

Then there is the problem of complex vs complicated. Almost all techniques we are aware of are suitable for solving complicated problems. These techniques involve reductionist approaches which fail miserably in solving complex problems.

One of the greatest assets for a problem solver is humility. “I don’t know” is a very powerful statement when you are trying to solve a problem. It means that you are open to challenge/validate every assumption. While you might have tonnes of experiences, you are ready to take a fresh look at problem at hand. The customers/stakeholders though like to only hire people who can show a high degree of confidence and claim that “I know the solution”, “ This is nothing new for me”, “I have already solved it” or “Here ..I have a panacea..don't think JUST DO IT”.

Another challenge is often people who pay an analyst/problem solver want immediate actions. They are not ready to accept that problem solving needs time .. time to observe, time to analyze, time to hypothesize, time to run experiments & time to see the outcomes of a change. Yes sounds like someone is going to sit on his butt for long periods but more complex a problem, more time you need to sit on your butt doing all these things rather than being “Action Jackson”.

When I ask for time to observe/collect data, I am told “that is already done, here is the data”. There is very less appreciation of the fact that, data that is gathered by a 3rd party while it is a good starting point for a problem solver, going through the process of capturing this data in itself is a learning experience & provides lot of insights. It also avoids biases that the data capturing machinery might have.

Last but not the least, urge to declare victory prematurely. This happens the most in large corporates as there timing is everything. Just before the quarter end, just before the appraisal cycle one needs to show the results. So the moment green shoots of progress are seen, out goes the email with 20 people in to, 200 in CC. Then there is long chain of “Kudos”, “Amazing Work” etc.. till one discovers that the victory was short lived. Concept of letting the changes “BAKE IN” , wait & watch are alien.

To conclude , here are things to keep in mind

• Correlation does not mean causation
• When multiple variables affect an outcome, individual as well as combined impact needs to be understood
• Complex & Complicated systems need very different problem solving approaches
• “I don’t know” is a very good starting point. If this person has solved problems in past , he will solve your present problem as well even if his current status is “I don’t know”
• Problem solving is a lot of brain work. Don’t get biased by lack of visible action
• Success of solution depends on how well data collection happens. Don’t short circuit it
• It takes time to provide long term/permanent solutions for problems. Don’t be like Bart & Lisa with your non stop “Are we there already?”

Having identified the problems..actual process of implementing the solutions has its own set of issues. But that is for some other day to write about