So, how do we get started with that problem?

I've never been someone who can "wing it".

I like to plan out whatever I do and minimise the likelihood of being surprised. Naturally, I am most comfortable in structured environments or situations- such as classroom learning. Everything is predictable - for example, I know when I have a particular class, I know what material to review before working on some assignments. During the third semester of undergraduate studies however, I found myself in a bit of a pickle. 

There was a course project which was an open-ended problem. This threw me off, I had absolutely no Idea what to do. I started to "do" something without understanding why I was doing so. At one point, I decided I was getting nowhere and gave up. The anxiety and helplessness increased - the deadline was soon approaching! Thankfully, I managed to find a solution online. I meticulously followed the instructions and things started to work. Eventually, I was awarded a good grade for it. Upon introspecting, I realised a couple of things. First of all, almost everything in life was going to be like this open ended problem. Second, If I want to be a good problem solver I will have to do something to deal with such situations better. 

I knew I needed to plan better. I had to introduce some structure and find some methodical approach. Since then, I have been figuring out what works for me. That is what I want to talk about - my approach to problem solving. A word of caution - this is not a scientifically proven method. I have arrived at these conclusions based on what I have read, worked on and experienced and I am open to learn more. I am certain that this process will also evolve with time and experience. 

An open ended problem mostly manifests itself in two forms - An observation which is not easy to explain, or an abstract set of requirements. The first step, is to translate the abstract requirements or observations into a more concrete problem statement, with a quantifiable end goal if possible. Let us consider the example of a problem driven by requirements - An apartment owner comes to us and asks for our help in reducing the monthly utility bill costs. This requirement is abstract and does not give us direction or tell us how to proceed further. We attempt to understand the problem better by asking some questions - "why is the utility cost at the present value, what contributes to this value and what would you want this value to come down to?" - answering this question will help us understand the major contributors to the current utility cost, and know the target value any proposed solution has to achieve. In this particular case, we see that the water consumption is pretty high for a household of that size. This is still not something concrete and actionable. We dig deeper, and ask the same question again. We soon realise that the person has installed a water purification system that wastes a lot of water. Now, the problem statement is trivial - How can we reduce this wastage of water? 

Sometimes, we may have vague observations. Let us consider an example of a car company trying to understand why a particular model is failing to do well in the market. Several user reviews say "Did not feel good, the pick up is good but overall response is poor". Everyone scratches their heads, what do these reviews even mean? To understand this problem better, we again ask questions - possibly in follow-up interactions with customers. We soon realise that people are unhappy about the fact that the vehicle is understeered. This was a conscious engineering decision, to ensure turning radius is sufficiently high at larger speeds. 

Once we have a proper problem statement, we come up with a set of possible solutions. The more the number of candidate solutions, the better. In the case of the inefficient water purifier, a solution could be to collect this waste water and reuse it elsewhere - bringing down the net consumption. Or, we could simply buy a new purifier which does not waste as much water. For the understeered car, we could reduce the maximum speed and correct the steering. Or, we can market the existing design as a feature. We could also reduce turning radius and maintain the higher speed limit. Any of this is possible. So how do we decide which path to take? There are multiple ways and we can use any combination of these methods. The first method is to see whether this problem has been looked into and study what has been done to solve it. This could be from other products, academic research papers etc. Academicians refer to this as a literature survey. The second method is to theoretically model the solution. A typical model will make efforts to incorporate the effect of various critical variables giving us an idea of whether the proposed solution has the desired outcome. It gives us a mechanism to study how varying each critical variable could impact the final output (models used to study disease spread are good examples). We could also physically model (prototype) the solution and experiment at a smaller scale. This will give us an idea of how easy or difficult it is to implement that solution in the long term. 

As an example, let us say we decide to collect the waste water and use it in the kitchen to reduce consumption and thereby the utility bill cost. We could experiment with this for a month and see whether there is any noticeable change. This may also give us insight into other possible consequences of an approach. For instance, if someone has to move this collected water from the purifier to the kitchen it may not be sustainable in the long term due to ergonomic concerns. However, let us say we are keen on purchasing a new purifier with lesser wastage - we could evaluate the solution by studying what is available in the market. This exercise would also help us understand the impact of this purchase on the utility bill cost - whether the setup cost, maintenance cost and electricity bill negate the monetary gains from the reduced water consumption.  

I like to call the above steps - from formulating the problem to evaluating solutions as the ideation phase. It will always be a messy, iterative process. As we can see above, not every solution is perfect. In some fortunate circumstances there is only one candidate approach that actually meets the primary requirement. Otherwise, there are always trade-offs. In case of the water purifier we have a choice between a solution with possibly poor ergonomics, or new equipment whose setup and maintenance cost could possibly negate gains from reduced water consumption. Ultimately, we may choose to live with the same purifier as that may be the better scenario . The vehicle problem is also an interesting one. The "perfect" solution would heavily depend on how the company wants to position the vehicle in the market. Modelling or prototyping might bring out theoretical flaws or impracticalities of an approach. It may take multiple iterations of the described process to adequately address these issues. 

At the end of the ideation phase, we have a good understanding of what is to be done, why is it that we need to do it, and how to go about doing it. Now, we need to take theory to practise. To give some logical conclusion to our efforts. The outcome of this phase will be the deliverable - maybe a research paper that summarises key results, or a commercial product, or the launch of some scheme. This phase too has its own nuances. Even the best of plans are ruined by poor execution. One must look for feedback even at this stage. For example, choosing to reuse wasted water from an inefficient purifier might not be scalable across a few tens of apartments. Feedback from this stage must be used to immediately make corrections, even go back to the ideation phase again if necessary. It is also a base upon which more work could be done in the future. 

There is a lot that holds this process together. We must have some fundamental knowledge of the "problem-space" if we have to ask the right questions or find answers to questions posed by someone. Fundamentals are indispensable. They are the bedrock upon which we build our skill set. These are simple, powerful ideas that find application across a variety of disciplines. The whole process falls flat without humility, team work, diversity and a spirit of collaboration. There is seldom one right approach and it is impossible for one individual to know everything. In such situations it is important that there is constant exchange of knowledge in a collaborative environment. Different people bring with them their own unique experiences, perspectives and outlook. Their thoughts could be influenced by their culture, learning mechanisms and their understanding of the problem. Understanding their view requires us to be humble and set aside our personal ego. It is important to consider all their inputs and go through this process as one entity. Apart from learning something new, we must also un-learn and re-learn what we learnt wrongly as we go through multiple iterations of the process. 

Everything said and done, the experience can still be a roller coaster of emotions. What is different, is that we have slightly better control of the situation. There are a fewer unknowns and a lesser number of nasty surprises. A significant difference between not having a structured approach and following this process is the amount of learning involved. When we rush through something in the last minute we have not learnt anything. If we follow the iterative process however, you form some opinion that is validated or contradicted by experiments. Either way, there is an "aha!" moment from which we learn something. Problem solving is not just about getting something done. It is about the journey to get there, which is where all the learning is - a journey which still gives me a lot of joy!