Analytic Questions, Part II

A new boss once asked me, “What questions should I be asking of your analysis?”

I said, “I think three questions should be asked on any analytic effort:

• What is the question you answered?
• What was the methodology?
• Is your answer supported by your methodology?”

On ‘the question you answered,’ every analyst knows the importance of getting the question right. Mark Twain is quoted as saying, “If I had three hours to chop down a tree, I’d spend the first two hours sharpening the axe.” But more than getting the question right, it’s critical to understanding why it’s being asked, why it’s being asked now, what decisions are being made, what are the 2nd and 3rd order effects, and what is going to be done with the insights garnered from the study. If you know all this, then you’ve come a long way to answering the question.

Answering the right question is more subtle than it seems. As an instructor at USAFA, often cadets would ask a question as I worked through a math problem. Sometimes I could see that the question they asked on the current step was not where they got confused; they lost it a few steps back. Over their protests I would go back to the previous board and explain again. I could see their eyes light up saying ‘I got it, I got it!’ and they would withdraw their original question. I have seen this happen throughout my career; once in the Tank briefing GOs. one Army GO asked a question and I could tell I lost him on the previous slide…

So, when your boss asks you a question, it may not be exactly what needs to be studied. I will say absolutely, you must always answer the question asked. The boss knows a lot more about why they’re asking the question, perhaps they are asking exactly what they want. This means once you think you’ve got it, go back and ask them. Confer with them often. If it’s not practical to check back, then answer both questions, the one they asked and the one you think they need. I once worked for a two star where 80% of the time I was right in answering the question I thought he should have asked. But sometimes he asked exactly what he needed.

Lastly, the question can change over time. Lots of decisions are temporal given the budget cycle, new information, what your boss’s boss wants to do…? Make sure you check in often with your boss (or someone in the front office that knows what’s going on) so that your insights remain relevant.

So, “What is the question you answered?”

Post your response here on LinkedIn. If you have a longer response, feel free to email me at: [email protected], with subject Analytic Questions, and I’ll summarize responses.