Importance of Confidence Intervals (CIs)

In statistics, a confidence interval (CI) is a type of interval estimate (of a population parameter) that is computed from the observed data. Statisticians use a confidence interval to describe the amount of uncertainty associated with a sample estimate of a population parameter.

Market research is about reducing risk. Good research provides information and understanding which allows us to more effectively value our alternatives and make better decisions. When we report results without confidence intervals we are not reducing risk; in fact we may be inadvertently increasing it.

Confidence intervals are a concept that everyone learns in their first stats course but I suspect few truly appreciate their importance. Confidence intervals are about risk. They consider the sample size and the potential variation in the population and give us an estimate of the range in which the real answer lies. Confidence intervals are a bright yellow caution sign telling you to take that sample result with a grain of salt because you can’t be more specific than this range.

Think about the implications of ignoring confidence intervals for a moment. If you only report a number but you can’t be more specific than a range you are over-representing the precision of your results? You are hoping that anyone reading that number will understand it came from a sample and make appropriate allowances for sampling error. But will they?

The science of decision analysis suggests they will not. In fact one of the major challenges in decision analysis is getting decision makers to make a realistic assessment of the uncertainty of their data. The natural tendency is to estimate too narrow a range for confidence intervals. When we underestimate confidence intervals we increase our risk.

The need for accurate confidence intervals has never been more important than it is today. Shrinking market research budgets have meant smaller samples – samples that are often further divided to provide information on segments. Smaller samples mean wider confidence intervals. It is not uncommon to see a confidence interval of +/- 5% on average shares in allocation studies – even with samples of 80 to 100 respondents. Reducing samples from 100 to 50 can increase that confidence interval range by over 40%. If the sample result is a 21% share do you think the decision maker realizes that the actual share could be anywhere from 14% and 28% ?

Do market research reports in your company include confidence intervals? Decision makers, how aware are you about the margin of error when considering the results from a study?

Visit https://www.pars.co.ke or Talk to us on +254 20 8331130/1 +254 716 555388