A Board of Directors Guide on how to get High Quality Insights from Business Surveys in the private, public and not-for-profit sectors

Image Credit

PS: This is genuine intellectual property: the "Board Effectiveness Guru" has no need for artificial intelligence! All Rights Reserved.

1.??????? Beware: Risks are Everywhere, even in the statistics of business research!

Sometimes, good business research, such as a customer satisfaction survey or an employee engagement study, may need be conducted in two steps. This is in line with best business science practice. Any statistical investigation will involve uncertainties. So, the risks of making errors in the findings of a survey are real and must be taken seriously.

Directors must be aware of these risks; to check whether management has adequately reduced the potential threat to the quality of business information coming to the Board of Directors. Garbage in, Garbage Out!

?2. ?????? Perception or Opinion: What does your business need?

Let us take the example of a customer (commercial sector) or beneficiary (public or not-for-profit sectors) satisfaction survey.

?First, it is important to work out whether your business needs a “opinion” survey or a “perception” survey.

?An “opinion” survey can be defined as the sort of work that I have annually directed for the World Economic Forum (WEF) in certain countries since 2012. ?In those countries, we have administered the Global Executive Opinion Survey for the WEF using a uniform questionnaire prepared by the WEF. The WEF has its technical indicators and wishes to compare across countries. So, it seeks the opinion of respondents on its own (WEF) perceptions of what is important to global competitiveness and risks.

However, a major telecommunications operator wanted information about what its customers and stakeholders independently perceive about its business. So, it ordered a “perception” survey of those stakeholders to allow it to get a unique picture of their views and for it to respond accordingly.

?It wanted to know how stakeholders view its business from their own experiences, warts and all.

This objective cannot be readily met by an “opinion” survey with questions created by a consultant or by business management. They require a more complex “perception” survey with questions coming from preliminary research into the target populations, as outlined below.

3.??????? Why Two-Steps?

In?statistics, a?Type I error?is a false positive conclusion e.g. the risk that we state that the survey population has a particular view X, when in fact it does not. A?Type II error?is a false negative conclusion e.g. the risk that we state that our survey population does not have a certain view Y, when in fact they do. Clearly, these errors will have significantly adverse impacts on your business decisions as resources will be wrongly applied and the expected results may not be attained.

Bad information leads to Bad decisions and Bad results!

These risks can be minimised through careful planning in your study design. A two-step approach is one such tool of statistical risk management intended to reduce the likelihood of Type 1 and Type 2 errors in a “perception” survey.

Instead of the business researcher guessing at the views of the population to develop questions, we investigate the population to obtain real views. Then, with relevant professional tools, we use those real views as the basis of a hypothesis about the views of the entire population (see Box 1).

Finally, we develop survey questions that have a better likelihood of avoiding Type 1 and Type 2 errors.

Figure 2 illustrates that survey questions developed from guesswork run a higher risk of Type 1 and Type 2 errors than questions that are grounded in the real perceptions of the targeted population.

4.??????? So, what are the Two Steps?

Step 1:

This will use qualitative research tools (key informant interviews, focus group discussions etc) to investigate the views of a careful selection of stakeholders from the targeted population on the key issues being addressed by the business research. It will not be a representative sample of the population. But it will seek the views of a carefully selected number of stakeholders. The outcome of this research will be a rich, in-depth insight into the real perceptions of these stakeholders (which will be in the central circle of Figure 2). We then use statistical analysis of the content of the interviews to investigate and classify the results of this insight.

Basically, we turn truckloads of words into small numbers of useful statistical data!

This classification will give an indication (see Box 2) of the possible views of the entire population and will form a hypothesis of the representative views of the full population. This hypothesis will be tested in Step 2.

Step 2:

This will test the hypothesis of the representative views that was developed in Step 1. It will start with the development of survey questions from the classifications of views that were developed in Step 1. Then the questions will be administered to a representative sample of the total population and “definitive” (see Box 2) conclusions drawn that will indicate the extent to which the hypothesis is true or false. This conclusion will inform the survey report and the implications for management decision making and control of the downstream business activities.

5.??????? Recap: Getting reliable and valid information to the Board for business direction and control

The two steps are outlined in Figure 3, which shows their contribution to a third step, which is the use by the Board of Directors and management of the results in decision making, direction and control of the business. ?In that Figure, S1 refers to Step 1; S2 to Step 2 and S3 to Step 3.

To recap, the Board of Directors needs to check on the following when receiving any business research from management that positions itself as a survey of a key element of the business:

1.?????? Is it an “opinion” or a “perception” survey?

2.?????? If an “opinion” survey:

2.1.?? How does the Board know that the questions being asked will provide valid and reliable insights of the business activity?

2.2.?? Is there independent and credible third-party business literature to support the questions?

3.?????? If a “perception” survey:

3.1.?? What steps have been taken to reduce the high risks of statistical errors in the questions being asked?

3.2.?? Have the questions been developed through guesswork or through a scientifically based two-step process?

Best wishes for reliable survey information for Board decision making, direction and control in the private, public and not-for-profit sectors!

? Dr Omodele Jones, “Board Effectiveness Guru”, 2024. All Rights Reserved.

PS: This is genuine intellectual property: Dr Jones has no need for artificial intelligence!