A Brief History of Survey Industry

Insight professionals, stakeholders, and customers have relied on surveys since the dawn of the insights industry. In recent years, the term 'survey' has become synonymous with 'quantitative research.’ The majority of the more developed and used quantitative insights methodologies include surveys, so surveys have become the vital spark of quantitative insights. It has proven to be a wonderful way of understanding the trends and gaining valuable insights.

The first documented census occurred in Babylon in 3800 BCE when the Empire counted livestock, butter, honey, milk, wool, and vegetables. The premise of learning about the lives, needs, and expectations of the public was the same as it is today. This method of data collection evolved into survey research as we know it today in the 1930s and 1960s. Around this time basic survey components along with preliminary tools were designed, and the use of statistical methods for data analysis was gradually developing.

Survey data provides a level of certainty that can be relied upon for the most part when it comes to decision-making and insight generation. It has proven to be a wonderful method of quantitative data collection that leads to conclusive, objective answers. Although the conclusions drawn may be relativistic, it is possible to work with the general direction of consumer trends until a more accurate analysis is available.

There has, of course, been some innovation in survey research since Babylon. In the past decades, insight teams were able to ensure high-quality survey data by shortening surveys, sending them at particular times of day to certain people, and developing incentive models to increase participant engagement. Survey research tactics have been optimized over many years of trial and error, and today's successful survey research largely relies on them.

However, with the ever-growing population, the amount of data produced and needs are both exponentially increasing. Stakeholders are now struggling to collect good-quality data. According to a study from the Harvard Business Review, only 3% of companies’ data meets basic quality standards. This essentially means that businesses are wasting their resources without any gains or meaningful insights.

The data collected through a survey do not show why participants acted or responded as they did - logical data provided logical answers that pointed the way to success. Generally, survey data does not provide a deep understanding of the participant's experience in the particular topic. There is a text box in NPS surveys for participants to provide more detail, but how many people actually use it? Getting a full picture of the experience requires more than asking follow-up questions. To add to this, there is a lack of incentive for the respondents to give truthful and meaningful answers. Surveys often only benefit the people who are conducting them and mean little to the people who are taking them.

The survey industry has recently started to consider qualitative and behavioral science principles when designing quantitative tools in order to better understand the reasoning behind the surface answers. Technologies like Machine Learning (ML) allow better prediction and analysis of data models. This enables businesses to identify trends and transform them into quantifiable actions.

The prediction made by ML model gets more and more accurate as it collects more data. It provides better quality data by enabling specific customization and categorization at a lower cost. These new trends are gradually transforming the survey industry and providing better value for data.

ML also reduces the workload by automating tasks like open text responses analysis and deduces valuable insights from it. This used to be such a tedious job that few survey analysts were really paying attention to it.

Tools like our own Augmented Survey are bringing this approach a huge step forward. With Augmented Survey we are solving three key issues of the current survey world:

1. Provide incentives for the respondents to provide honest and truthful answers.

2. Reduce survey fatigue by reducing the redundant questions.

3. Maintain data integrity and retain data ownership by using decentralized technologies.

As decentralization is dawning into the new world, we are bringing the latest technological advancements into the old and rust survey world to slowly revolutionize the industry.