How is artificial intelligence disrupting market research?

AI is just the tool; it’s not the craftsman. Relying solely on AI for data is like trusting a robot to paint a masterpiece. Sure, it can mimic patterns, but it still lacks soul. We should keep mixing traditional human-driven research. Our competitor's AI can gather data, but our human touch can find the narrative that resonates. Data doesn’t convert; stories do. We should keep it real.

Market researchers can collect data from social media, online reviews, and sensors with AI. NLP and machine vision harvest information, analyse customer sentiments, and engage with customers via chatbots and voice assistants.

For everyone hoping to “streamline data collection” oftentimes meaning skipping data collection altogether, be aware that without continuous input of new, fresh, clean data the models behind AI will degrade and collapse at worse or become irrelevant pretty soon at best. Companies should invest in quality data collection before putting their money on black boxes regurgitating summaries based on low quality data.

Rather than disrupting Market Research, AI is elevating it as an industry. From data collection to reporting, AI is driving time and cost efficiencies. Quantitative surveys are faster to deploy, Qualitative content analysis & instrument design is being informed by AI, AI-enabled Social Media Analytics is helping researchers access, mine, cluster millions of unsolicited, first-party data-points, for rich insights.

The only real effect of AI on market research I have seen to date is its text analytics and text generation capabilities. These capabilities, as impressive as they are, apply to the very last stages of market research. I have not seen any application of AI to the crucial planning and execution stages.

AI uses machine learning and deep learning to find patterns, trends, and correlations in big data sets, improving segmentation, pricing optimisation, product testing, and brand awareness assessment.

Sticking to qual, just getting transcripts from your groups or interviews could take weeks or days just a couple of years ago and now, you have your raw data and a summary to start within minutes of an interview's close. The AI advantage snowballs from there and you can scale qual data collection operations because your data analysis volume problem has been resolved. The AI-enabled tools for mrx, cx, ux data analysis have access keep getting better, especially wrt suggestive paths. But just like GPS - the driver has to remember who's in charge or you will end up driving off a strategic cliff.

As others have mentioned, the ability to get transcripts in minutes is a huge and welcome change. Additionally, AI tools give our curiosity some breathing room. The ability to quickly run a hunch through AI means we don't need to spend an hour or more sifting through notes or discussion guides just to explore an idea. AI enhances efficiency, allowing us to focus more on insights and less on the mechanics of data processing. This is a game-changer for our industry. #AI #MarketResearch #Innovation

AI can do a lot, but’s think the most important is to make complex tools more accessible to users. Not every SMB user will use AI to increase market research complexity, but he can use existing information easier and in a more user-friendly way. Just imagine the local store manager asking his AI chatbot: tell me what you noticed in yesterday sales and what shall i do about this today?

AI can also be a great tool for coding unstructured, open-ended data, efficiently. In my experience, researchers always tried to avoid including too many open-ended questions in quantitative survey design because of how expensive and long it takes to code the answers. This additional research tool can now make it possible to collect higher level of qualitative, open-ended responses, in a quantitative way.

AI presents data in charts, graphs, maps, dashboards, and tales and uses AR and VR to immerse market researchers and clients.

Yes! Even "easy" tools like ChatGPT, with its Advanced Data Analysis, make generating quick visualizations for hidden insights much more straightforward.

How I normally tell when an insights person doesn't know how to tell a good story is when he/she uses tools to show it (e.g. dashboard bar charts with no notifications). Where humans excel (for now) and AI falls short is the ability to visualize a powerful human story. AI can quickly provide the first draft of a particular report, but eventually the human has to fine tune the information to create powerful messaging to his/her stakeholders.

In the field of data visualisation, I can think of two main contributions of AI: 1) visual representation of the distribution of large amounts of data can provide visual support to decision makers or even researchers, and 2) AI can provide the best visual solution for the presentation of specific results.

With AI tool we use, which is based in NLP organizing unstructured open text, we get clear maps of meaning, drivers, organized response volume by topic, sentiment and other interesting insights. Data set up takes most time, but the actual visual outputs take just a few minutes.

AI confronts market research ethics in data privacy, security, quality, and bias. According to GDPR and CCPA, researchers must acquire and utilise data clearly, ethically, and respectfully without bias or discrimination.

AI has definitely changed the way we have collected data assimilating information from many sources within minutes. I believe there is a still need a long way to go to generate insights and perspectives where in a human intelligence is required. I always say that Artificial Intelligence is always Artificial, Original Intelligence always lies with us.

As AI starts collecting more and more data (we use ai to carry out 60+ minute interviews on my platform researchGOAT.com for instance). It's super important to be upfront about the use of AI in research processes. Why? Because trust and transparency are the bedrock of any meaningful research. When participants know they're interacting with an AI, it sets the stage for authenticity and informed consent. They have the right to know who (or what) they're sharing their insights with, and this knowledge can actually shape their responses and the quality of data collected.

As far as using LLMs to analyze data - both structured and unstructured - there’s lots of error right now. I’d encourage researchers to apply AI to query data in new ways, while also knowing how to write a structured query to validate. Find errors, and keep training your models. As for bias, market research has been pushed so hard on speed and cost over time, and self-serve access for unqualified researchers, that the current state is far from perfect. I’d like to be optimistic that time savings in reporting and scaled coding of unstructured data may enable more care in study design and more access to perspectives of hard-to-sample “respondents” via open-internet data analysis. It’s good news that we’re talking so much about bias!

Training data, proprietary data, synthetic data, convenient data are all ways of saying "WATCH OUT! HUMANS AHEAD!" In addition to racist and misogynistic, tendencies. LLM must be "fed" something in order to obtain answers. Sure, it can do a lit review without any trouble, but your clients' data are not in the LLM already. You must put them there. And, you must decide whether all of our ethics and guidelines are represented.

Data skills are one of the things I worry about us, as an industry, losing over time due to AI. As we automate a lot of data elements, we run the risk that newer people in the industry won't learn these skills. It may not have an impact as the advancements in AI could make this completely obsolete, but I do want to make sure we don't lose some of the core skills we need.

The ease of AI has exposed the disconnect in what market researchers considered value added services in this industry. The industry’s focus on representative sample, data quality, data cleaning can all be automated and/or can be supplemented with synthetic data. This disruption is going to create a tension within the industry among those who will advocate and those who will actively resist these changes. The bottom line is that these disruptions will evolve and solve all the objections that are currently being raised. It’s only a matter of time. For those who are in the industry - the focus needs to be on what technical skills will help them stay competitive. New use cases will emerge and this will create change.

It will be a learning curve, but researchers will need to develop skills in data literacy to effectively interpret and communicate AI driven insights, as well as analytical thinking to critically evaluate the outputs generated by AI, but additionally storytelling is essential to translate complex data into compelling narratives and collaboration skills are crucial for working alongside a variety of experts. Proficiency in storytelling is crucial because it enables researches to transform complex data insights into engaging, and relatable narratives, ensuring that stakeholders can easily understand and act upon the information.

While some speak of completely transforming the industry with AI, the reality is that there will be a lot of incremental improvements before true transformation happens in most places. Skill building should be a key part of every professional career plan. Also, simply saying AI cannot exhibit creativity, curiosity, or empathy misses the point - people also struggle in these areas sometimes. There are plenty of examples of AI being more creative, and exhibiting more empathy than humans, Matching the right people, the right tech, the right AI and right approach with the right problem is a key skill - it is not new per-se, but AI does create a new realm.

AI will change the future of work and the worker of tomorrow by making information better understood, and readily available to key communities that will be required to assess AI. Along the way we need to make sure we upskill, reskill, and train, the right people to be as creative as possible with AI. If doen effectively, AI and the worker of tomorrow will accelerate data, analysis, image/video creation, projects, and a host of other things with the right understanding and empathy required. Knowledge is power, it's all about the delivery and how the next data scientists, engineers, plant floor professionals, designers, and everyone else collaborate in a seamless way leveraging all that AI has to offer.

I think clients of MR are being lead to believe that AI is a magical tool that can provide precise answer to complex problems, based on a simple question. Not so. Consider, for instance, the question 'Give me a model that can calculate the probability that a convenience store will be profitable in an specific location?' Many other problems that I currently help my clients solve come to mind.

AI fuels data innovation, enabling market researchers to uncover fresh insights, experiment more efficiently, and personalize offerings to meet customer expectations. This innovation not only sets businesses apart but also ensures they remain agile and competitive in the ever-evolving market research landscape.

Segmentation and Targeting: AI excels in segmenting consumers based on various behavioral and demographic factors, enabling more targeted and effective market research. Chatbots for Data Collection: AI-powered chatbots can interact with consumers to collect data in a conversational manner, making the process more engaging and less intrusive.

How about a real world example? I recently used Gen AI to help me write VBA code that would let me distill tens of thousands of comments to just those related to my topic of interest and any number related keywords or phrases I want. Then I used Gen AI to give me very specific targeted actions I could take to solve the issues raised in distilled data set. That would have taken me 20-30h before, and now I can do it in 5 minutes with super detailed and valid results with any data set and any topic I choose from here on out! And since I learned how to do this myself, I could recreate this for any employer or client I wanted if I so chose.

The title of the article presumes that AI is disrupting market research. I am not convinced yet that is the case. Currently, generative AI has shown its amazing ability to understand, analyze, and generate language and even images. That is truly impressive. Beyond that I have not seen example of AI "revolutionizing" market research in all its complexity - panning, data collection, data clean-up, data analysis (except text analytics), and insights generation and socialization. I believe that AI has the potential to help in all those areas as well, just waiting to see real life examples beyond all the hype :)

AI makes many tasks more accessible, raising the ante for researchers. I anticipate good researchers will move towards experimentation and specific application to businesses, with less focus on general findings about a category or a topic. Researchers recently used AI to create a perceptual map of car brands, with 90% accuracy for a paid market research study— suggesting that research around general population topics and broad understanding will likely be replaced by AI, while custom audiences, B2B research and recommendations for specific business situation will be where researchers will add the most value in the future.

As we dive deeper into the exciting world of AI in research, there's one skill that's becoming increasingly important: crafting effective research prompts for AI. This might seem like a small detail, but trust me, it's a big deal. Creating the right prompts for AI isn't just about feeding it information; it's an art and a science. The way we frame questions and prompts can significantly influence the kind of data and insights we get back. It's like having a conversation – the quality of your questions often determines the depth and usefulness of the answers you receive. This skill is more than just technical know-how. It involves understanding the nuances of language, the objectives of the research, and even a bit of psychology.

When working with customer data, market researchers need to prioritize privacy and security above all else. To this end, be cautious when using free AI tools. Don't submit PII or proprietary data in those tools without doing your due diligence. If you're paying for a market research platform that has new AI capabilities, make sure you talk to their security and compliance team to understand how they handle data when feeding their AI algorithms.