登录查看更多内容

What are some best practices for incorporating qualitative data into machine learning for market forecasting?

由人工智能和领英社区提供技术支持

Qualitative data, such as customer feedback, reviews, interviews, or social media posts, can provide valuable insights into the preferences, motivations, and behaviors of your target market. However, incorporating qualitative data into machine learning for market forecasting can be challenging, as it requires transforming unstructured and subjective information into numerical and objective features. In this article, you will learn some best practices for collecting, processing, and analyzing qualitative data to enhance your machine learning models and improve your market forecasting accuracy and reliability.

此文章中的业界达人

由社区从 5 条内容中精选。了解更多

J. Benjamin Greisman

Senior Associate at The Strategic Counsel

1 Choose relevant sources

The first step is to identify and select the most relevant sources of qualitative data for your market forecasting goals. You need to consider the quality, quantity, and diversity of the data, as well as the availability and accessibility of the sources. For example, if you want to forecast the demand for a new product, you might want to collect data from online reviews, surveys, or focus groups of potential customers, rather than from general news articles or blogs. You also need to ensure that the data is representative of your target market segments, and that you have enough data to train and test your machine learning models.

添加您的观点

Gijsbertus J.J. van Wulfen

Innovation keynote speaker, Number One Thought Leader Design Thinking 2024, LinkedIn Top Voice helping you and your organisation, to become amazing innovators with keynotes, workshops, and a proven innovation method.
举报内容
In my experience as innovation facilitator of 100+ projects, the best way for customer discovery is: “GO OUT AND TALK TO THEM YOURSELVES”! 4 concrete tips for you: 1. Immerse as an anthropologist among your potential customers without being judgmental. 2. Find out what really relevant for them in their lives at this moment. 3. Identify their jobs-to-be-done and issues related to the context of your scope. 4. Do customer discovery before your ideation process YOURSELVES with your whole tam, as the experience is as important as the results. When as immerse in your target group yourself you will understand their situation, and their jobs to be done more profoundly than any other method. You will never forget your customer vists ?????.

已翻译

赞
Vinay Kumar Chaganti

The Extreme Generalist
举报内容
Source is different from method. If you want to forecast demand, your source of data are the people in the supply chain - suppliers through to customers. Online surveys, focus groups etc are methods of gathering relevant qualitative or quantitative data. First - choose the source well. It adds to the validity of your findings. Then - choose an appropriate method of data collection. It adds credibility to your research design.

已翻译

赞

2 Preprocess and label the data

The next step is to preprocess and label the data to make it suitable for machine learning. Preprocessing involves cleaning, filtering, and standardizing the data to remove noise, errors, duplicates, or irrelevant information. For example, you might want to remove punctuation, stopwords, or low-frequency words from text data, or apply stemming or lemmatization to reduce words to their base forms. Labeling involves assigning numerical values or categories to the data based on predefined criteria or rules. For example, you might want to label text data as positive, negative, or neutral based on the sentiment or tone of the content.

添加您的观点

J. Benjamin Greisman

Senior Associate at The Strategic Counsel
举报内容
Standardization of language, or, taxonomy, should ideally be done in the manner in which an interconnected digital dictionary and translator is built, for one is very likely to encounter multi-lingual research projects.

已翻译

赞

3 Extract and select features

The third step is to extract and select features from the data that capture the most relevant and useful information for your machine learning models. Features are the attributes or variables that describe the data and influence the outcome of the forecasting task. For example, you might want to extract features such as word frequency, word length, word order, or n-grams from text data, or use techniques such as topic modeling, word embedding, or sentiment analysis to create more complex features. You also need to select the most important and relevant features for your models, and avoid features that are redundant, noisy, or irrelevant. You can use methods such as correlation analysis, feature importance, or feature selection algorithms to help you with this step.

添加您的观点

Sayan Khastagir

Head - Research & Insights at Share Creative | Ex Prime Video & Amazon | ITC | Kantar
举报内容
It is harder to include traditional qualitative data into ML training sets for forecast. There could be survey-based/VOC datasets where text-encoding features can be applied and included in training sets. However, this would need a dedicated QA process to review the NLP-driven outcomes. In addition, there are some new feature extraction techniques that have come out for image, audio, and video data which could be included with descriptive features. Finally, sentiment analysis tools can classify open-ended feedback and might be used as outcome predictors. However, the quality of qualitative data that the ML models accommodate is very broad and minimal compared to the true potential of qualitative research.

已翻译

赞

4 Train and evaluate models

The final step is to train and evaluate your machine learning models using the features extracted from the qualitative data. You need to choose the appropriate models and algorithms for your forecasting task, such as regression, classification, clustering, or time series analysis. You also need to tune the parameters and hyperparameters of your models to optimize their performance and avoid overfitting or underfitting. You can use methods such as cross-validation, grid search, or random search to help you with this step. Finally, you need to evaluate your models using appropriate metrics and criteria, such as accuracy, precision, recall, F1-score, mean squared error, or mean absolute error, and compare them with baseline models or benchmarks.

添加您的观点

J. Benjamin Greisman

Senior Associate at The Strategic Counsel
(已编辑)
举报内容
A continuous feedback loop as one inputs new waves of data (for example, if you are surveying people in real-time on satisfaction - their why, or, weekly, monthly, etc) to ensure that the model fits, is helpful. It’s important to mark the point in time when important events occur that may generate new categories of qualitative data, or changes to the model, such as recessions, new stores opening, new product features, new medical treatments, etc. Those are typical catalysts to changing qualitative sentiment vis a vis your research subject.

已翻译

赞

5 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?

添加您的观点

Market Research

+ 关注

给文章评分

我们借助人工智能创建了此文章。您认为这篇文章怎么样？

很棒不太好

举报此文章

查看全部

What are some best practices for incorporating qualitative data into machine learning for market forecasting?

1

2

3

4

5

1 Choose relevant sources

2 Preprocess and label the data

3 Extract and select features

4 Train and evaluate models

5 Here’s what else to consider

Market Research

给文章评分

感谢您的反馈

更多Market Research相关文章

更多相关阅读内容

What are some best practices for incorporating qualitative data into machine learning for market forecasting?

1

2

3

4

5

1 Choose relevant sources

2 Preprocess and label the data

3 Extract and select features

4 Train and evaluate models

5 Here’s what else to consider

Market Research

给文章评分

感谢您的反馈

查看其他技能