You want to improve your recommendation engine. What algorithms should you use?

1 Collaborative filtering

Collaborative filtering is a family of algorithms that use the ratings or feedback of other users to recommend items to a target user. The idea is that users who have similar preferences or tastes will like similar items. For example, if you enjoyed a movie that many other users also liked, you are more likely to enjoy other movies that those users liked. Collaborative filtering can be divided into two main types: user-based and item-based. User-based collaborative filtering finds similar users to the target user, and recommends items that they rated highly. Item-based collaborative filtering finds similar items to the ones that the target user rated highly, and recommends them. Collaborative filtering algorithms are simple, intuitive, and scalable, but they also have some drawbacks, such as data sparsity, cold start, and popularity bias.

3 Hybrid filtering

Hybrid filtering is a combination of collaborative filtering and content-based filtering algorithms, that tries to leverage the strengths and overcome the weaknesses of both approaches. The idea is that using different sources of information or methods of recommendation can improve the quality and diversity of the suggestions. For example, a hybrid filtering algorithm can use content-based filtering to generate a candidate set of items, and then use collaborative filtering to rank them according to the user's preferences. Or, it can use collaborative filtering to generate a candidate set of items, and then use content-based filtering to filter out irrelevant or redundant items. Hybrid filtering algorithms can be implemented in various ways, such as weighted, mixed, switching, or feature combination. Hybrid filtering algorithms are more complex, adaptive, and effective than single-method algorithms, but they also require more data, computation, and tuning.

4 Matrix factorization

Matrix factorization is a technique that reduces a large and sparse matrix of user-item ratings into two smaller and dense matrices of user features and item features. The idea is that the ratings can be approximated by the product of the user features and the item features, and that the features capture the latent factors that influence the user's preferences and the item's characteristics. For example, a matrix factorization algorithm can discover that some users like action movies, and some movies have a high action factor, and then use those features to predict the ratings. Matrix factorization algorithms can be seen as a special case of collaborative filtering, but they have some advantages, such as dimensionality reduction, noise reduction, and generalization. Matrix factorization algorithms are based on optimization methods, such as gradient descent or alternating least squares, and they can incorporate regularization, bias, or context.

5 Deep learning

Deep learning is a branch of machine learning that uses neural networks to learn complex and nonlinear patterns from large and heterogeneous data. The idea is that neural networks can extract high-level and abstract features from the data, and use them to make predictions or recommendations. For example, a deep learning algorithm can use convolutional neural networks to process images or videos, recurrent neural networks to process texts or sequences, or attention mechanisms to capture the relevance or importance of the inputs. Deep learning algorithms can be applied to any of the previous methods of recommendation, such as collaborative filtering, content-based filtering, hybrid filtering, or matrix factorization, and enhance their performance and flexibility. Deep learning algorithms are powerful, expressive, and innovative, but they also require a lot of data, computation, and expertise.

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?