Machine Learning Lessons Thru Movies ?? Movie Reviews Using SVD !!

Singular Value Decomposition (SVD) is a powerful matrix factorization technique with broad applications in machine learning, particularly in recommendation systems. It allows us to uncover latent relationships between users and items (like movies) by decomposing a user-item interaction matrix.

Let's explore how it works in the context of movie recommendations. ?

Imagine a user-movie rating matrix, where each row represents a movie, and each column represents a user. The entries in the matrix are the ratings given by users to movies. SVD decomposes this matrix (let's call it A) into three matrices: U, Σ, and V?, such that:

A = UΣV?

Here's what each component represents:

? U (Left-singular vectors): The columns of U represent "stereotypical movies" or latent movie themes. These themes are not predefined genres but rather emergent patterns discovered from the rating data. For example, one column of U might represent a "science fiction" theme, while another might represent a "romantic comedy" theme. The values in each column indicate how strongly each movie is associated with that particular theme.

? Σ (Singular values): Σ is a diagonal matrix containing the singular values of A. These values are non-negative and are ordered from largest to smallest. The magnitude of a singular value indicates the "strength" or "importance" of the corresponding movie theme captured in the associated left and right singular vectors. Larger singular values correspond to more dominant themes in the data. ?

? V (Right-singular vectors): The columns of V represent "stereotypical viewers" or user profiles. These profiles capture latent user preferences. For example, one column of V might represent a "science fiction fan" profile, while another might represent a "drama enthusiast" profile. The values in each column show how strongly each user aligns with that particular profile.

How SVD Enables Recommendations:

By analyzing these components, we can:

? Identify user preferences: A user's preferences can be represented as a linear combination of the stereotypical user profiles (columns of V). The weights in this linear combination are determined by the user's ratings.

? Recommend movies: A movie can be represented as a linear combination of the stereotypical movie themes (columns of U). We can then recommend movies to a user based on the similarity between the user's profile and the movie's theme representation. This similarity can be measured using metrics like cosine similarity.

? Reduce dimensionality: By keeping only the top k singular values (and the corresponding singular vectors), we can create a lower-dimensional representation of the data that captures most of the important information. This reduces computational complexity and can improve the performance of recommendation algorithms.

Example (from "Mathematics for Machine Learning"):

The book "Mathematics for Machine Learning" (Deisenroth, Faisal, Ong, 2020) provides a clear example (Example 4.14) where SVD is applied to a small movie rating matrix. They show how the first left-singular vector captures a "science fiction" theme, while the corresponding right-singular vector captures users who tend to rate sci-fi movies highly.

Important Considerations:

This approach relies on the assumptions of consistent user ratings and no noise in the data. In real-world scenarios, these assumptions rarely hold perfectly. However, SVD still provides a valuable framework for understanding user-item interactions and building effective recommendation systems. ?

Related Content

• Spectral Theorem - https://youtu.be/ScQoa8UWFR0
• EVD - https://youtu.be/I9rax7R_jkw
• SVD - https://youtu.be/f7Y_Ihic6tU