BIG DATA RECOMMENDATION SYSTEMS

What are Recommendation Systems in Big Data?

The recommendation system provides the facility to understand a user’s taste and find new, desirable content for them?automatically. It is capable of predicting the future preference of a set of items for a user and recommending the top items.

There are many different ways to build recommender systems, some use algorithmic and formulaic approaches like Page Rank while others use more modeling-centric approaches like collaborative filtering, content-based, link prediction, etc. All of these approaches can vary in complexity, but complexity does not translate to “good” performance. Often simple solutions and implementations yield the strongest results.?

Types of Recommendation Systems in Big data :

Three major types of recommender systems:

Content-based filtering.

Collaborative filtering.

Hybrid recommender systems.

1. Content-based filtering: It uses characteristic information.

Content-based filtering works based on the relevant items shown using the content of the previously searched items by the users. Here, content refers to the attribute/tag of the product that the user like. In this type of system, products are tagged using certain keywords, then the system tries to understand what the user wants, looks in its database, and finally tries to recommend different products that the user wants

An example of the movie recommendation system where every movie is associated with its genre, which in the above case is referred to as tag/attributes. Now let's assume user A comes and initially the system doesn’t have any data about user A. so initially, the system tries to recommend the popular movies to the users or the system tries to get some information about the user by getting a form filled by the user. After some time, users might have given a rating to some of the movies, like it gives a good rating to movies based on the action genre and a bad rating to the movies based on the anime genre. So here system recommends action movies to the users. But here you can’t say that the user dislikes animation movies because maybe the user dislikes that movie due to some other reason like acting or story but actually likes animation movies and needs more data in this case.

2. Collaborative filtering: It is based on user-item interactions.

The process of predicting the interests of a user by identifying preferences and information from many users. This is done by filtering data for information or patterns using techniques involving collaboration among multiple agents, data sources, etc.

Collaborative Filtering algorithms are two types:

• User-User based: This algorithm first finds the similarity score between different users. Based on this similarity score, it then picks out the most similar users among the pool and recommends products that these similar users have liked or bought previously.

• Item-Item-based: This algorithm looks for items that are similar to the items that the user has already watched/rated and recommends the most similar items. It might look similar to a content-based recommendation system, but it is different. Let us understand item-item similarity clearly. It doesn't matter if two items are the same by attributes, such as two movies under the comedy genre. Instead, what similarity means here is how people treat two items the same in terms of watch/rating. So, a user might like (gave a higher rating) two different movies from two different genres, such as comedy and action. So, these two different movies have a higher item-item similarity.

3. Hybrid recommender systems: It combines the above two recommendation systems.

The hybrid filtering approach can be implemented in two ways. One is both content-based and Collaborative filtering are applied separately and then combines the result as per need. Second, first, we apply collaborative filtering and then apply content-based filtering to the result.

Recommendation systems are widely used in a variety of applications for recommending products or items to the user.

Types of Data for Generating a Recommendation System :

To generate a recommendation system we have divided the data into two types :

• Explicit Feedback:?The data contains the user’s explicit feedback.It can be a kind of rating from the user to the item which tells about the status of the user whether he liked the product or not.
• Implicit Feedback:?This data is not about the rating or score which is provided by the user, it can be some information that can inform about clicks, watching movies, playing songs, etc.

Losses used by Recommendation Systems :

?We have two different loss approaches :

• Bayesian Personalised Ranking(BPR)?pairwise loss: It is used when the positive interaction from the user on the data is presented and we are required to optimize the ROC(Receiver Operating Characteristic), AUC(Area Under the Curve). By using the pairwise loss we try to maximize the prediction difference between positive feedback and a randomly selected negative feedback.
• Weighted Approximate-Rank Pairwise(WARP) loss: It is useful when the positive interaction is available in the feedback and we are required to optimize some top recommendations. Here it repeatedly samples the negative feedback until it finds the one feedback which is violating the rank and this procedure maximizes the rank of positive feedback.?

THANK YOU !