Personalized Vendor Recommendations with Semantic Search on The Knot

Written by: Christopher Bronner , Senior Machine Learning Scientist

Business Context: The Knot’s Marketplace and Style Quiz

The Knot is a leading wedding Vendor Marketplace in the U.S. that connects more than 1 million couples with over 200,000 local wedding professionals, such as reception venues, florists, and photographers. We offer a comprehensive suite of planning tools that includes personalized wedding websites, invitations and registry services that make planning easier.

Wedding Vendor Marketplace

The Knot Vendor Marketplace is a central component of the site and we monetize this product through paid subscriptions that allow vendors to purchase listings in more prominent parts of our site.

From the vendor perspective, the value of our Marketplace product is rooted in the leads they receive from couples who can contact vendors directly on our site. Therefore, the rate of couples who submit a lead to a vendor (i.e. conversion to lead submission) is one of our most important metrics to assess the performance of our product.

Conversion to lead submission, in turn, depends on our ability to recommend to couples vendors that are relevant to their personal preferences, including their taste and style. Improving the vendor recommendations can be assumed to increase conversion to lead submission.

Personalization via Style Quiz

In order to make high-quality, personalized recommendations that speak to the couple’s individual needs, we first need information about their taste and style. This is where our “Style Quiz” comes in, which is one of our most popular planning tools.

In the Style Quiz, couples rate images from four galleries (reception venues, table settings, bouquets and dessert decorations) and based on this information, we categorize their style by showing a result consisting of a set of primary and secondary style terms. For example, we might categorize a given couple as having primarily a “classic” style, but also leaning toward “elegant” and “traditional” (secondary styles). In addition to the primary and secondary style, we determine the ideal setting for their wedding—for example, a hotel, loft or barn.

These results give valuable insight into the couple’s individual style preferences, especially for their reception venue. We therefore set out to leverage this information to improve the quality of our vendor recommendations (specifically for reception venues) for our couples. This in turn should lead to an increase in conversion to lead submission which benefits our subscribing vendors.

Methodology: Semantic Search based on Style Quiz Results

Our goal is to identify the venues that are the best match for a couple, based on the results they got from our Style Quiz.

To that end, we applied a semantic search approach that finds venues whose description texts best match the couple’s style terms. More specifically, we represent venues and style terms as embeddings, quantify their semantic similarity with a distance measure in the embedding space, and recommend venues similar to the couple’s individual style.

Semantic Search

The overall concept is to represent venues on the one hand, and the couple’s style terms on the other hand as embeddings (high-dimensional numerical vectors) that encode the semantic meaning of the respective texts, and use a distance metric in the embedding space to quantify the semantic similarity of any pair of venue and style. Recommendations are then made by selecting venues with the highest similarity score.

We started by creating embeddings for the description texts of all venues in our Marketplace. In addition to the descriptions which are written by the vendors themselves, we also included textual meta-information in the embeddings such as amenities provided by the vendor on their storefront (e.g. that they offer a dressing room or wireless internet).

The next step is to encode a couple’s individual style in the same embedding space as the venue descriptions. We do this by creating embeddings for all style terms generated by our Style Quiz—meaning the primary style term, secondary style terms, and setting terms. We then determine an overall style embedding as the linear combination of these embeddings with the coefficients being weights that represent the importance of each of the style terms. Putting it differently, the overall style is the center of gravity of the different styles in the embedding space.

Having represented the venues and the couple’s style with embeddings, it’s straightforward to make venue recommendations for a couple: after filtering venues according to a few rules that implement our business logic (e.g. filter for venues in a specific geographic area and considering whether venues are paid or unpaid, or whether the storefront includes a photo), we simply recommend the venues closest to the couple’s overall style in the embedding space.?

The distance between two embeddings is calculated as the inner product of the two embedding vectors, which is a good approximation to Euclidean distance in a high-dimensional space, but less computationally expensive.

Match Score

Our semantic search approach for venue recommendations intrinsically provides us with a quantitative measure for the quality of each recommended venue, namely the distance between the venue and the couple’s overall style in the embedding space.?

We surface the distance to couples, albeit after inverting and scaling it. The resulting “match score” is shown next to each recommended venue as “94% match,” for example. We hypothesize that this match score highlights to couples that the recommendation is personalized, ultimately increasing conversion to lead submission.

LLM-Generated Recommendation Reasonings

To further highlight to couples the fact that the recommendations are personalized and based on their individual style quiz results, we added LLM-generated copy that provides a reasoning for why a specific recommended venue matches the couple’s style. The screenshot below shows a tested variation of our venue recommendations that features the reasoning copy.

Experimentation and Results

Measuring the performance of a style-based recommendation system is difficult because whether a recommended venue matches a specific style is highly subjective. Ultimately, the only reliable way to assess the impact of our tactics is through A/B tests.?

We conducted different A/B tests for the different tactics mentioned above to quantitatively attribute the impact to each change separately.

Test of Semantic Search-Based Venue Recommendations

We exposed 50% of couples with an impression of the Style Quiz results page to the new semantic search-based recommendations. At the same time, the remaining 50% served as a control group and were given recommendations based on an older methodology that didn’t take the style quiz result into account.

The couples in the variant group that were shown the semantic search-based recommendations had a significantly higher conversion rate to lead submission than the couples in the control group. By using semantic search, we achieved a 5.9% lift in this metric.

Test of Recommendation Reasonings

We conducted a second, multivariate test in which one variant featured an updated design of the Style Quiz results page that included the recommendation reasonings among other design updates. This variation led to a lift in conversion to lead submission of 4.1% but was ultimately not rolled out for different considerations.

Test of Match Score

A different variation in the same multivariate test included the match score. The addition of the match score alone led to a statistically significant 4.2% increase in conversion to lead submission. Together with the 5.9% lift from the semantic search-based venue recommendation, this means an overall lift of 10.3%.

Conclusion

The quality of product recommendations depends on their relevance to the preferences of the individual customer. By basing our recommendations on our couples’ choices from the Style Quiz, we can suggest products in a highly personalized manner that we wouldn’t be able to with just behavioral analytics, especially for new members immediately after sign-up.

We find that instead of just serving recommendations by themselves, it adds value for our customers to have additional data points that explain why the recommendations are a good fit for them, namely the match score and the reasoning copy.