Temporal Fashion Recommender
Selected Spring, Summer, Autumn, Winter Outfits from DeepFashion, open-source by Liu Z. et al.

Temporal Fashion Recommender

Building a Recommender That Evolves with Seasons

One of the biggest challenges in the design of any recommender system (RecSys) is in handling temporal shifts. Since the world of fashion evolves with time, recommenders we design must adjust to the changing tides as well.

In this article, we will consider how we can factor in the seasonal outfit subtleties into our Convolutional Neural Network (CNN) model using temporal weights.

The Business Problem

Our imaginary eCommerce company, HappyPanda Co., is expanding globally and will like to launch Seasonal Collections. The product requirements are:

  • Given collections of outfits, how can we adjust them to suit the various seasons across different countries? Note that each country will have different fashion sense and weather conditions for different seasons.

The Data

From the deep fashion data, I select outfits suitable for each of the four seasons. For example, in winter, most people will don on a heavier jacket for warmth. On the other hand, spring and summer is the time to enjoy some time under the sun with lighter clothes. Temperature thus plays a significant role in influencing what people wear. For instance, if we consider the average temperature across the year in a particular city, Madrid (Spain), we can place our selected outfits across time to observe the relationship between outfits and temperature.

No alt text provided for this image
More importantly, the transition between the seasons takes place over weeks, and our model should be able to adjust the recommendations over time, helping our users transit into the next season gradually.
Of course, in the real world, we will have access to a richer set of time series data. For example, customers’ view/purchase trends over time, hot sellers, or fads like Korean Kpop Fashion. Weather data here is an illustration of the potential of incorporating time series data and not the definitive nor sole data solution for the problem.

Reviewing Results from Seasonal Collection

Before we dive into the code, let us take a look at the input/output of the model to gain intuition on the problem and proposed solution.

Input:

  • I select 24 outfits for each season. They will act as the seed images for Seasonal Collections.
  • I will only pass images into the model; the model does not ingest additional attributes/descriptions/product details; this is because we want the model to automatically learn and detect the style of fashion images passed into it without further human/machine labeling.

Output:

  • The goal is for Seasonal Collections to blend in the influence of each season, weighted by how deep into the season we are.
No alt text provided for this image

Madrid’s weather transits in a gradual manner from Winter to Spring from January to May. Seasonal Collections can generate a new collection for the transition period by blending the features of winter (coat, long sleeves, full-length wear) with features from Spring (lighter materials, colorful wear, less layering) into a new “Spring is coming collection”. The ratio we use here is 50–50, so both Winter and Spring have equal weight in the final blend (we will explore this concept in the code section).

No alt text provided for this image

In contrast to Madrid, Melbourne’s Summer transits to Autumn rapidly, with temperatures falling several °C every week. Thus, we need to aggressively introduce elements from the Autumn collection to match our customers’ requirements for warmth. Using a blending ratio of 25–75, we assign a higher weight to the Autumn collection, allowing our model to pull out essential attributes such as layering, long sleeves, and thicker materials to keep our customers warm.

No alt text provided for this image
We can see the importance of localization; to tune the model for different cities across different periods. There is no one size fits all solution for RecSys, especially when we want to recommend across temporal data.

The Code

Link to code on Colab (you just need a free Google Account to run the code on GPU in the cloud).

Seasonal Collections is built on Convolutional Neural Networks, with transfer learning from ResNet and approximate nearest neighbors. I will not dive into the individual theories as they have been well covered in many articles:

Explanation of Overall Setup:

Convolutional Neural Networks:

ResNet and Transfer Learning:

Approximate Nearest Neighbors:

Instead, I will focus on the core logic of Seasonal Collections:

No alt text provided for this image

The command to get the list of outfits for the Melbourne example above is this line of code:

get_similar_image(generate_image_mix(summer_ids, autumn_ids, number_of_items=24, ratio=0.25))
  • In essence, we are taking two collections (summer_ids and autumn_ids) and mixing them up based on the ratio (0.25) and selecting 24 items from the mix.
  • This means that 6 outfits from summer and 18 outfits from autumn will be used as the seed to generate the new collection.
  • The ratio is thus the critical hyperparameter that can be tuned based on the time-series pattern. If we want to increase the influence of a collection, we tune the ratio in order to include more outfits of that collection in the mix.
  • Centroid embeddings will then merge all the seed images into one representation by averaging the values across all dimensions. Finally, approximate nearest neighbors is applied to return outfits closest to the representation.
  • We can place the get_similar_image() code fragment above in a loop and generate a mix of collections based on different ratios:

Ratio 1.00 Winter vs. Spring. Outfits from DeepFashion, open-source by Liu Z. et al.

No alt text provided for this image


Ratio 0.75 Winter vs. Spring. Outfits from DeepFashion, open-source by Liu Z. et al.

No alt text provided for this image


Ratio 0.50 Winter vs. Spring. Outfits from DeepFashion, open-source by Liu Z. et al.

No alt text provided for this image


Ratio 0.25 Winter vs. Spring. Outfits from DeepFashion, open-source by Liu Z. et al.

No alt text provided for this image


Ratio 0.00 Winter vs. Spring. Outfits from DeepFashion, open-source by Liu Z. et al.

No alt text provided for this image

What have we learned

We can intuitively incorporate seasonality and time-series data into RecSys by tweaking the ratio to blend the different collections. However, it is not easy to determine this ratio in the real world. We will need to conduct a lot of AB testing, and gather feedback from users, domain experts, product teams. Furthermore, we will have to localize the solution across different geographies and markets to cater for the difference in fashion trends and even the weather.

Explore the rest of Modern Visual RecSys Series

  1. How does a Recommender Work? [Foundational]
  2. How to Design a Recommender? [Foundational]
  3. Intro to Visual RecSys [Core]
  4. Convolutional Neural Networks Recommender [Pro]
  5. COVID-19 Case Study with CNN [Pro]
  6. Building a Personalized Real-Time Fashion Collection Recommender [Pro]
  7. Advanced topics — Temporal Modeling [Pro][we are here]
  8. The Future of Visual Recommender Systems: Four Practical State-Of-The-Art Techniques [Foundational]

要查看或添加评论,请登录

Kai Xin Thia的更多文章

社区洞察

其他会员也浏览了