Fashion E-Commerce: Using Computer Vision to Find Clothing that Fits Like a Glove

Never let online trends get in the way of creating a great outfit for yourself.

It was October 2019, and my cousin’s wedding was taking place in about a week.

Like everyone else, I was faced with the dilemma — what to wear??The struggle was real.

So there I was, in the mall with my brother picking out shirts, because everyone knows you can’t go wrong with a great shirt, right?

I picked a few and was searching for a place to try them on before deciding which ones to buy. Just because it looks good on the hanger didn’t mean it would look good on me.

Into the fitting?room

I tried them on, and after checking myself out (and getting my brother’s nod of approval), I bought the ones that fit just right. The following week, I wore one of them to the wedding. I looked great. Just call me GQ.

Like many guys, I don’t, particularly like shopping, but I tolerate the fitting room because it gives me the opportunity to see how everything looks before going out in public. Getting the right clothes the first time means not having to head back to the store for a return and round 2. I’m efficient like that.

It goes by different names — changing room,?dressing room, trial room.

You know, those small cubicles with the big mirrors and lighting that makes you look like an extra in The Walking Dead?

The place where before you enter, the staff stops you and barks,?“Sorry, you can only bring in 4 pieces at a time”, and you’re like — “Can I please bring in all 7? I won’t take long!” — just so that you can play dress-up for the next 45 minutes.

They were a necessary evil. But then 2020 happened. With quarantines, social distancing protocols, and strict sanitation rules, shopping for clothes just got a lot harder.

A Problem of Proportions

It’s not like 2020 forced us to spin our own cloth. It was simple enough to?order?clothes online, but it created a whole new set of issues.

Most of us are used to shopping online. Electronics, household appliances, books, or kitchenware is all pretty standard. It’s pretty straightforward to purchase a smartphone, laptop, or other gadgets— you only need to worry if the specs meet your needs and if it’s got decent reviews—Ditto with books, handbags, or even jewelry.

But clothes? Reviews alone won’t cut it.

Shopping for clothes gets old — even for the most avid fashionistas. Finding something you like is the hardest part. Once you find it, things get simple. Grab your size, try it on, and check yourself out.

With online shopping, however, everything gets flipped. Searching for just the right thing is easy — thanks to the ever-improving search features from all the mainstream e-commerce platforms.?But getting the right fit??It’s not as easy as it looks.

It’s no wonder that the average return rate for online apparel shopping is a whopping 32%, and it?burns a hole in the pocket?of e-commerce retailers. Here are some?wrinkle-inducing?stats:

• Almost 1 out of every 3 online apparel purchases gets returned.
• In 2019, about $260 Billion worth of retail goods were returned, out of which $30 Billion were from clothing.
• The cost of returns is estimated to be nearly $1.5 Billion. The cost of each return is between $5-$15.

What are the top 3 reasons people return clothes bought?online?

• Incorrect sizing?is the biggest factor, with?43%?of total online shoppers citing poor fit.
• Quality and color mismatch accounts for?29%.
• 22%?of shoppers want to recreate the fitting room at home.

Despite these stats, online shopping isn’t going away.

More users are shopping online post-COVID:

• 27% of customers switched to online shopping due to COVID-19 in the USA.
• 12% of customers would prefer to continue purchasing online even after COVID.

So needless to say, online shopping is here to stay, leaving apparel retailers with a big problem.?Sizing.

Incorrect sizing contributes $1 Billion in returns annually and affects inventory planning leading to heavy losses for e-retailers.

Sizing variance is not a mere wardrobe malfunction. It is a huge?problem.

Egen decided to?iron?this one?out.

Introducing SmartTry

How do you determine fit when you can’t physically handle the product? The next best thing is to compare it to clothing you already know fits well.

That’s what SmartTry does — it enables users to compare their existing best-fitting clothes to new ones they want to purchase online.

How does SmartTry?work?

Say you’re trying to buy a T-shirt online from Amazon or from mango.com like the below screenshot —

You find the ultimate white T-shirt for the ultimate in effortless cool in your next Zoom meeting. But you’ve never bought a T-shirt from Mango before, so how will you know how it fits?

(We’ll spare you the rant on inconsistent sizing. You know it. We all know it.)

Returning things during COVID is a whole new level of inconvenience. So what can you do to ensure you get the right fit the first time?

Here’s where SmartTry’s user experience is guaranteed to?knock your socks off!

Features

Clicking on the green SmartTry button opens the SmartTry plugin in a popup on your screen:

• The SmartTry plugin shows you the T-shirt or outfit you want to buy, along with your previous best fits for you to compare.
• Clicking on the?Add a new apparel to Compare?icon lets you upload a picture of an outfit you own into the SmartTry platform. You’ll use this to upload a photo of the T-Shirt that is your current best fit.
• On uploading apparel, it gets added to the?Recommended Items?section and to your?Collection, with a button underneath that says?Compare. Clicking on the?Collection?tab on the left will also show you all your previous uploads neatly organized within categories:

Finally, when you click on?Compare, SmartTry runs its magic in the background and shows you how good of a fit your selection is!

• The blue T-shirt border represents your current best fit for the comparison, while the black border is the T-shirt you want to buy.
• On the right, you’ll be able to see precise measurements for the T-shirt you selected and your current best fit. This helps you easily compare the differences between the two. (Let’s face it. Every site has a size chart. No one is whipping out a measuring tape before clicking add to cart. Problem solved.)
• In this example, the?Small?size seems a little too small — a?67% match. But not to worry! Since you like the T-shirt so much, you can switch between different sizes:

Medium size is a 94% fit.

You can now buy the right size, knowing it’ll fit you to a T!

Life ain’t perfect, but your outfit can be!

Unmasking the?Magic

If you’re a programmer, I’m sure you’re wondering by now —

How does SmartTry really work behind the scenes, and how do you build?it?

Well,?roll up your sleeves because we put together a tutorial for you to follow along.

What you will?learn

Whatever the mode of user interaction may be, the power of SmartTry comes from?Computer Vision?techniques, which allow it to compare two different outfits and analyze the difference in measurements. I’ll walk you through how to build this algorithm.

• Use?OpenCV , one of the most popular and widely used computer vision and machine learning libraries.
• For writing code, use Jupyter-Notebook , an open-source web application that allows you to create and share documents that contain live code, equations, visualizations, and narrative text.

A word to the?wise

Some knowledge of Python is essential in order to understand this tutorial. If you haven’t worked on Python before, I highly recommend this free course on?Introduction to Python before proceeding.

Why Python??Because it is fairly easy to learn, and the Python ecosystem has a lot of tools and libraries for computer vision, machine learning, and more. Once you’ve built the algorithm, Python has several web frameworks that allow you to build a service around this feature.

Tutorial

Go to this?GitHub link, ?and follow the instructions to download and install the project dependencies. Then, run jupyter-lab in your project root and create a new Python3.x notebook file. Next,

Head over to the?Jupyter tutorial ?and follow the steps outlined, to build your very own program that calculates accurate measurements for a given?T-Shirt!

Alternatively

Head over to the below?Binder?link and execute the code right within your browser without the hassles of installation.

The?smarttry.ipynb?notebook has the complete source code with instructions for you to read. You could choose to run this directly or write code on your own in a new notebook file as you follow the tutorial.

Slides

If you prefer taking just a quick peek into the logic, here’s a short slide deck .

So go ahead, put on your thinking cap, and have?fun!

The first version of SmartTry that we developed at Egen achieved an accuracy of?91.7%?for clothing measurements!

Hopefully, you’ll be able to use this as a performance benchmark when following the tutorial and writing the program. I’m curious to know what your accuracy score looks like!

Next Steps

Once you’ve developed your program, you can integrate this into a web service and expose it via an API. Here’s the architecture implementation that we designed at Egen:

It doesn’t stop here! The future scope for such a feature is immense:

• On websites, SmartTry can be easily integrated into any existing e-commerce platform — as a popup, or an installable plugin, or a platform integration, or even as a browser extension.
• Integrate SmartTry into mobile applications, which will allow you to upload photos of your outfit more easily.
• You could maintain a common SmartTry profile that’ll exist across multiple e-commerce websites.
• A history page to quickly review past comparisons.

and more!

Potential Revenue?Models

Being an intermediary between different parties that interact with a product is a model that many businesses are exploring. SmartTry being designed as an easy-to-add plugin with high accuracy makes it a really good contender for adoption in new businesses and existing e-commerce platforms. Here are some revenue models that can be quite viable (although they’re by no means an exhaustive list):

• At-cost SaaS model for online shopping. The pricing could be based on the number of API calls made to the SmartTry platform, for example — $0.001 per 1 API call.
• Size and shopping data of users and their different demographics
• Sending traffic from our platform to “preferred” e-commerce websites
• In-store enterprise yearly licensing model
• A freemium model with basic features accessible for free and upgrades or more advanced features available at a monthly subscription.

Summary

Convenience is king, especially in fashion e-commerce. Solving the sizing problem can help businesses reduce nearly $1 Billion USD in returns. Reduced apparel returns would also translate into reduced carbon emissions, logistics, and waste.

Thanks for?reading!

If you’d like to know how SmartTry can provide value to your business,?reach out to us .

All credits for developing the idea and concept goes to the SmartTry team at?Egen , consisting of?Arif,?Gagan,?Anmol?and?Aatman.

Written by Ashwin Hariharan