A journey of wine, food, and machine learning.

The quick version: After a couple of years spent exploring ways that data could be used to create a useful product for people who love food and wine, I'm excited to share a first version of Vi (rhymes with eye). The objective of Vi is to help you find great wine and food pairings and discover new wines similar to those you already know and enjoy. To achieve this, Vi captures an enormous amount of knowledge from wine and food experts, augments that data using machine learning techniques, and incorporates a unique user interface for exploring the results.

Birth of an idea

In 2018 after nearly?20 years in the digital agency world, I had the luxury of taking a year off as a mid-career reset. I used this time to get my Court of Master Sommeliers, Americas certification and spend many hours reading about data modeling.

Those two activities went together better than you might think; both are fundamentally about understanding relationships, patterns, and context. Also, a glass of wine while reading about data modeling materially enhances the experience.

As I went deeper into both worlds, I found two questions gnawing at me:

• When pairing with food is such a huge part of enjoying wine, why was it so hard to find good information about it? "Pairs with Chicken" is the level of detail provided by the most popular web sites and apps today, and just isn't helpful information. Roast chicken??Chicken Cacciatore??Jerk chicken??Coronation chicken?in honor of King Charles?* These are wildly different dishes calling for different wines.
• Given the amazing diversity of wines in the world, why were the vast majority of them unknown to most American wine drinkers? It seemed like we were missing out on a lot.

There are plenty of wine education sites and e-commerce offerings out there, but shouldn't there be a tool that?can help you discover a new wine and quickly line it up with what you're making for dinner tonight?

Having considered these underserved needs,?I did the obvious: nothing. If this was a good idea, surely someone smarter than me would have done it already.

Prototyping and data creation

Despite my initial inaction, months later the idea remained stuck in my head, so I turned it into a series of sketches in a notebook, and then into a little prototype to see if those sketches made sense in the form of an app. I shared the prototype with a handful of people. When they said they would use such a tool, they were convincing enough to push me forward to the first challenge: conceptualizing a data structure that could capture the knowledge that's truly important about wine and food, and the relationships between them.

At the time my new data science consulting firm was working on a large Natural Language Processing and Knowledge Representation project, and my colleague Nadjet Bouayad-Agha opened my eyes to the power of the knowledge graph, which is a network of concepts (such as a Wine or a Region), connected by relationships (a Wine is Produced in a Region). A knowledge graph does need to be strictly hierarchical or tabular – structures that define most traditional databases, but are totally impractical in the messy, organic world of wine.

Nadjet and I spent a couple of months exploring all the wine and food datasets we could get our hands on, to see if we could find some useful data to start with. We discovered that none of it could meet our needs. All of the available data was too messy and too inconsistent. We'd need to build from the ground up.

To do that, we started a new project to define the critical attributes of a couple hundred "wine styles" (think New Zealand Sauvignon Blanc or Chablis or Willamette Valley Pinot Noir), and thousands of popular ingredients and dishes across a range of global cuisines. Creating these data sets took a full year, with the wine effort led by Krystal Wen and Mladen Sovilj , and the food effort led by food anthropologist (most interesting job title in the world?) Alison Wong and food scientist Rochelle K. .?

Creating proprietary data sets is hard and time-consuming. There were plenty of debates (some heated) about how to generalize the acidity levels of certain wines, or the true cultural origins of certain dishes. But once we were finished, our new datasets could be used for some very cool tricks.

In the diagram below, you can see how conceptually we can measure the differences in generalized wine styles across one, two, or three dimensions:

In fact, we can use a very modern vector database and very ancient Euclidean geometry to do such calculations across dozens of important dimensions. This enables us to measure the similarities and differences between wines across many different taste attributes and search for wines that have very specific similarities or differences.

Building a pairing neural network

Now came the hard part. Wine and food pairing is an elusive blend of art and science. There are certain rules, for sure; don't pair delicate white fish with bold and tannic red wines, for example. But what makes a pairing good or great? Is Salmon Teriyaki closer to Chicken Teriyaki or Salmon with a Brown Sugar Glaze for wine pairing purposes? There is an ocean of nuance here. Even experts can't write out a precise formula for the rules. It's something to be learned and absorbed over time.

To approach this thorny problem, we had 20 sommeliers from around the world, with diverse cultural and cuisine expertise, each create their own recommended pairings, effectively drawing pairing relationship lines between our wines and dishes. Each added line strengthened the connection, and stronger connections meant greater confidence. If 15 out of 20 sommeliers said Chianti Classico was the perfect pairing with Pepperoni Pizza, then we could probably be highly confident in that recommendation.?Across the somm team over the course of four or five months, we captured about 50,000 expert pairing recommendations. A shoutout to Airtable , which works beautifully as a structured data management interface for a non-technical team to capture knowledge in this way.

Now we were starting to build a critical mass of data, which not only represented a huge amount of human expertise, but could also be used to train a machine learning model that could assess and predict pairing quality. Machine learning models are very hungry for data. Working together with ?a?lar Aytekin, Ph.D. , a true expert on deep learning (and an Albari?o lover), we prototyped some initial models and came to the conclusion that 50,000 pairings were actually not enough.

We realized what we had captured from the somm team was only what makes a good or great pairing - not what makes a bad or terrible pairing. That kind of insight would be both conceptually helpful and also valuable for training a recommendation model.

Rather than asking the somm team to spend time documenting the worst pairings they could think of, we realized we could surface that knowledge by seeking the pairings explicitly avoided in our current data. To do that, we created "clusters" of similar wines using a deep learning-based clustering algorithm. Then, we looked for cases where dishes were never paired with certain clusters. Using this technique, bad pairings which had been invisible in the "negative space" of the data previously suddenly appeared by the tens of thousands.?We were able to identify another hundred thousand pairings ranging from "objectively terrible" to "you wouldn't want to drink this if given a choice", which our somm team verified.

Finally, we had a huge set of data points to feed into a machine learning model. We could satisfy its demand for rich training data and get quality results in return.?

Our training set for the model included:

• Our full wine knowledge graph which captures grapes, regions, flavors, winemaking techniques, and many other relevant facts about the wines of the notable regions of the world.
• Our full food knowledge graph which captures ingredients, cooking techniques, regional cuisines, flavor attributes, and more.
• Our large database of pairing scores based on our somm team's work, augmented with the synthetic "bad pairings" data.

Neural network models, like the human brain, are mysterious and hard to decipher. We can evaluate how well they work but it's very hard to understand why they work. It's critical to create a feedback loop where the human experts can provide feedback and tune the model. With a few cycles of model tuning, we started getting meaningful results that we could feed into our web experience.

The current Vi neural network model has the capability to assess a wide range of wine and food pairings. For now, we're keeping its scope tightly focused on a predefined set of wines and dishes while we continue to improve our modeling approach and deepen the human expertise we're capturing in our knowledge graph.

What you can do with Vi

The current product is an MVP ("Minimally Viable Product") and there is a long way to go on this journey. With that said, Vi has some very useful functions that are available now. What can you do with Vi today?

1. Use the search to find wines, regions, grapes, dishes, ingredients, available wines for a particular budget, etc.

2. Learn about wines – taste profiles, audio pronunciations of unfamiliar names, and similar wines to expand your options and experience.

Once Vi has provided a list of recommended wine pairings, clicking into each wine provides concise details about that wine, with original writing by Lucie R. .

3. Dig into food pairings and personalize to your taste.

Tailor your wine pairings, for example by color, country, price point, or a variety of taste parameters using an interactive infographic within the interface designed by Melanie Stirner .

We'll be releasing another tranche of pairings soon, followed by new features which will enable:

• The ability to scan any recipe to get pairing results.
• Finding a wine to pair with multiple dishes or courses, for when you're in a restaurant sharing a bottle.
• The ability for recipe or wine blog publishers to incorporate elements of Vi within their own sites.

Why it matters

Finally, a little context on why I believe this product matters – and why its importance to me goes beyond the intellectual stimulation that comes from tackling these unique challenges and working with an amazing team.

There are many wine regions around the world making beautiful, memorable wines at very reasonable prices. Many of those wines never make it into the hands of American wine drinkers, because the bottles are labeled with confusing, foreign-language names that are hard to decode and pronounce, leaving people uncomfortable and uncertain about buying them. Even many small and mid-sized American wine regions struggle with awareness.

Ideally, everyone would have a trusted local wine shop or somm that could introduce them to these wines. But not enough people do. These lesser-known regions are full of hardworking winemakers whose wines just need the right intro, and since so many of the wines are made to enjoy with food, a bit of guidance on what to eat with them.?

If we can lower the barrier to discovering these wines by having Vi as a companion on the journey, perhaps we can create a burst of enthusiasm about them, to everyone's benefit.

Footnotes

* One of the most informative and entertaining aspects of Vi is the ability to see the trends of what people are searching for. I had never heard of Coronation Chicken before, but it was the most-searched dish, by far, on Vi during the first week of May when King Charles was crowned in the UK.