How Does Spotify Know What You'll Like?

Just a short few days ago, I was discussing the impact of recommender systems with some students on a course I’m teaching. 

Netflix, Amazon, Facebook, and many other online services, use our data to suggest other products we might like. 

Is this helpful, or does it serve only to entrench our existing preferences?

How do we discover anything truly new or unexpected when our options are dictated by systems designed to deliver “engagement”?

As such, they are programmed to give us more of what we already like, which can lead to an inward spiral of familiar influences.

It’s a bigger concern than I thought, and it got me thinking some more. 

We are more creative when exposed to diverse influences, but to make this work in the machine age, our tastes must fit into a mathematical model.

Technology is not just within our environment; it envelops our perceptions and shapes them from the outset. It matters that we figure this out. 

And so, I got to thinking of how we consume our tunes — particularly on streaming platforms like Spotify. 

This story will cover:

• What is Spotify, anyway?
• Spotify in numbers.
• How Spotify knows you so well.
• The recommender system.

WHAT’S A SPOTIFY, ANYWAY?

Spotify: Digital, cloud-based music platform that provides cross-device access to over 50 million songs, and a rapidly-rising number of podcasts and videos. Founded in Stockholm in 2008. 

It offers a free, ad-supported option and a paid, ad-free version at $9.99 (US) per month. You can get Spotify + Hulu for $12.99 a month (again, US). Other plans (family and student deals are common) are available, but they differ by market. 

Spotify pays royalties to musicians (it actually contracts a third party to handle this) based on the number of plays their songs receive. Typically, this is between $0.006 and $0.0084 per play. No wonder songs are getting shorter and catchier. 

A new focus for the company is the Spotify Marketplace, which is open to a range of “creators”. This two-sided marketplace aims to encourage artists to upload and edit their work on the Spotify platform, where fans can discover and enjoy said work.

Spotify acquired SoundBetter to add a new suite of creation tools earlier this year. It hopes that this approach will reduce operating costs and increase its consumer appeal, faced with increasing Apple/Amazon competition.

The company Swedish, but its HQ is in Luxembourg these days. Listeners can sign up in over 80 countries. 

OK, LET’S KICK OFF WITH SOME NUMBERS

On October 28, Spotify released its Q3 2019 earnings report. I’ve read it, so you perhaps don’t have to. 

The Swedish music streaming platform has had a pretty good year, all told. 

The key statistics:

• Total revenue of €1.7 billion for the quarter, up 28% on 2018.
• Premium revenue was €1.56 billion, outperforming forecasts.
• Ad-supported revenue was €170 million, significantly below forecast.
• Operating margin of 3.1% (€54 million); just the third quarter in which Spotify has delivered a profit since its launch in 2008. Due to some impending investments, the company does expect to report a loss for Q4.
• Premium (paid, ad-free) subscribers increased 31% YoY, from 87 million to 113 million.
• Free (unpaid, ad-supported) subscribers increased 29%, from 109 million to 141 million.
• This led to a 30% rise in overall monthly active users (MAUs).
• Podcast streaming hours increased 39% Q3 vs. Q2.
• Spotify believes that it is adding twice as many subscribers to its music services as Apple, and its users are singificantly more engaged with the Spotify platform. This is based on the public data released by Apple in its recent statement.

Spotify cites Latin America and Asia as two significant growth areas, both for this year and for 2020. Spotify Lite, now available in 36 countries, has certainly helped adoption in countries like India. 

The headline figures come from North America, however. The chart below shows the breakdown of Spotify subscribers by region:

Spotify built its foundations in its native Europe, but faces stiffer competition in the US, where it launched in 2011.

Pandora (the music platform, not the charm bracelets or the jar-opening miscreant) is Spotify’s most direct competitor in this market.

At the end of Q2 2015, Pandora had more active users worldwide than Spotify (79.4m vs. 75m).

Pandora has focused inwards, closing down its expensive operations in Australia and New Zealand in 2017 to redouble its US efforts.

Meanwhile, Spotify has expanded to more markets every year, with splashy advertising campaigns to boot. 

As of Q3 2019, Spotify has nearly four times as many MAUs as Pandora (248m vs. 63.1m).

We might expect this outcome; Pandora still has a sizeable following in the market it has chosen to prioritize.

What we may not have anticipated is that Spotify now has more MAUs in the US than Pandora (67m vs. 63m).

Moreover, Pandora’s user base is shrinking; it had 68m listeners in the US this time last year.

This is not the only statistic that matters — far from it, in fact. 

Pandora’s advertising revenues hit $315m in Q3 2019 versus $190m for Spotify in the US. 

Pandora has announced innovative new ad formats that may help brands connect with audiences through online audio. This has been a significant challenge; much of what we hear today is reminiscent of old-fashioned radio advertising, down to the sound effects. 

One of Pandora’s ideas is to create interactive ads, allowing listeners to skip or ask for more information by speaking to their device. 

This helps tackle another challenge with audio advertising, when compared with text-based online advertising.

It is simply very hard to know what impact an audio ad has on its recipient.

Interactive voice ads create additional data points, which will feed back into Pandora’s reporting and help advertisers measure and improve performance. 

Spotify’s founders set out to create an open platform with every song available for free, supported by advertisements. One of the founders (current CEO, Daniel Ek) made his early money from Advertigo, which sold to TradeDoubler. 

It is no surprise that Spotify is moving into interactive audio ads, which will start playing soon for users who have their microphone enabled in the Spotify app.

Spotify’s aim is to become synonymous with music streaming.

For example, if you type playlist.new into the Google Chrome browser, it will take you straight to Spotify.

The app is available through smart speakers and soon, Fitbit wearers will be able to control their music with gestures.

GOT IT, IT’S POPULAR. BUT WHY?

We talk a lot about switching costs these days. Well, I certainly do.

It’s a term used in microeconomics (the economics of decisions made by individuals), and it is applied to the costs a consumer incurs by changing from one supplier to another.

This could mean money, time, or effort. For an iPhone owner it’s simpler to upgrade to the new iPhone than it would be to get on the Samsung bandwagon. That would mean setting up new accounts, answering a load of questions, and so on.

These aren’t insurmountable barriers, by any means, but our brains love routine. That’s why you read this newsletter every week. 

Many Uber users also have the Lyft app, and it wouldn’t kill them to add a third or a fourth ride-hailing alternative. The switching costs are low; if a cheaper, faster, better ride-hailing app comes along, we would have little allegiance to Uber, I imagine. 

Similarly, a lot of companies get wrapped up in their new capabilities (anyone need another camera on their smartphone?) and lose sight of what the customer really wants.

This scenario opens the possibility of “disruption”; a new entrant pops up and delivers what we want, not what the company thinks we want. 

In that regard, Spotify has to be vigilant. Amazon can envelop music streaming within its Prime subscription machine, while Apple and Google are also active in this space. 

Sure, this still doesn’t explain Spotify’s enduring popularity, but we’re moving in that direction.

Spotify has a clear and unwavering focus on using engagement data to understand, and then provide, what their customers crave. 

It estimates that 60% of the time listeners are in a “closed” mindset when they are on the app; they know what they want to listen to and they just need to find it. 

The remaining 40% of time is spent in an “open” mindset. 

In this mindset, users put in less effort, they scroll less, they skip tracks more, and they click less on the artist for further information. 

Essentially, they are receptive to new ideas, but in a passive yet impatient state. 

I believe that Spotify’s initial appeal lay in that majority “closed” mindset. To be able to play any song, for free, was a very appealing prospect when Spotify launched.

It still is appealing, but many other services can also offer this now.

Spotify’s lasting attraction lies in how it caters to that 40% “open” mindset.

It has developed a mathematical model for understanding the relationships between artists and listeners, but also imbued this with the ethereal thrill of discovering a new favorite song. 

At the very core of this success is the recommender system.

“Music isn’t like news, where it’s what happened five minutes ago or even 10 seconds ago that matters. With music, a song from the 1960s could be as relevant to someone today as the latest Ke$ha song.”

— Daniel Ek

SPOTIFY & THE RECOMMENDER SYSTEMS

“Music is a language that doesn’t speak in particular words. It speaks in emotions, and if it’s in the bones, it’s in the bones.”

- Keith Richards

Well maybe, Keith.

But he didn’t reckon with deep learning algorithms when he made those far out remarks. 

The truly fantastic https://pudding.cool has produced a range of big data-based analyses on song genres, with ear-opening results.

These visual essays have sparked debate about the patterns they uncover.

Country music employs a smaller range of vocabulary, perhaps due to the familiarity of its audience and the locales it depicts.

Country music also talks about booze more than any other genre, while those abstemious Christians steer clear.

Ok, that finding isn’t altogether scandalous, but it’s still fun to see these trends crystallized in a chart.

And apparently, music is getting more repetitive over time.

Who knew Sinatra was so varied? Although as the great Frasier Crane put it, “after four hours his “ba-bap-ba-ba-a-a-a”s sound a lot like his “scoopety, boop, bop, bam!””

One of Spotify’s data scientists runs the website https://everynoise.com/, which maps out every musical genre based on their interrelatedness. 

The below is a screenshot from said website. Essex indie is a lot like Spanish indie pop, it says.

These charts reveal only a little in isolation, of course. However, the analyses are conducted by individual data scientists, primarily for entertainment purposes.

The risk with this approach is that we take the enjoyment out of music altogether. If it’s in the bones, as our pal Mr Richards suggests, we suck the bones dry with our insatiable hunger for data-driven everything.

Spotify wants to avoid this, for some sensible business reasons. 

Instead, it wants to play the role of that knowing insider that tips you off on the latest bands, those B-sides and off-cuts you missed, but would certainly “dig”. It wants to do so by combining diversity with straightforward relevance:

With that in mind, let’s return to the introductory question in this week’s story:

How can Spotify use data to recommend new music, without simply recommending more of the same?

Well, Spotify’s recommender system provides suggestions for users based on their historical interactions (listen/skip/add to playlist), the attributes of the songs/artists they listen to, and the preferences of what it deems “similar” users.

Where other music services (Songza, Pandora) used manual tagging to categorize songs, Spotify uses deep learning to automate the process and identify latent patterns between artists, genres, and user preferences. 

There are three recommendation models at work on Spotify:

• Collaborative filtering: Uses your behavior and that of similar users.
• Natural Language Processing (NLP): For song lyrics, playlists, blog posts, social media comments.
• Audio models: Used on raw audio.

Collaborative filtering

This model is very important. It uses “nearest neighbors” to make predictions about what other users might enjoy.

This is similar to Netflix’s model, but Spotify’s engine is not powered by star ratings. Spotify must use implicit feedback signals like stream counts to infer what we like.

In fact, Netflix has moved to this approach as it is more reliable than using explicit feedback. If you want to understand people, listen to what they do, not what they say.

Each user on Spotify has their own taste profile, shaped by what they listen to, when they listen to it, how often, and so on. 

The image below looks like a weather system, but it is the taste profile of a journalist at Quartz; Spotify gave him access to his profile, but generally prefers to keep this part of the platform guarded. Unfortunately, we can’t log in and look at our own profiles.

Although at Spotify, these taste profiles look a bit more like this:

This profile is used to shape each user’s Discover Weekly playlist. 

Discover Weekly has been a phenomenal success for Spotify; over 5 billion tracks were streamed through these playlists in the first year after launch. 

The idea is as simple as the execution is complex: Find 30 songs that the listener would probably like, but has not listened to (on Spotify) yet. The binary system uses the labels 1 (streamed) and 0 (never streamed) for songs, with the latter then organized based on the likelihood the user will engage with them. 

The list is created automatically and is available every Monday.

This allows Spotify to surface the “long tail” of artists and songs that may otherwise go unheard.

If Spotify focused only on that 60% “closed” mindset, when users search for something specific, it would be difficult to get these artists in front of an audience.

That would have a negative impact on the supply side of the platform, as artists would have little incentive to prioritize Spotify. In turn, that would make Spotify less appealing for listeners; we like the idea that 50 million songs are on Spotify, even if we listen only to a minuscule percentage of the whole.

Amazon uses this approach to recommend products; in fact, it is only of the many reasons it assumed a leading position in e-commerce. With so many products on offer, the platform needs to help us navigate to the items that matter. 

The sense of curated exploration goes beyond Discover Weekly. Spotify’s homepage, mood-based playlists, and radio sections all play host to additional personalization.

Spotify’s dedication to catering for that 40% of time spent in an “open” mindset makes it more difficult for competitors to replicate its advantages.

It also adds randomized selctions to generate more feedback for the recommender system. 

The stakes are relatively low, after all. If worst comes to worst, the user can simply skip the tracks they don’t like. If Spotify recommends more hits than misses, that is an acceptable success ratio for the listener.

This all ensures a welcome re-statement of Heraclitus’ teachings for the digital age: One cannot step in the same stream twice. 

Natural Language Processing

Spotify’s engineers, who are wonderfully open about their work, state on a regular basis that natural language processing is more effective on music than one might assume.

They can turn playlists into text documents and analyze how lyrical patterns relate to each other. 

Similar to Google’s NLP algorithms, Spotify identifies the co-location of individual terms and uses this to predict the meaning of phrases. 

In the example below, we see the score certain terms receive in relation to the stimulus ‘Abba’. ‘Dancing queen’ has a slightly higher score than ‘mamma mia’, almost certainly due to the latter’s ubiquity in Italian pop music.

Abba is likely to be described as ‘perky’ and ‘nonviolent’, although I’ve seen wedding dancefloors that offer evidence to the contrary.

This insight then feeds into Spotify’s vast web of entities, with highly rewarding results.

Their tests have found that recommending songs based on these common adjectives surfaces new links that were previously unseen.

For example, music that is described as ‘dark’ or ‘moody’ will cut across genres. We tend to navigate based on the categories we know, such as classical, heavy metal, or disco dancing. That serves a useful purpose, but Spotify’s approach to recommendations brings us new alternatives that we otherwise would not know how to find.

Music is a language that speaks in particular words, it turns out. We just need a bit of technological assistance to help find them. 

Audio models

The first two models we have assessed have delivered Spotify’s recent success; this third model holds the key to its future success.

If a new song is added to Spotify through the expanding marketplace, how can Spotify know which users to the song should be served to?

If the song is from a new artist with a small following, it will generate very few data points. The artist will likely tag the song with category and genre attributes, but this would be sufficient only for an archaic recommender system.

In addition to this, Spotify uses the kind of neural network that is employed by search engines to understand the contents of images. These networks process raw audio to produce a range of characteristics, including key, tempo, and even loudness. 

It looks like this for the highly-repetitive ‘Around the World’ by Daft Punk:

As a result, Spotify can slot these new songs into the right playlists, where they can layer on additional data points as users interact with the content. 

This will be crucial for the next phase of Spotify’s evolution. 

If it can introduce new artists to receptive fans, those artists will start to receive royalty fees. That should encourage other, unknown artists to bring their work to Spotify and, in some cases, even to record and edit their songs using Spotify’s new tools.

IS IT ALL GOOD NEWS?

Nope. For its many and impressive virtues, there are flaws and threats in such a set-up.

A platform like Spotify holds a great deal of control over significant aspects of our culture.

The challenge here arrives when music departs the platform, likely to be forgotten. That is a prosaic, contractual concern, but we should recall that this is not a historical record of culture; it is a private enterprise. 

As Spotify grows its advertising business, it will also collect more data in the background and sell it on to the highest bidder.

Spotify’s incentive is to keep people listening, so it will do what it can to achieve this outcome.

Sometimes, that means delighting customers with new artists; other times it will mean bland repetition. The system doesn’t know the difference — it sets out to reach a result.

As Spotify moves towards on open “marketplace” approach, new threats will emerge. Video and podcast content will provide avenues for bad actors (in every sense) to grab attention.

These creators, for their part, are also incentivized to keep us watching, listening, clicking, just doing anything other than leaving.

Undoubtedly, these are not new concerns and nor are they endemic to Spotify. 

In this week’s main story, we set out to answer the question of whether an algorithmic recommendation system will, of necessity, lead to a diminished pool of personal experience. More of the same, ad infinitum. 

The Spotify example provides an illuminating case in the benefits of blending different models to deliver diversity, as well as relevance. 

This is all shot through with a psychological and cultural understanding of how we listen to and share music. 

Spotify aims to take the nostalgic appeal of the mix tape and upgrading the concept for the digital age.

If we want to pinpoint the switching costs that keep Spotify’s churn rate low (it is less than half that of Apple Music), they lie primarily in how it entertains us in the “open” mindset of exploration. 

Spotify takes calculated risks and optimizes for engagement, without succumbing to the lure of the same easy wins. 

In the process, it demonstrates how algorithmic recommendations can extend our sphere of experience.

Go ahead and sign up to the hi, tech. newsletter.