Three Principles for Designing ML-Powered Products

Machine Learning (ML) has become an indispensable tool at Spotify for delivering personal music and podcast recommendations to over 248 million listeners across 79 markets and in 24 languages. We believe designers have a vital role in ML-driven initiatives, by bringing a human-centered perspective to a technology that can too easily overlook the end-user. If we don't apply a human-centered lens to our design process, we risk optimizing for solutions that don't resonate with users—or worse, completely deliver the wrong solution.

In a world where the product uniquely adapts to each user, we’ve found that creating deeply personalized products requires a new type of mindset and approach to design. Equipped with research about our users, and a deep understanding of business goals, we traditionally define detailed flows of a user’s journey, invent and refine interactions, and visually brand the experience—hopefully with some delight. But when working with Machine Learning at Spotify, we’re now tackling entirely new types of challenges.

Reflecting on a handful of projects at Spotify, we’ve come up with three principles we believe will help others design ML-powered experiences.

1. Identify friction and automate it away.

A focus on removing friction should feel familiar to every designer because we do that work every day. We define friction as anywhere in the user experience where a human struggles in pursuit of their goals. Machine Learning—and more broadly Artificial Intelligence—is a new tool to help us in our mission to make experiences frictionless.

One key to Spotify’s early success was creating a frictionless listening experience. Instead of waiting two minutes to download a specific song, Spotify users could immediately play any song anytime, anywhere. Removing the friction of waiting every time you wanted to play music helped Spotify win over piracy and enabled the streaming revolution to take off.

To view friction another way, let’s break down the success of one of Spotify’s most popular playlists, Discover Weekly. The playlist is comprised of music at the edges of your musical taste—songs you haven’t heard before on Spotify—and refreshes itself with new music every Monday. Discover Weekly was a hit not because the process of discovering new music is new but because the feature simply helped remove the friction inherent in the discovery process.

Before Discover Weekly, finding a new favorite song or artist felt serendipitous because it was so cumbersome and labor-intensive. Searching for new music had friction in the time it took to sift through the massive amounts of music available. Once you found new music, you had the friction of requiring attention, energy, and focus to listen and organize everything into what you liked and disliked. To top it off, the anxiety or frustration of not actually finding anything you liked caused friction in the form of stress.

Clearly, Discover Weekly wasn’t successful simply because of its cover art, catchy name, or great branding—though they certainly helped. It was successful because it automated the personal music expert; it made discovery effortless. Discover Weekly removed the friction of chasing everything down yourself and instead brought the music to you in a neat little package every Monday morning.

Identifying sources of friction is a win, a big win. But automating its reduction with ML — however tempting it may be — shouldn’t be your default. As designers, we should first consider whether augmentation—or assisting—might result in a better user experience. Often, a combination of augmentation and automation can enhance and also improve the experience...instead of simply automating it away. One way of uncovering whether you should augment or automate an experience is to ensure that you’re asking the right questions.

2. Ask the right questions.

Picasso’s quote is a useful lens on Machine Learning; an answer without a clear understanding of the question can lead products astray. This mindset should be familiar to designers, as we often try to validate a solution by challenging our peers to prove they’ve asked the right questions. This becomes even more important when distilling how Machine Learning can improve the user experience.

We embraced this perspective when designing the Spotify mobile app’s Home screen, which is where every user accesses their music, podcasts, or personalized recommendations. When we first started designing Home as a personalized experience, we used Machine Learning to suggest content based on a user's listening history. However, we soon discovered this approach was inadequate because it offered a one-size-fits-all approach to human taste and didn't consider the nuances of the human experience.

We realized we needed to reshape the algorithms in a human-centered way, so we started to dig deeper and ask ourselves questions like: What does it mean to like an artist, album, playlist or podcast? How does a user’s context shape their decision of what to listen to? What does someone need to know before making the choice of what to listen to?

Asking these questions and dozens—maybe hundreds—more forced us to put our users’ needs front and center. We found this approach changed how we designed the page and contextualized our recommendations.

To answer our questions, we started by evaluating the Home screen experience through a manual process — by both assessing user feedback and identifying behavioral patterns in the data. We took the insights from this phase and started testing new hypotheses. It was only after we proved those hypotheses that we started to apply Machine Learning.

3. Go manual before you go magical.

Products that leverage Machine Learning can feel like magic. However, behind the veil of magic is a lot of hard work. Once you’ve identified the friction you want to eliminate and you’ve asked all the right questions, it’s time to find out if the answers actually show up in the data. Are there patterns in the data from which a machine can learn?

As Spotify designers, during this stage of product development, our typical deliverable would usually be a wireframe or prototype of the intended product experience. But when we’re designing an experience that leverages Machine Learning, our deliverable might instead look like a set of rules to follow or the definition of the result you’re hoping to achieve.

To help illustrate this point, let’s look at a similar Spotify product:

Release Radar is a playlist which matches new releases to a user based on their taste. The playlist updates each Friday. Aside from the cover art and the name—which are important to help people connect with the concept—there isn’t really much to “design.” Release Radar uses the same playlist format you find all over Spotify. The product, in this case, is the experience of the content. So as a designer you may ask yourself: how do I prototype a personal experience? What tracks should I include in the playlist? How should I order the tracks? How will I make the songs match your taste?

One way to answer these questions could be to start by using Machine Learning right away, and work with engineers to gather data, train, and tune a model, and see what pattern the machine thinks are relevant. But that process takes time, specialist knowledge, and isn’t even guaranteed to work. Also, as we established before—asking a question like, “What is relevant?”—is too broad. At Spotify, we use a process that is more iterative and more collaborative with our partners on the Engineering, Insights, and Product teams.

When we prototype a new experience, we first create a hypothesis. Using the example of Release Radar, we might begin with this educated guess: By keeping Release Radar fresh and up-to-date, we will keep people engaged with new music they will love. This is a good starting point but opens up a new line of questioning: How will we define “fresh”? How might our users define “freshness”? The list goes on…

We continue to give shape to our content hypothesis by coming up with a set of heuristics. Heuristics are rules which govern how something works. A first heuristic for Release Radar might be: Only include new releases in the last thirty days. A reason to choose thirty days would be because songs released prior would probably not be considered “fresh” by our users.

With our set of heuristics established, we can review the resulting set of songs and assess the quality of this version of Release Radar. Our goal is to use our heuristics to prove our hypothesis first, without applying ML.

Finally, after multiple rounds of hypothesis, heuristics, and testing, we show our prototype to users. This step is critical to keep us focused on their needs and prevents us from building products that only satisfy us, the makers. The true judges of how valuable a product is—with or without ML—are the people who will use it.

Our focus throughout the prototyping process is to go from a hypothesis to a quality experience in the quickest way possible in order to learn the value of the product. We strongly believe in proving the user-benefit of a product or feature as early as possible before investing in Machine Learning. As product designers, we should focus on a process to prove the utility and value of a product before using ML, AI, or any other expedient technology to make an experience feel magical.

To Conclude…

It’s tempting to get caught in the trap of applying a promising new technology like Machine Learning to every problem, but try and resist this temptation. Instead, try to follow Joshua Porter’s advice as he states in his Principles of Product Design:

As designers, we should use Machine Learning as a tool to help address existing user needs in more efficient ways. When we apply ML selectively and appropriately, we can fundamentally reshape the products we bring to market and help people achieve their goals in ways they could never imagine.

The ideas shared here aren’t fundamentally new; these are all techniques derived from long-established human-centered design principles. We are simply applying a new lens of Machine Learning informed by lessons we’ve learned from users’ reactions to our ML-driven products.

Building products with Machine Learning is still nascent, and we’re excited to see how designers and human-centered thinking can have an impact across a variety of initiatives. We hope our experiences at Spotify can help you and your team evaluate and consider the implications of applying Machine Learning to your products and experiences. If this article is helpful or if you have stories of your own, we’d love to hear from you.