Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning

Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Machine learning has shifted from backend tech to a frontline role shaping consumer experiences like music recommendations.

  • Spotify personalizes not just music content but also merchandising and UI elements like playlist covers and home screen layout.

  • Music preferences are deeply contextual, changing by time of day, activity, and listener persona.

  • The assumption that skipping a song is a purely negative signal is often incorrect; skip behavior varies by user goals and trust.

  • Mixed methods research at Spotify combines data science, qualitative user research, and now increasingly machine learning itself.

  • Personas such as 'Nick' (curators) and 'Shelley' (casual listeners) help segment skip behaviors and user needs effectively.

  • Editorial playlists blend human curation with ML personalization, balancing communal music experience and individual taste.

  • Contextual factors, including social and environmental considerations, greatly influence music listening behavior and signal interpretation.

  • User mental models around music and playlists differ significantly across personas and impact engagement and skipping.

  • User research teams at Spotify have grown significantly, highlighting the rising importance of research in ML-driven product development.

Notable Quotes

"Machine learning used to be backend technology, but now millions of consumer experiences are decided by machines or algorithms."

"These playlists have some element of machines recommending music in addition to editors behind the scenes."

"There’s been a fundamental shift in consumer products in the last five years driven by machine learning."

"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."

"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."

"Human behavior is complex and machine learning models need to understand those nuances."

"We consider machine learning as the third leg of mixed methods research, alongside data science and user research."

"Music is personal but also cultural and communal, which creates complexity for personalization."

"Personas like Nick and Shelley help us understand different user attitudes towards music listening and skipping."

"There hasn’t been a better time to be a user researcher, especially in machine learning enabled experiences."

Ask the Rosenbot
Rima Campbell
Increase Productivity and Drive Business Impact
2024 • DesignOps Summit 2024
Gold
Mansi Gupta
Women-Centric Research: What, Why, How
2023 • Advancing Research 2023
Gold
Kristin Skinner
8 Types of Measures in Design Operations
2020 • DesignOps Community
William Newton
How to Lead With Data, and Without Data
2023 • Enterprise UX 2023
Gold
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
James Lang
If you can design an app, you can design a community
2025 • Rosenfeld Community
Christian Crumlish
AMA with Christian Crumlish, author of Product Management for UX People
2022 • Enterprise Community
Shanti Mathew
Civic Design at Scale: Introducing the Public Policy Layer Cake
2021 • Civic Design 2021
Gold
Weidan Li
Qualitative synthesis with ChatGPT: Better or worse than human intelligence?
2024 • Designing with AI 2024
Gold
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • Enterprise UX 2023
Gold
Angy Peterson
More Than Technology: Personalized Public Sector Experiences
2021 • Civic Design 2021
Gold
John Devanney
The Design Management Office
2017 • DesignOps Summit 2017
Gold
Saara Kamppari-Miller
Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
2024 • DesignOps Community
Eniola Oluwole
Lessons From the DesignOps Journey of the World's Largest Travel Site
2019 • DesignOps Summit 2019
Gold
Daniel J. Rosenberg
Digital Medicine Design
2019 • Enterprise Community
Louis Rosenfeld
GenAI for UXers: A Rosenbot Demo and Discussion
2025 • Designing with AI 2025
Gold

More Videos

Helen Armstrong

"Elizabeth Churchill framed AI as a pedal assist system, helping us go further and faster but sometimes needing to dial it back."

Helen Armstrong

Augment the Human. Interrogate the System.

June 7, 2023

Christian Bason

"Governments pretending to be rational is a flawed assumption; we must incorporate emotion and bodily experience in policymaking."

Christian Bason

Expand—Rethinking Design for Public Challenges

September 14, 2022

Onur Kocan

"Political parties at the provincial level have their own mandates, which complicates unified service design."

Onur Kocan Ayhan Ensici

Understanding the Strategy for Civic Design in a Complex City: Istanbul

November 16, 2022

Peter Van Dijck

"You don’t want to give too many options like rating from one to ten because consistency gets lost between different LLM calls."

Peter Van Dijck

Building impactful AI products for design and product leaders, Part 2: Evals are your moat

July 23, 2025

Nalini Kotamraju

"What happens if research does not report into UX? What might happen if we uncouple research from UX?"

Nalini Kotamraju

Research After UX

March 25, 2024

Brendan Jarvis

"That night at 11:59 PM the entire nation of New Zealand would move into what was called a level 4 lockdown."

Brendan Jarvis

Framing Tomorrow by Questioning Today

June 8, 2022

Bria Alexander

"Everything can be designed – ourselves, relationships, processes, policies, systems."

Bria Alexander Ariel Kennan Charlotte Lee Sarah Brooks Emily Lessard Gordon Ross Joanne Dong

Reflect and Chart Forward

December 10, 2021

Frances Yllana

"If you take away one thing, it’s get hands-on and familiarize yourself with what AI can and cannot do as both a tool and design material."

Frances Yllana Jorge Arango Maria Taylor Briana Thomas

The Big Question about Impact: A Panel Discussion

September 24, 2024

Liz Ebengo

"We had conversations with donors over dinners and asked about their vision for innovation and future impact—it’s about asking the right questions."

Liz Ebengo

The Burden on Children: The Cost of Insufficient Post-Conflict Services and Pathways Forward

December 4, 2024