Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Machine learning has shifted from backend tech to a frontline role shaping consumer experiences like music recommendations.

  • Spotify personalizes not just music content but also merchandising and UI elements like playlist covers and home screen layout.

  • Music preferences are deeply contextual, changing by time of day, activity, and listener persona.

  • The assumption that skipping a song is a purely negative signal is often incorrect; skip behavior varies by user goals and trust.

  • Mixed methods research at Spotify combines data science, qualitative user research, and now increasingly machine learning itself.

  • Personas such as 'Nick' (curators) and 'Shelley' (casual listeners) help segment skip behaviors and user needs effectively.

  • Editorial playlists blend human curation with ML personalization, balancing communal music experience and individual taste.

  • Contextual factors, including social and environmental considerations, greatly influence music listening behavior and signal interpretation.

  • User mental models around music and playlists differ significantly across personas and impact engagement and skipping.

  • User research teams at Spotify have grown significantly, highlighting the rising importance of research in ML-driven product development.

Notable Quotes

"Machine learning used to be backend technology, but now millions of consumer experiences are decided by machines or algorithms."

"These playlists have some element of machines recommending music in addition to editors behind the scenes."

"There’s been a fundamental shift in consumer products in the last five years driven by machine learning."

"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."

"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."

"Human behavior is complex and machine learning models need to understand those nuances."

"We consider machine learning as the third leg of mixed methods research, alongside data science and user research."

"Music is personal but also cultural and communal, which creates complexity for personalization."

"Personas like Nick and Shelley help us understand different user attitudes towards music listening and skipping."

"There hasn’t been a better time to be a user researcher, especially in machine learning enabled experiences."

Ask the Rosenbot
Josh Clark
Sentient Scenes and Radically Adaptive Experiences
2025 • Designing with AI 2025
Gold
Jemma Ahmed
Redefining the research toolkit: Expanding methodologies for a changing world
2025 • Advancing Research 2025
Gold
Prayag Narula
HCI 2.0: Humanity Deserves the Attention that UX Research has to Offer
2023 • Advancing Research 2023
Gold
Anat Fintzi
Delivering at Scale: Making Traction with Resistant Partners
2022 • Design at Scale 2022
Gold
Charlotte Vorbeck
Pipeline to Civic Design
2021 • Civic Design 2021
Gold
Billy Carlson
Principles of Team Wireframing
2023 • DesignOps Summit 2023
Gold
Vicky Teinaki
Short Take #3: UX/Product Lessons from Your Industry Peers
2022 • Design in Product 2022
Gold
Scott Stephens
The Next Generation in DesignOps Toolsets
2022 • DesignOps Community
Hugh Dubberly
Problems with Problems: Reconsidering the Frame of Designing as Problem-Solving
2019 • Enterprise Community
Carol Smith
Operationalizing Responsible, Human-Centered AI
2023 • Enterprise UX 2023
Gold
Holly Cole
Understanding Experiences: When you have to do more than work
2018 • DesignOps Summit 2018
Gold
Carla Casariego
DesignOps in Wonderland
2019 • DesignOps Summit 2019
Gold
Kristin Skinner
Theme 2: Introduction and Provocation
2024 • DesignOps Summit 2020
Gold
Ned Dwyer
Right horses for the right courses – how and when to democratize research
2025 • Advancing Service Design 2025
Conference
Victor Lombardi
Bridging Design and Climate Science
2024 • Climate UX Interest Group
Alana Washington
Theme 3 Intro
2021 • DesignOps Summit 2021
Gold

More Videos

Brian T. O’Neill

"Users want to understand why this happened, what will happen, and how can we make it happen—explainability is critical."

Brian T. O’Neill Maria Cipollone Luis Colin Manuel Dahm Mike Oren

Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?

February 18, 2022

Jorge Arango

"I see these AI tools as augmenting human work, letting us focus on what matters most, not replacing humans."

Jorge Arango

Scale Smart: AI-Powered Content Organization Strategies

September 24, 2024

Bria Alexander

"If you want to take advantage of asking speakers questions and having discussions, join our Slack."

Bria Alexander

Opening Remarks

October 1, 2021

Louis Rosenfeld

"Please put your questions in the Slack thread associated with the talk, not just the general channel, or they’ll get lost."

Louis Rosenfeld Bria Alexander

Day 1 Welcome

September 23, 2024

Mansi Gupta

"How can we not forget about women?"

Mansi Gupta

Women-Centric Research: What, Why, How

March 29, 2023

Bria Alexander

"You don’t have to take notes; session videos and decks will be shared as soon as possible."

Bria Alexander

Opening Remarks

October 3, 2023

Andy Polaine

"People are losing the grasp of the customer experience while UX teams are very strong."

Andy Polaine Lavrans Løvlie

What is the role of service design in product-led organizations?

December 3, 2024

George Abraham

"The CSS is only what’s relevant for recreating the layer; components know how to handle theming and styling internally."

George Abraham Stefan Ivanov

Design Systems To-Go: Indigo.Design Overview and Exploring the Developer Workflow (Part 3)

October 1, 2021

Robin Beers

"Power core is where senior executives really care about outcomes or functions that hold power."

Robin Beers

Beyond Insights: Researchers as Organizational Change Catalysts

March 25, 2024