Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Machine learning has shifted from backend tech to a frontline role shaping consumer experiences like music recommendations.

  • Spotify personalizes not just music content but also merchandising and UI elements like playlist covers and home screen layout.

  • Music preferences are deeply contextual, changing by time of day, activity, and listener persona.

  • The assumption that skipping a song is a purely negative signal is often incorrect; skip behavior varies by user goals and trust.

  • Mixed methods research at Spotify combines data science, qualitative user research, and now increasingly machine learning itself.

  • Personas such as 'Nick' (curators) and 'Shelley' (casual listeners) help segment skip behaviors and user needs effectively.

  • Editorial playlists blend human curation with ML personalization, balancing communal music experience and individual taste.

  • Contextual factors, including social and environmental considerations, greatly influence music listening behavior and signal interpretation.

  • User mental models around music and playlists differ significantly across personas and impact engagement and skipping.

  • User research teams at Spotify have grown significantly, highlighting the rising importance of research in ML-driven product development.

Notable Quotes

"Machine learning used to be backend technology, but now millions of consumer experiences are decided by machines or algorithms."

"These playlists have some element of machines recommending music in addition to editors behind the scenes."

"There’s been a fundamental shift in consumer products in the last five years driven by machine learning."

"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."

"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."

"Human behavior is complex and machine learning models need to understand those nuances."

"We consider machine learning as the third leg of mixed methods research, alongside data science and user research."

"Music is personal but also cultural and communal, which creates complexity for personalization."

"Personas like Nick and Shelley help us understand different user attitudes towards music listening and skipping."

"There hasn’t been a better time to be a user researcher, especially in machine learning enabled experiences."

Anna Avrekh
Diversity In and For Design: Building Conscious Diversity in Design and Research
2021 • Design at Scale 2021
Gold
Florence Okoye
AfroFuturism and UX Research
2023 • Advancing Research 2023
Gold
Daniel Gloyd
Warming the User Experience: Lessons from America's first and most radical human-centered designers
2024 • Rosenfeld Community
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • DesignOps Summit 2022
Gold
Sarah Fathallah
Beyond insights: Rethinking the role of researchers as stewards of organizational wisdom
2025 • Advancing Research 2025
Gold
Adam Cutler
Discussion
2016 • Enterprise UX 2016
Gold
Ivana Ng
Level Up Your Program with ProductOps
2024 • Enterprise Experience 2020
Gold
Scott Stephens
The Next Generation in DesignOps Toolsets
2022 • DesignOps Community
Briana Thomas
When Design Ops Comes in H.O.T. : A Tale of a Transformed Design Org
2021 • DesignOps Summit 2021
Gold
Ariel Kennan
Theme 2 Intro
2021 • Civic Design 2021
Gold
Kit Unger
Theme 1 Intro
2022 • Design at Scale 2022
Gold
Smitha Papolu
Theme 3 Discussion
2024 • Enterprise Experience 2020
Gold
Lada Gorlenko
Theme 2 Intro
2022 • Design at Scale 2022
Gold
Tanya Snook
Designing the team experience: Building culture through onboarding
2021 • Enterprise Community
Craig Brookes
"Just Make it Look Good" and Other Ways We're Misunderstood
2021 • Design at Scale 2021
Gold
Frances Yllana
DesignOps Exposed: What do our peers really think of us?
2025 • DesignOps Summit 2025
Conference

More Videos

Jon Fukuda

"Through DP&M, designers get more contextual detail than what might be captured in a Jira ticket."

Jon Fukuda Ellie Krysl

Design Planning and Management Support

October 3, 2023

Saara Kamppari-Miller

"Map tours are my favorite thing—taking a new hire on a guided walk of the entire community."

Saara Kamppari-Miller

Cartography for Design Communities

September 10, 2025

Kevin Bethune

"I can’t tell you how many times recruiters ask, can I have a beer with this person, instead of focusing on true inclusion."

Kevin Bethune

Reimagining Design: Unlocking Strategic Innovation

June 8, 2022

Daniel Orbach

"Sometimes organic culture pieces, like playing Wordle at the end of standup, shorten meetings and keep them tight."

Daniel Orbach

Zero to One: Co-Creating Operating Models with your Team

September 23, 2024

Marc Fonteijn

"If you don’t feel connected and you’re standing alone, the challenge looks much bigger."

Marc Fonteijn Ru Butler

Increase your confidence, influence, and impact (through a Professional Community)

December 3, 2024

Clara Kliman-Silver

"There’s a perception that generative AI is gonna change everything, but I think it’s gonna transform, not totally overhaul mine."

Clara Kliman-Silver

UX Futures: The Role of Artificial Intelligence in Design

June 7, 2023

Robin Beers

"Synthesis is a researcher’s superpower, not just analysis or breaking things down."

Robin Beers Nalini Kotamraju Andy Warr

Panel: Excellence in Communicating Insights

March 26, 2024

Ana Maria Montero Barrantes

"For the right audience, if applied correctly, humor and music can move people in many ways."

Ana Maria Montero Barrantes Aditi Dhar Michelle Kaplan Nate Osborne Matt Laurence

The Authentic UX Talent Show

January 8, 2024

Kristen Honey

"I got my PhD so I could get a seat at the table."

Kristen Honey

"Let’s Talk About Data and Crisis”: Public Digital Service Delivery = Open Data + Human Centered Design

November 18, 2021