Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Machine learning has shifted from backend tech to a frontline role shaping consumer experiences like music recommendations.

  • Spotify personalizes not just music content but also merchandising and UI elements like playlist covers and home screen layout.

  • Music preferences are deeply contextual, changing by time of day, activity, and listener persona.

  • The assumption that skipping a song is a purely negative signal is often incorrect; skip behavior varies by user goals and trust.

  • Mixed methods research at Spotify combines data science, qualitative user research, and now increasingly machine learning itself.

  • Personas such as 'Nick' (curators) and 'Shelley' (casual listeners) help segment skip behaviors and user needs effectively.

  • Editorial playlists blend human curation with ML personalization, balancing communal music experience and individual taste.

  • Contextual factors, including social and environmental considerations, greatly influence music listening behavior and signal interpretation.

  • User mental models around music and playlists differ significantly across personas and impact engagement and skipping.

  • User research teams at Spotify have grown significantly, highlighting the rising importance of research in ML-driven product development.

Notable Quotes

"Machine learning used to be backend technology, but now millions of consumer experiences are decided by machines or algorithms."

"These playlists have some element of machines recommending music in addition to editors behind the scenes."

"There’s been a fundamental shift in consumer products in the last five years driven by machine learning."

"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."

"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."

"Human behavior is complex and machine learning models need to understand those nuances."

"We consider machine learning as the third leg of mixed methods research, alongside data science and user research."

"Music is personal but also cultural and communal, which creates complexity for personalization."

"Personas like Nick and Shelley help us understand different user attitudes towards music listening and skipping."

"There hasn’t been a better time to be a user researcher, especially in machine learning enabled experiences."

Sam Proulx
Mobile Accessibility and You
2022 • Design at Scale 2022
Gold
Michele Marut
Research Repositories Reconsidered
2019 • DesignOps Community
Ted Booth
Discussion
2016 • Enterprise UX 2016
Gold
Maria Skaaden
Continuous Design: One eye on the horizon and the other on the next wave
2018 • DesignOps Summit 2018
Gold
Bryce Benton
[Demo] AI-powered UX enhancement: Aligning GitHub documentation with USWDS at Austin Public Library
2024 • Designing with AI 2024
Gold
Sha Hwang
The Lost Year
2021 • Design at Scale 2021
Gold
Rich Mironov
How Can Product Managers and UXers Help Each Other (and Why are Product Folks so Annoying Sometimes)?
2022 • Design in Product 2022
Gold
JD Buckley
COMMUNICATE: Discussion
2018 • Enterprise Experience 2018
Gold
Kristin Skinner
Group Activity: A Deep Dive Into Value and Outcomes
2019 • DesignOps Summit 2019
Gold
Bria Alexander
Opening Remarks
2024 • Advancing Research 2021
Gold
Uday Gajendar
Theme One Intro
2023 • Enterprise UX 2023
Gold
Jim Kalbach
Jazz Improvisation as a Model for Team Collaboration
2019 • Enterprise Experience 2019
Gold
Brenna Fallon
Learning Over Outcomes
2019 • DesignOps Summit 2019
Gold
Noreen Whysel
Short Take #4: UX/Product Lessons from Your Industry Peers
2022 • Design in Product 2022
Gold
Daniel J. Rosenberg
Designing with and for Artificial Intelligence
2022 • Enterprise Community
Niko Laitinen
Adaptable Org Design for Resilient Times
2021 • Design at Scale 2021
Gold

More Videos

Yoel Sumitro

"The wrong learning pedagogy might be fueling impostor syndrome among design researchers."

Yoel Sumitro

Actions and Reflections: Bridging the Skills Gap among Researchers

March 9, 2022

Zariah Cameron

"If you are silent about your pain, they will kill you and say you enjoyed it."

Zariah Cameron

ReDesigning Wellbeing for Equitable Care in the Workplace

September 23, 2024

Doug Powell

"We make the invisible visible and turn abstract strategy into something people can see, touch, and act on."

Doug Powell

DesignOps and the Next Frontier: Leading Through Unpredictable Change

September 11, 2025

Darian Davis

"When fellow co-workers label an individual as difficult, it's hard not to treat that individual as a problem."

Darian Davis

Lessons from a Toxic Work Relationship

January 8, 2024

Mike Oren

"The essence of strategy isn’t painting a grand vision of the future, it's deciding what not to do."

Mike Oren

Why Pharmaceutical's Research Model Should Replace Design Thinking

March 28, 2023

Jacqui Frey

"It’s not about giving up power, it’s about gaining authority through partnership and trust."

Jacqui Frey

Scale is Social Work

March 19, 2020

Kim Holt

"Always remember the power of your values in times of doubt and change; lean into what your team really stands for."

Kim Holt Emma Wylds Pearl Koppenhaver Maisee Xiong

A Salesforce Panel Discussion on Values-Driven DesignOps

September 8, 2022

Samuel Proulx

"It can cost up to 200 times more to remediate and retrofit a solution as opposed to designing it correctly in the first place."

Samuel Proulx

From Standards to Innovation: Why Inclusive Design Wins

September 10, 2025

Laine Riley Prokay

"Soft skills doesn’t always represent active listening, so we updated to be more specific."

Laine Riley Prokay

How DesignOps can Drive Inclusive Career Ladders for All

September 30, 2021