Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Machine learning has shifted from backend tech to a frontline role shaping consumer experiences like music recommendations.

  • Spotify personalizes not just music content but also merchandising and UI elements like playlist covers and home screen layout.

  • Music preferences are deeply contextual, changing by time of day, activity, and listener persona.

  • The assumption that skipping a song is a purely negative signal is often incorrect; skip behavior varies by user goals and trust.

  • Mixed methods research at Spotify combines data science, qualitative user research, and now increasingly machine learning itself.

  • Personas such as 'Nick' (curators) and 'Shelley' (casual listeners) help segment skip behaviors and user needs effectively.

  • Editorial playlists blend human curation with ML personalization, balancing communal music experience and individual taste.

  • Contextual factors, including social and environmental considerations, greatly influence music listening behavior and signal interpretation.

  • User mental models around music and playlists differ significantly across personas and impact engagement and skipping.

  • User research teams at Spotify have grown significantly, highlighting the rising importance of research in ML-driven product development.

Notable Quotes

"Machine learning used to be backend technology, but now millions of consumer experiences are decided by machines or algorithms."

"These playlists have some element of machines recommending music in addition to editors behind the scenes."

"There’s been a fundamental shift in consumer products in the last five years driven by machine learning."

"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."

"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."

"Human behavior is complex and machine learning models need to understand those nuances."

"We consider machine learning as the third leg of mixed methods research, alongside data science and user research."

"Music is personal but also cultural and communal, which creates complexity for personalization."

"Personas like Nick and Shelley help us understand different user attitudes towards music listening and skipping."

"There hasn’t been a better time to be a user researcher, especially in machine learning enabled experiences."

Ask the Rosenbot
Aletheia Delivre
New Shapes and Emerging Identities for Design Ops
2025 • DesignOps Summit 2025
Gold
Megan Blocker
Getting to the “So What?”: How Management Consulting Practices Can Transform Your Approach to Research
2024 • Advancing Research 2024
Gold
Katie Hansen
Finding the unknown in the known: Harnessing meta-analysis and literature review
2025 • Advancing Research 2025
Gold
Benjamin Wiedmaier
Redefining Toolkits: Unbundling to Create a Perfect Match
2025 • Advancing Research 2025
Gold
Adrian Howard
Sturgeon’s Biases
2024 • DesignOps Summit 2024
Gold
Peter Merholz
The Trials and Tribulations of Directors of UX
2023 • Enterprise Community
Stefanie Owens
Optimizing for Outcomes: Transformation Design in Systems at Scale
2024 • Advancing Service Design 2024
Gold
Trisha Terhar
Empathizing with the Empowered: Non-Researcher Responses to Democratization
2022 • Advancing Research 2022
Gold
Marc Fonteijn
Increase your confidence, influence, and impact (through a Professional Community)
2024 • Advancing Service Design 2024
Gold
Kristin Skinner
Group Activity: A Deep Dive Into Value and Outcomes
2019 • DesignOps Summit 2019
Gold
Ilana Lipsett
Anticipating Risk, Regulating Tech: A Playbook for Ethical Technology Governance
2021 • Civic Design 2021
Gold
Carol Scott
Avoid Harming Your Team and Users: Promoting Care and Brand Reputation with Trauma-Informed UX Practices
2025 • Rosenfeld Community
Matt Duignan
HITS, Microsoft's internal human insight system: From research library to living body of knowledge
2019 • Advancing Research Community
Shipra Kayan
Make your research synthesis speedy and more collaborative using a canvas
2025 • Rosenfeld Community
Daniel J. Rosenberg
Designing with and for Artificial Intelligence
2022 • Enterprise Community
Craig Villamor
Resilient Enterprise Design
2017 • Enterprise Experience 2017
Gold

More Videos

Lada Gorlenko

"Field usability testing showed stakeholders immediately how much value UX can add by catching issues before deployment."

Lada Gorlenko Sharbani Dhar Sébastien Malo Rob Mitzel Ivana Ng Michal Anne Rogondino

Theme 1: Discussion

January 8, 2024

Dr. Jamika D. Burge

"Cultural narratives often depict women’s indulgence as submissive and emotional, masking other values like determination."

Dr. Jamika D. Burge Mansi Gupta

Advancing the Inclusion of Womxn in Research Practices

September 15, 2022

Chloe Amos-Edkins

"Traditional design thinking and human-centered design rarely take localized nuances and global trends into account."

Chloe Amos-Edkins

A Cultural Approach: Research in the Context of Glocalisation

March 27, 2023

Robin Beers

"The executive’s interpretation was completely out of line with the customers’ realities."

Robin Beers

Beyond Insights: Researchers as Organizational Change Catalysts

March 25, 2024

Courtney Kaplan

"Start with making a big list of all the things breaking within design and bucket them to prioritize."

Courtney Kaplan

Taking it to the next level: Career paths in DesignOps

November 8, 2018

Indi Young

"We live in a solution culture that glorifies people who create solutions but not those who create knowledge."

Indi Young

Paying Better Attention to the Problem with Indi Young

December 12, 2019

Russ Unger

"One of our biggest pushes was accessibility out of the box—508 and ADA compliance can’t be overlooked."

Russ Unger

Getting Out from Under Everyone: How to Escape the Paralysis of Getting Started

June 8, 2016

Josina Vink

"We have this tendency that once we’ve made this big mega map, we think we’ve captured the system in its entirety."

Josina Vink

Navigating the pitfalls of systems thinking in service design

December 4, 2024

Kritika Yadav

"It's not just about fixing the chatbot, it's about bridging the gap between user expectations and actual experiences."

Kritika Yadav

Optimizing AI Conversations: A Case Study on Personalized Shopping Assistance Frameworks

June 10, 2025