Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Machine learning has shifted from backend tech to a frontline role shaping consumer experiences like music recommendations.

  • Spotify personalizes not just music content but also merchandising and UI elements like playlist covers and home screen layout.

  • Music preferences are deeply contextual, changing by time of day, activity, and listener persona.

  • The assumption that skipping a song is a purely negative signal is often incorrect; skip behavior varies by user goals and trust.

  • Mixed methods research at Spotify combines data science, qualitative user research, and now increasingly machine learning itself.

  • Personas such as 'Nick' (curators) and 'Shelley' (casual listeners) help segment skip behaviors and user needs effectively.

  • Editorial playlists blend human curation with ML personalization, balancing communal music experience and individual taste.

  • Contextual factors, including social and environmental considerations, greatly influence music listening behavior and signal interpretation.

  • User mental models around music and playlists differ significantly across personas and impact engagement and skipping.

  • User research teams at Spotify have grown significantly, highlighting the rising importance of research in ML-driven product development.

Notable Quotes

"Machine learning used to be backend technology, but now millions of consumer experiences are decided by machines or algorithms."

"These playlists have some element of machines recommending music in addition to editors behind the scenes."

"There’s been a fundamental shift in consumer products in the last five years driven by machine learning."

"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."

"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."

"Human behavior is complex and machine learning models need to understand those nuances."

"We consider machine learning as the third leg of mixed methods research, alongside data science and user research."

"Music is personal but also cultural and communal, which creates complexity for personalization."

"Personas like Nick and Shelley help us understand different user attitudes towards music listening and skipping."

"There hasn’t been a better time to be a user researcher, especially in machine learning enabled experiences."

Ask the Rosenbot
Sarah Kinkade
Design Management Models in the Face of Transformation
2022 • Design at Scale 2022
Gold
Alexia Cohen
Increasing Health Equity and Improving the Service Experience for Under-Served Latine Communities in Arizona
2024 • Advancing Service Design 2024
Gold
Jacqui Frey
Setting the Table for Dynamic Change
2019 • DesignOps Summit 2019
Gold
Victor Udoewa
Radical Participatory Research: Decolonizing Participatory Processes
2022 • Advancing Research 2022
Gold
Louis Rosenfeld
Opening Remarks
2023 • Advancing Research 2023
Gold
Kara Kane
Communities of Practice for Civic Design
2022 • Civic Design Community
Sol Mesz
Hands or Brains? How to Hire for Strategy, Strategically
2024 • Enterprise Experience 2020
Gold
Mark Boulton
Ops without Designers
2018 • DesignOps Summit 2018
Gold
Sam Proulx
Accessibility: An Opportunity to Innovate
2022 • Design at Scale 2022
Gold
Sheryl Cababa
Thinking in systems to address climate with Sheryl Cababa
2024 • Climate UX Interest Group
Maggie Dieringer
Creating Consistency Through Constant Change
2024 • DesignOps Summit 2020
Gold
John Donmoyer
Shipping your code generation experiments to production
2025 • Designing with AI 2025
Gold
Niko Laitinen
Adaptable Org Design for Resilient Times
2021 • Design at Scale 2021
Gold
Deanna Smith
Leading Change with Confidence: Strategies for Optimizing Your Process
2024 • DesignOps Summit 2024
Gold
Lisa Welchman
Cleaning Up Our Mess: Digital Governance for Designers
2018 • Enterprise Experience 2018
Gold
Uday Gajendar
Day 1 Welcome
2024 • Designing with AI 2024
Gold

More Videos

Michelle Chin

"I’ve found that being approachable and providing space for Q&A helps build trust and confidence in the design ops practice."

Michelle Chin

The DesignOps Starter Kit

September 29, 2021

Louis Rosenfeld

"I talk about being data driven and expanding the definition of data to include both quantitative data and qualitative data."

Louis Rosenfeld Jose Coronado Rachel Posman Guneet Singh Crystal Yan

The Bigger Picture: A Panel Discussion

October 23, 2019

Kristin Skinner

"Google’s HEART framework is great because different teams—engineering, product, and design—can all use it."

Kristin Skinner Kamdyn Moore

Group Activity: A Deep Dive Into Value and Outcomes

October 23, 2019

Josh Clark

"This isn’t about Skynet; it’s about simple signals of awareness that adapt to immediate context."

Josh Clark Veronika Kindred

Sentient Scenes and Radically Adaptive Experiences

June 11, 2025

JP Allen

"Tools are getting much more complex and have more crossover, which made redesigning the map a challenge."

JP Allen Holly Holden

Navigating the UX Tools Landscape

October 1, 2021

Dr. Jamika D. Burge

"The ultimate measure of our research is its impact—did we build the right thing and did it have the expected effect?"

Dr. Jamika D. Burge Robert Fabricant Peter Merholz Noam Segal Teresa Torres

A Genuine Conversation about the Future of UX Research

March 20, 2024

Dan Willis

"When you’re new and inexperienced, asking for help feels like admitting you don’t belong, but really, it shows wisdom."

Dan Willis

Enterprise Storytelling Sessions

June 8, 2016

"Every one of us is probably the lubricant in a system—you only notice us when things go bad."

Panel Discussion: Communicating the Value of DesignOps

November 7, 2018

Husani Oakley

"The scale of the enterprise combined with AI’s power has earth-shattering implications."

Husani Oakley

Theme Three Intro

June 6, 2023