Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Machine learning has shifted from backend tech to a frontline role shaping consumer experiences like music recommendations.

  • Spotify personalizes not just music content but also merchandising and UI elements like playlist covers and home screen layout.

  • Music preferences are deeply contextual, changing by time of day, activity, and listener persona.

  • The assumption that skipping a song is a purely negative signal is often incorrect; skip behavior varies by user goals and trust.

  • Mixed methods research at Spotify combines data science, qualitative user research, and now increasingly machine learning itself.

  • Personas such as 'Nick' (curators) and 'Shelley' (casual listeners) help segment skip behaviors and user needs effectively.

  • Editorial playlists blend human curation with ML personalization, balancing communal music experience and individual taste.

  • Contextual factors, including social and environmental considerations, greatly influence music listening behavior and signal interpretation.

  • User mental models around music and playlists differ significantly across personas and impact engagement and skipping.

  • User research teams at Spotify have grown significantly, highlighting the rising importance of research in ML-driven product development.

Notable Quotes

"Machine learning used to be backend technology, but now millions of consumer experiences are decided by machines or algorithms."

"These playlists have some element of machines recommending music in addition to editors behind the scenes."

"There’s been a fundamental shift in consumer products in the last five years driven by machine learning."

"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."

"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."

"Human behavior is complex and machine learning models need to understand those nuances."

"We consider machine learning as the third leg of mixed methods research, alongside data science and user research."

"Music is personal but also cultural and communal, which creates complexity for personalization."

"Personas like Nick and Shelley help us understand different user attitudes towards music listening and skipping."

"There hasn’t been a better time to be a user researcher, especially in machine learning enabled experiences."

Ask the Rosenbot
Sam Proulx
Mobile Accessibility and You
2022 • Design at Scale 2022
Gold
Catt Small
Moving from Execution to Strategy as a Designer
2022 • Design in Product 2022
Gold
Erin Malone
Understanding the past to prepare for the future
2024 • Rosenfeld Community
Kristen Honey
"Let’s Talk About Data and Crisis”: Public Digital Service Delivery = Open Data + Human Centered Design
2021 • Civic Design Community
Charlotte Vorbeck
Pipeline to Civic Design
2021 • Civic Design 2021
Gold
Brigette Metzler
Research Repositories: A global project by the ResearchOps Community
2020 • Advancing Research Community
Emily Eagle
Can't Rewind: Radio and Retail
2019 • Enterprise Experience 2019
Gold
Andreas Huebner
What Is It Like To Be Part of The UX Team at Compass?
2021 • Advancing Research 2021
Gold
Emily DiLeo
Stronger Together: Lessons Learned from UX Research Ops
2024 • DesignOps Summit 2024
Gold
Sheryl Cababa
Thinking in systems to address climate with Sheryl Cababa
2024 • Climate UX Interest Group
Brenna Fallon
Learning Over Outcomes
2019 • DesignOps Summit 2019
Gold
Michelle Bejian Lotia
Rolling Out a Repository: How Zapier Centralizes Insights from Across their Organization
2023 • Advancing Research 2023
Gold
Sheri Byrne-Haber
The Importance of Accessible Design Systems
2024 • DesignOps Summit 2020
Gold
Lada Gorlenko
Theme 1: Intro
2024 • Enterprise Experience 2020
Gold
Saara Kamppari-Miller
Theme Three Intro
2023 • DesignOps Summit 2023
Gold
Erika Kincaid
Connecting the Dots: How to Foster Collaboration and Build a Strong Design Review Culture
2022 • Design at Scale 2022
Gold

More Videos

Billy Carlson

"If you don’t know what should go in a specific area, it is really powerful to leave it as a question mark."

Billy Carlson

Tips to Utilize Wireframes to Tell an Effective Product Story

June 6, 2023

Alexis Lucio

"You can have a single page that’s got inaccessible, compliant, and accessible experiences all in one."

Alexis Lucio

Scaling Accessibility Through Design Systems

June 9, 2022

Sara Conklin

"You have to get super specific about the climate problem you want to solve and why it's important for you."

Sara Conklin

A UXer’s 12-Month Journey from Climate Concern to Climate Credibility

June 26, 2025

Prayag Narula

"Design research—they need each other. They are much, much better together."

Prayag Narula

How to Empower Your Designers to Do Good Research – And Why You Want To

June 10, 2022

Ryan Rumsey

"MBA programs teach a lot of networking and collaboration but not always the realities of day-to-day decisions."

Ryan Rumsey

Business Influence Without Losing Your Soul

January 14, 2021

Theresa Marwah

"We use Zoom waiting rooms to anonymize observers and create space for private consent conversations."

Theresa Marwah

How Atlassian is Operationalizing Respect in Research

February 27, 2020

Christian Crumlish

"Critique sessions work best when the topic is shaped so answers are relevant and help the process, especially by including business goals."

Christian Crumlish

AMA with Christian Crumlish, author of Product Management for UX People

March 24, 2022

Jemma Ahmed

"We need to create an army of design thinkers across the organization who can make better decisions on what to build and how to build it."

Jemma Ahmed

Theme 2 Intro

January 8, 2024

Suzan Bednarz

"Accessibility ops is like design ops but focused on accessibility, including accessible tools and vendor requirements."

Suzan Bednarz Hilary Sunderland

AccessibilityOps for All

January 8, 2024