Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Summary
Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.
Key Insights
-
•
Machine learning has shifted from backend tech to a frontline role shaping consumer experiences like music recommendations.
-
•
Spotify personalizes not just music content but also merchandising and UI elements like playlist covers and home screen layout.
-
•
Music preferences are deeply contextual, changing by time of day, activity, and listener persona.
-
•
The assumption that skipping a song is a purely negative signal is often incorrect; skip behavior varies by user goals and trust.
-
•
Mixed methods research at Spotify combines data science, qualitative user research, and now increasingly machine learning itself.
-
•
Personas such as 'Nick' (curators) and 'Shelley' (casual listeners) help segment skip behaviors and user needs effectively.
-
•
Editorial playlists blend human curation with ML personalization, balancing communal music experience and individual taste.
-
•
Contextual factors, including social and environmental considerations, greatly influence music listening behavior and signal interpretation.
-
•
User mental models around music and playlists differ significantly across personas and impact engagement and skipping.
-
•
User research teams at Spotify have grown significantly, highlighting the rising importance of research in ML-driven product development.
Notable Quotes
"Machine learning used to be backend technology, but now millions of consumer experiences are decided by machines or algorithms."
"These playlists have some element of machines recommending music in addition to editors behind the scenes."
"There’s been a fundamental shift in consumer products in the last five years driven by machine learning."
"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."
"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."
"Human behavior is complex and machine learning models need to understand those nuances."
"We consider machine learning as the third leg of mixed methods research, alongside data science and user research."
"Music is personal but also cultural and communal, which creates complexity for personalization."
"Personas like Nick and Shelley help us understand different user attitudes towards music listening and skipping."
"There hasn’t been a better time to be a user researcher, especially in machine learning enabled experiences."
Or choose a question:
More Videos
"Groupings and networks of cities create political cover and co-learning spaces necessary for innovation."
Jayne Engle Tanya Chung-Tiam-FookCivic Design for the Next Seven Generations—A Discussion on Sacred Civics
August 25, 2022
"The average salary in the Netherlands might be high because there are more senior participants skewing the data."
Marc FonteijnFirst Insights from the 2025 Service Design Salary(+) Report
December 4, 2024
"Research tells stories data alone cannot, helping stakeholders understand customer feelings."
Steve Portigal Alba Villamil Sam LadnerThe Future of Research: Bridging the Gaps
July 29, 2021
"It's not about going out to communities but changing your team and leadership to genuinely reflect those communities."
Tricia WangThe most popular design thinking strategy is BS
January 27, 2022
"Dynamic permission management and turning on SSO or SIEM help manage tools at scale without nano work."
Farid SabitovTheme Four Intro
September 9, 2022
"It’s about getting our stakeholders what they need, not just what they ask for."
Elena Naids Liza McRuerThe Power of Difficult Conversations: A Case Study on How We Introduced Design Ops in the Federal Government Space
October 2, 2023
"This circle represents the size of IBM, 385,000 IBMers. The CIO team is only 3% of that, and our design team is just 1% of the CIO."
Kristin WisnewskiMeasuring What Matters
October 23, 2019
"I’m on the listings team, where accuracy and speed of update are paramount because our agents rely on that information every day."
Caitlyn Hampton Monica Lee Jina YoonCompass 101: Growing Your Career In A Startup World
June 11, 2021
"We couldn’t even begin to demonstrate our team’s impact unless we could answer the question compared to what."
JD BuckleyCommunicating the ROI of UX within a large enterprise and out on the streets
June 14, 2018
