Rosenverse
Global Research Panels

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Global Research Panels

Saturday, August 8, 2020 • DesignOps Community
Share the love for this talk
Global Research Panels
Speakers: Wyatt Hayman
Link:

Summary

In this talk, Wyatt conducts a deep dive into the Airbnb Global Research Panels, the program he has been developing at Airbnb over the past two years. Launch: why Airbnb created the Global Research Panels, the key decisions and the internal alignment required for launching the first version of the Global Research Panel Process: detailed look into the current version of the Global Research Panels, including the mechanics and tools used to manage it Impact: demonstration of ways the Global Research Panels have provided value over the past two years at Airbnb Evolution: see how the Global Research Panels has grown through six iterations so far and the vision for the Global Research Panels program going forward

Key Insights

  • Airbnb’s research panels focus on qualitative, not statistically significant, feedback from global users across 60 countries in 11 languages.

  • The panels were created to solve the gap of quick access to non-English speakers for product research, as most researchers operate primarily in English.

  • Early pilot panels included a mostly English-speaking group in Sydney and a non-English pilot in Tokyo.

  • Participants commit to six months with a reward equivalent to $400 US purchasing power, adjusted per country, paid in two parts and contingent on participation.

  • Qualtrics’ Target Audience product was chosen for panel management due to existing relationships and capabilities.

  • Human translation for surveys and responses is preferred to maintain quality, though machine translation is used occasionally for urgency.

  • Panels evolved from complex mechanisms with prototypes and online discussions to simpler survey and remote interview formats to improve operational efficiency.

  • Smaller panel sizes (~90 per market) allow for qualitative insights, focusing on proportional representation aligned to Airbnb’s user base in each market.

  • Non-research teams at Airbnb have started using the panels for quick feedback without needing to recruit or handle NDAs themselves.

  • Future ambitions include expanding live multilingual interviews and integrating more diversity and accessibility data into participant profiles.

Notable Quotes

"Our research operations team’s mission is to let researchers focus on research without worrying about operational details."

"Half of Airbnb’s traffic is from non-English devices and 80% of bookings involve at least one party outside the U.S."

"We ended up piloting a group in Tokyo for non-English speakers, which was key to testing the language challenge."

"Participants are paid based on purchasing power parity — $400 USD equivalent spread over six months for active participation."

"We decided to limit panel access initially to the research team but are exploring expanding to other teams."

"We run a two-week cycle now because translations back and forth require more time for quality."

"The panels are primarily qualitative tools and not meant for statistically significant quantitative research."

"We send out surveys weekly, but respondents have two weeks to answer, accommodating different time zones and bandwidth."

"The panels help researchers test assumptions, check cultural sensitivities, and quickly iterate globally."

"Non-research teams can use the panels easily — one program manager arranged interviews with hosts in 6 countries within a week."

Ask the Rosenbot
Jacqui Frey
Scale is Social Work
2020 • DesignOps Community
Robin Beers
Beyond Insights: Researchers as Organizational Change Catalysts
2024 • Advancing Research 2024
Gold
John Taschek
Making People the X-Factor in the Enterprise
2018 • Enterprise Experience 2018
Gold
Harry Max
Priority Zero: Some Things are More Equal than Others
2016 • Enterprise UX 2016
Gold
Irina Tikhonova
Small Wins, Big Impact: Leveraging and Elevating User Engagement
2021 • Civic Design 2021
Gold
Peter Van Dijck
Designing AI-first products on top of a rapidly evolving technology
2025 • Designing with AI 2025
Gold
Steve Baty
Breaking Out of Ruts: Tips for Overcoming the Fear of Change
2016 • Enterprise UX 2016
Gold
Andy Barraclough
From Costly Complexity to Efficient Insights: Why UX Teams Are Switching To Voxpopme
2024 • DesignOps Summit 2024
Gold
Maria Rosala
AI-Assisted Qualitative Analysis: What to Automate and What to Own
2026 • Rosenfeld Community
Katie Hansen
Finding the unknown in the known: Harnessing meta-analysis and literature review
2025 • Advancing Research 2025
Gold
Chris Geison
Theme Two Intro
2023 • Advancing Research 2023
Gold
Matt Bernius
Trauma-informed Research: A Panel Discussion
2021 • Advancing Research Community
Sean Dolan
A Practical Look at Creating More Usable Enterprise Customer Journeys
2019 • Enterprise Community
PJ Buddhari
Meet Spectrum, Adobe’s Design System
2021 • Design at Scale 2021
Gold
Sha Hwang
The First Fifty Years of Civic Design
2022 • Civic Design 2022
Gold
Ashley Cortez
Shifting Toward Community-Led Innovation in Local Government
2021 • Civic Design 2021
Gold

More Videos

Natalie Dunbar

"Creating frameworks helps identify responsibilities, dependencies, and critical handoffs across disciplines."

Natalie Dunbar

DesignOps and Content Strategy: Envisioning the Future Together

October 1, 2021

Andrea Gallagher

"Tools like Mural are great for remote real-time sticky note brainstorming and affinity mapping."

Andrea Gallagher

The Problem Space

May 16, 2019

Deanna Mitchell

"Conversational analysis is not about replacing human understanding, it’s about amplifying it."

Deanna Mitchell

Designing with culture: Unlocking impactful insights for Product and UX

March 12, 2025

Meghan Hellstern

"Emergence means that something new arises through simple interactions; we design interactions so systems find their own desirable solutions."

Meghan Hellstern Joanne Dong

The Next 100 Years of Civic Design: How Might We Better Rise to Meet the Challenges of Today and Tomorrow?

December 10, 2021

Smitha Papolu

"The value of research is magnified when we democratize it and let others do research with us and on their own."

Smitha Papolu Nova Wehman-Brown Melissa Schmidt Adam Menter

Theme 3 Discussion

June 4, 2019

Robin Beers

"The executive’s interpretation was completely out of line with the customers’ realities."

Robin Beers

Beyond Insights: Researchers as Organizational Change Catalysts

March 25, 2024

Cennydd Bowles

"People remember how you made them feel, and that’s something AI can fake but not truly replicate."

Cennydd Bowles Dan Rosenberg Lisa Welchman

Day 1 Panel

June 4, 2024

Sam Proulx

"You may need to operate the prototype on behalf of users whose assistive tech can’t click or interact with clickable prototypes."

Sam Proulx

Prototype Reviews, People With Disabilities, and You

December 8, 2021

Dan Hill

"Technology affects cities directly and profoundly, shaping how we live, work, and play."

Dan Hill

Designing for the infrastructures of everyday life

June 4, 2024