Anatomy of a Strong User Panel
Summary
In this session, Sydney Lawson, Senior Product Marketing Manager at Rally, dives deep into real user panels—tearing them up, not down—to uncover what top research teams are doing well and what’s worth stealing for your own panel. From sign-up to participation, she’ll break down what makes panels effective and how to optimize every step of the experience.
Key Insights
-
•
Every successful user panel has a dedicated landing page with a clear call to action and explanation of participation benefits.
-
•
Keeping screener questions between five to ten helps maintain low friction and improves sign-up rates.
-
•
Using single-select, closed-ended questions for job functions avoids messy and unusable data compared to open-text fields.
-
•
Including questions about AI usage and interest helps target relevant participants for specialized research.
-
•
Large companies like Google can afford more extensive screener forms with conditional logic due to their broad user base and resource availability.
-
•
Separate FAQ pages linked from the landing page reduce clutter and provide potential panelists with detailed sign-up information.
-
•
Panel engagement improves when participants see study results and understand how their feedback influences product decisions.
-
•
Creative incentive models like digital badges (from Outsystems) can gamify participation when budgets for monetary rewards are limited.
-
•
Including fake tools or options in screening questions (Genesys example) helps identify and filter out insincere panelists motivated only by incentives.
-
•
Tracking participant location enables compliance with regional incentive regulations and data privacy laws.
Notable Quotes
"Every single one of these companies has a landing page on their marketing website with a call to action to sign up."
"The most successful way to ask about job function is a single-select question, not an open field."
"We love to see questions around AI and even asking if participants want to join AI studies specifically."
"Incentives are a benefit but should not be the number one reason people join a panel."
"Google’s sign-up has a progress bar which is really helpful so participants know how far they are along."
"If you ask accessibility questions, it’s so important to actually respect the answers."
"Outsystems gamified their research panel with badges because they didn’t have budget for monetary incentives."
"Genesys includes a fake tool in the software list to stop incentive-driven participants from checking all boxes."
"You want your panel to be representative of your customer base, not exactly all your customers."
"Sharing the results of studies with panelists gets them excited about contributing and keeps them engaged."
Or choose a question:
More Videos
"One ops person to five researchers seems pretty legit, according to Kate Towsie."
Brigette MetzlerScaling ResearchOps: Helping Researchers do Their Best Work
March 30, 2020
"In a chat interface, you’re able to fill in the information that you missed and iterate on the output as it’s being generated."
Trisha Causley[Demo] Complexity in disguise: Crafting experiences for generative AI features
June 5, 2024
"Communication matters—sending an email to an executive with spelling mistakes can ruin your respect."
Ian SwinsonDesigning and Driving UX Careers
June 8, 2016
"We see AI as a tool that can quickly sketch ideas and ask thoughtful questions to take designs to the next level."
Louis Rosenfeld Billy Carlson Jon Fukuda Maria TaylorHow AI will Change DesignOps Tooling
October 3, 2023
"The silver bullet for all things UX is talking to real customers."
Leah Buley Joe NatoliAsk Me Anything with Leah Buley and Joe Natoli, co-authors of The User Experience Team of One (2nd edition)
October 8, 2024
"We’re not looking to attack individuals, but to attack these systems."
Spencer L. A. StultzWhy Social Justice Frameworks are Necessary for Successful DEI/JEDI Initiatives
October 4, 2023
"Bringing data science into the core product process is like adding a fourth leg to the stool — design, development, product management, and data together create delight."
Chris ChapoData Science and Design: A Tale of Two Tribes
May 13, 2015
"We are living in a world that for so many feels like it has been flipped on its head."
Jemma AhmedTheme 2 Intro
March 10, 2022
"Travel is impactful to let your remote teammates experience your context, which helps build empathy."
Jilanna WilsonDistributed DesignOps Management
February 26, 2019
Latest Books All books
Dig deeper with the Rosenbot
What advice do Julia and Clay give for transitioning into product management from design or other adjacent roles?
In what ways do rough sketches invite iteration rather than defensiveness compared to polished diagrams?
What pitfalls should designers avoid when using AI to get unstuck on complex design problems?