Testing and Experimentation Tools
This video is featured in the UX metrics & analytics playlist.
Summary
Join us for a deep dive into testing and experimentation tools at the UXR Tools Summit. Design for Impact author Erin Weigel will lead a broad discussion on the category, and will also be joined by UXtweak, who will reveal their vision for the future of experimentation, showcase innovative capabilities. Ideal for UX, design, and market researchers eager to explore how emerging tools are transforming decision-making, optimization, and evidence-based product design.
Key Insights
-
•
AI-generated follow-up questions in usability tests do not uncover novel user issues as effectively as human moderators.
-
•
Participants tend to get frustrated with AI moderators due to repetitive or obvious questioning.
-
•
Context is crucial for effective research moderation, which AI currently lacks.
-
•
Synthentic participants are useful for brainstorming but cannot replace real user research.
-
•
Democratization of research increases access but risks diluting research quality without expert involvement.
-
•
AI features that save researchers time by handling repetitive tasks are promising and more practical.
-
•
AI moderation should operate as a human-in-the-loop assistant, not a full replacement.
-
•
Bias in AI and lack of diversity representation remains a critical challenge currently unaddressed.
-
•
UX Tweak focuses on researcher-centered tools, expert support, and end-to-end UX research platform.
-
•
There is healthy skepticism about AI fully simulating unpredictable human behavior in usability testing.
Notable Quotes
"Testing tools can help us bring the body language of the internet in front of us to see what people are doing."
"AI cannot substitute the research expertise achieved through multiple interviews and synthesis by humans."
"There was no meaningful difference between AI follow-up questions and static researcher questions in uncovering usability issues."
"AI follow-up questions led to frustrated participants because it asked highly obvious questions and ignored detailed context."
"If all tools try to be for everyone, they risk serving no one properly."
"AI should be a force multiplier handling repetitive tasks, not a replacement for human judgment."
"Synthetic participants shouldn't be used as a methodology but are great for cold starting research."
"Human moderation will always have a place because humans are unpredictable and random."
"We never train AI models on your data, and all AI features are strictly opt-in."
"There is a huge space for improvement in AI features for UX research if these have any future."
Or choose a question:
More Videos
"The stories you hear inside an organization are like its soul."
Laurent ChristophScale the impact of DesignOps in 3D: Diligence, Decision, Discipline
September 17, 2025
"The more I work with evals, the more I think UX and product people need to be involved because of the need for diverse perspectives."
Peter Van DijckHands-on AI #2: Understanding evals: LLM as a Judge
October 15, 2025
"You’re in the right spot, and that’s the right instinct to be building."
Eric ShumakeAn AMA on UX's Role in Healthcare
April 22, 2026
"If you told me about a conference like this 30 years ago, you could have gotten all the participants in the backseat of a minivan."
Bob BaxleyTheme 4: Intro
January 8, 2024
"Journalists experiencing the same trauma as samurai during the Meiji Restoration – once warriors, now unsure of their role."
Patrick Boehler Madison KarasThe service shift: transforming media organizations to create real value through design
November 19, 2025
"Users have more than one mental model; they can be constructed on the fly depending on context."
Dane DeSutter Stephanie ScopelitisWhat co-speech gestures reveal about users’ thinking during interviews
June 30, 2023
"Partnership is everything; to operationalize respect, legal, privacy, procurement, and research must collaborate."
Theresa MarwahHow Atlassian is Operationalizing Respect in Research
February 27, 2020
"Rolling out a research repository is complex but crucial to avoid duplicate research and make insights discoverable."
Michelle Bejian Lotia Anne-Marie MorellRolling Out a Repository: How Zapier Centralizes Insights from Across their Organization
March 28, 2023
"If you’re a Theory X manager, you believe work is something people hate and must be controlled."
Sam LadnerData Exhaust and Personal Data: Learning from Consumer Products to Enhance Enterprise UX
June 8, 2016
