Summary
Join us for a deep dive into testing and experimentation tools at the UXR Tools Summit. Design for Impact author Erin Weigel will lead a broad discussion on the category, and will also be joined by UXtweak, who will reveal their vision for the future of experimentation, showcase innovative capabilities. Ideal for UX, design, and market researchers eager to explore how emerging tools are transforming decision-making, optimization, and evidence-based product design.
Key Insights
-
•
AI-generated follow-up questions in usability tests do not uncover novel user issues as effectively as human moderators.
-
•
Participants tend to get frustrated with AI moderators due to repetitive or obvious questioning.
-
•
Context is crucial for effective research moderation, which AI currently lacks.
-
•
Synthentic participants are useful for brainstorming but cannot replace real user research.
-
•
Democratization of research increases access but risks diluting research quality without expert involvement.
-
•
AI features that save researchers time by handling repetitive tasks are promising and more practical.
-
•
AI moderation should operate as a human-in-the-loop assistant, not a full replacement.
-
•
Bias in AI and lack of diversity representation remains a critical challenge currently unaddressed.
-
•
UX Tweak focuses on researcher-centered tools, expert support, and end-to-end UX research platform.
-
•
There is healthy skepticism about AI fully simulating unpredictable human behavior in usability testing.
Notable Quotes
"Testing tools can help us bring the body language of the internet in front of us to see what people are doing."
"AI cannot substitute the research expertise achieved through multiple interviews and synthesis by humans."
"There was no meaningful difference between AI follow-up questions and static researcher questions in uncovering usability issues."
"AI follow-up questions led to frustrated participants because it asked highly obvious questions and ignored detailed context."
"If all tools try to be for everyone, they risk serving no one properly."
"AI should be a force multiplier handling repetitive tasks, not a replacement for human judgment."
"Synthetic participants shouldn't be used as a methodology but are great for cold starting research."
"Human moderation will always have a place because humans are unpredictable and random."
"We never train AI models on your data, and all AI features are strictly opt-in."
"There is a huge space for improvement in AI features for UX research if these have any future."
Or choose a question:
More Videos
"We often don’t prototype everything ourselves; instead, we role play with client team members to simulate complex organizational aspects."
Milan Guenther Benjamin KumpfThe $212 billion ‘so what?’: unlocking impact in development cooperation
November 20, 2025
"Always be getting feedback, even informally over coffee, or sending out surveys twice a year to improve and mature relationships."
Alnie FigueroaTeamwork: Strategies for Effective Collaboration with Other Program Management Teams
September 8, 2022
"Our KSPs do not change year over year because changing them frequently is disruptive for the business direction."
Saara Kamppari-Miller Nicole Bergstrom Shashi JainKey Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
November 13, 2024
"I wrote the book because people have a lot of questions about how to become a product manager or how to work with product managers or what is product."
Christian CrumlishIntroduction by our Conference Chair
December 6, 2022
"Working with AI is a lot less like working with another human and more like working with some weird force of nature."
Helen ArmstrongAugment the Human. Interrogate the System.
June 7, 2023
"We were educated to minimize the value of our own practice and just reproduce what was done elsewhere."
Verónica Urzúa Jorge MontielThe B-side of the Research Impact
March 12, 2021
"Knowing in action is the intuition practiced by competent designers, a core of artistry."
Yoel SumitroActions and Reflections: Bridging the Skills Gap among Researchers
March 9, 2022
"Instead of playing the melody Miles Davis wrote, the soloist creates a melody spontaneously."
Jim KalbachJazz Improvisation as a Model for Team Collaboration
June 4, 2019
"Sometimes you’re just laying the foundational bricks so an idea or process can get off the ground."
Bud Caddell Kristin Skinner Alana WashingtonDesignOps Community Sensing Session
May 13, 2021
