Summary
Join us for a deep dive into testing and experimentation tools at the UXR Tools Summit. Design for Impact author Erin Weigel will lead a broad discussion on the category, and will also be joined by UXtweak, who will reveal their vision for the future of experimentation, showcase innovative capabilities. Ideal for UX, design, and market researchers eager to explore how emerging tools are transforming decision-making, optimization, and evidence-based product design.
Key Insights
-
•
AI-generated follow-up questions in usability tests do not uncover novel user issues as effectively as human moderators.
-
•
Participants tend to get frustrated with AI moderators due to repetitive or obvious questioning.
-
•
Context is crucial for effective research moderation, which AI currently lacks.
-
•
Synthentic participants are useful for brainstorming but cannot replace real user research.
-
•
Democratization of research increases access but risks diluting research quality without expert involvement.
-
•
AI features that save researchers time by handling repetitive tasks are promising and more practical.
-
•
AI moderation should operate as a human-in-the-loop assistant, not a full replacement.
-
•
Bias in AI and lack of diversity representation remains a critical challenge currently unaddressed.
-
•
UX Tweak focuses on researcher-centered tools, expert support, and end-to-end UX research platform.
-
•
There is healthy skepticism about AI fully simulating unpredictable human behavior in usability testing.
Notable Quotes
"Testing tools can help us bring the body language of the internet in front of us to see what people are doing."
"AI cannot substitute the research expertise achieved through multiple interviews and synthesis by humans."
"There was no meaningful difference between AI follow-up questions and static researcher questions in uncovering usability issues."
"AI follow-up questions led to frustrated participants because it asked highly obvious questions and ignored detailed context."
"If all tools try to be for everyone, they risk serving no one properly."
"AI should be a force multiplier handling repetitive tasks, not a replacement for human judgment."
"Synthetic participants shouldn't be used as a methodology but are great for cold starting research."
"Human moderation will always have a place because humans are unpredictable and random."
"We never train AI models on your data, and all AI features are strictly opt-in."
"There is a huge space for improvement in AI features for UX research if these have any future."
Or choose a question:
More Videos
"We turned all of these files in boxes into an Excel file, and then into a shared Google Doc accessible to multiple people."
Emily Danielson“I mean, I can lift a shovel”: Design Skills in Disaster Response
June 9, 2022
"Whatever you were a year ago, whatever you believe design ops looks like today, don't let those be moment prisons for you."
Jon FukudaTheme One Intro
October 2, 2023
"After swapping our remote meeting solution in early 2020, ESAT went from 69% to 92%, a huge improvement."
Adel Du ToitGet Your CFO To Say: 'Our Strategic Goal is User Obsession'
June 10, 2022
"What is in a name in a title is not much. It’s more of a mad libs plug and play where every title and role word can be swapped out."
Abbey SmalleyToday’s Design Ops and Programs Landscape & Career Paths
October 4, 2023
"Users are smart, innovative, and a lot of the innovation actually comes from users rather than big companies."
Prayag Narula Rida QadriHCI 2.0: Humanity Deserves the Attention that UX Research has to Offer
March 28, 2023
"Noba submitted my name without asking me to present today, which shows how much we've grown to trust each other."
Nova Wehman-BrownWe've Never Done This Before
June 4, 2019
"Manhole cover questions belong in the past. Focus on real skills and relevant experience instead."
Russ UngerOnboarding: The Ecosystem, not the Afterthought
November 7, 2017
"Inclusion and accessibility are really core and important to what we do in design ops."
Saara Kamppari-MillerTheme Three Intro
October 4, 2023
"Inclusion, collaboration, and iteration are the three pillars of how we work with authors to make a book."
Louis RosenfeldCoffee with Lou: Should You Write a (UX) Book?
March 7, 2024
Latest Books All books
Dig deeper with the Rosenbot
What HIPAA-compliant platforms are recommended for quickly developing healthcare apps?
How does Rally’s research infrastructure facilitate participant management and integration with enterprise data systems?
What hidden costs should be accounted for beyond license fees when budgeting for research tools?