Rosenverse

Testing and Experimentation Tools

Gold
Thursday, March 12, 2026 • Advancing Research 2026

This video is featured in the UX metrics & analytics playlist.

Share the love for this talk
Testing and Experimentation Tools
Speakers: Erin Weigel and Petra Rajkov
Link:

Summary

Join us for a deep dive into testing and experimentation tools at the UXR Tools Summit. Design for Impact author Erin Weigel will lead a broad discussion on the category, and will also be joined by UXtweak, who will reveal their vision for the future of experimentation, showcase innovative capabilities. Ideal for UX, design, and market researchers eager to explore how emerging tools are transforming decision-making, optimization, and evidence-based product design.

Key Insights

  • AI-generated follow-up questions in usability tests do not uncover novel user issues as effectively as human moderators.

  • Participants tend to get frustrated with AI moderators due to repetitive or obvious questioning.

  • Context is crucial for effective research moderation, which AI currently lacks.

  • Synthentic participants are useful for brainstorming but cannot replace real user research.

  • Democratization of research increases access but risks diluting research quality without expert involvement.

  • AI features that save researchers time by handling repetitive tasks are promising and more practical.

  • AI moderation should operate as a human-in-the-loop assistant, not a full replacement.

  • Bias in AI and lack of diversity representation remains a critical challenge currently unaddressed.

  • UX Tweak focuses on researcher-centered tools, expert support, and end-to-end UX research platform.

  • There is healthy skepticism about AI fully simulating unpredictable human behavior in usability testing.

Notable Quotes

"Testing tools can help us bring the body language of the internet in front of us to see what people are doing."

"AI cannot substitute the research expertise achieved through multiple interviews and synthesis by humans."

"There was no meaningful difference between AI follow-up questions and static researcher questions in uncovering usability issues."

"AI follow-up questions led to frustrated participants because it asked highly obvious questions and ignored detailed context."

"If all tools try to be for everyone, they risk serving no one properly."

"AI should be a force multiplier handling repetitive tasks, not a replacement for human judgment."

"Synthetic participants shouldn't be used as a methodology but are great for cold starting research."

"Human moderation will always have a place because humans are unpredictable and random."

"We never train AI models on your data, and all AI features are strictly opt-in."

"There is a huge space for improvement in AI features for UX research if these have any future."

Ask the Rosenbot
Robin Beers
Panel: Excellence in Communicating Insights
2024 • Advancing Research 2024
Gold
JP Allen
Navigating the UX Tool Landscape
2021 • Advancing Research 2021
Gold
Julie Norvaisas
Back to basics, or start from scratch?
2025 • Advancing Research 2025
Gold
Milan Guenther
The $212 billion ‘so what?’: unlocking impact in development cooperation
2025 • Advancing Service Design 2025
Gold
Dianne Que
Real Talk: Proving Value through a Scrappy Playbook
2019 • DesignOps Summit 2019
Gold
Marissa Cui
Climate Design Product Showcase
2024 • Climate UX Interest Group
John Donmoyer
Shipping your code generation experiments to production
2025 • Designing with AI 2025
Gold
Sean Fitzell
Craft of User Research: Building Out Jobs to be Done Maps
2021 • Advancing Research 2021
Gold
Steve Sanderson
Discussion
2015 • Enterprise UX 2015
Gold
Mila Kuznetsova
How Lessons Learned from Our Youngest Users Can Help Us Evolve our Practices
2022 • Advancing Research 2022
Gold
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Bud Caddell
DesignOps Community Sensing Session
2021 • DesignOps Community
Husani Oakley
Theme Two Intro
2023 • Enterprise UX 2023
Gold
Maria Skaaden
Continuous Design: One eye on the horizon and the other on the next wave
2018 • DesignOps Summit 2018
Gold
Frances Yllana
Theme 2 Intro
2024 • DesignOps Summit 2024
Gold
Luca Rager
Empowering Gaming at Scale: How Xbox Builds Powerful, Automated, and Distributed Design Systems with Sketch
2021 • DesignOps Summit 2021
Gold

More Videos

Daniel Gloyd

"Design is often framed as problem solving, but conversation is a better framework to respect users as active experts."

Daniel Gloyd

Warming the User Experience: Lessons from America's first and most radical human-centered designers

May 9, 2024

Aletheia Delivre

"AI and automation have meaningfully raised the bar for how I operate as an ops."

Aletheia Delivre

New Shapes and Emerging Identities for Design Ops

September 11, 2025

Dr. Jamika D. Burge

"Being radically honest about who is in the research helps avoid sweeping generalizations that exclude marginalized groups."

Dr. Jamika D. Burge Mansi Gupta

Advancing the Inclusion of Womxn in Research Practices

September 15, 2022

Peter Van Dijck

"User testing for AI still means watching people use it, plus collecting analytics and running evaluations."

Peter Van Dijck

Designing AI-first products on top of a rapidly evolving technology

June 10, 2025

Ricardo Martins

"Sixty-one factors influence students’ decisions when choosing a college—that’s a lot more than you might think."

Ricardo Martins

Unlocking the power of advanced quantitative methods

March 12, 2025

Emily Danielson

"Disasters aren’t just big front page events, they include minor crises like loss of a loved one or housing insecurity."

Emily Danielson

“I mean, I can lift a shovel”: Design Skills in Disaster Response

June 9, 2022

Bria Alexander

"There is no need to take notes. Our session notes, resources, videos, and slide decks will all be shared on the website."

Bria Alexander

Opening Remarks

March 29, 2023

Jerome “Axle” Brown

"Sending reports early and inviting comments turns the insight session into more of a discussion than a presentation."

Jerome “Axle” Brown

How to Use Self-Directed Learning to Ensure Your Research Insights are Heard and Acted Upon

March 11, 2021

Sarah Williams

"Our CMO incorporated our experience principles into the brand strategy, which was a huge and unexpected boost."

Sarah Williams

A Framework for CX Transformation

June 11, 2021