Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Coexisting with AI: A practical guide for researchers to navigate tools, ethics, and integration
Gold
Tuesday, March 11, 2025 • Advancing Research 2025

This video is featured in the PDI 2026 playlist.

Share the love for this talk
Coexisting with AI: A practical guide for researchers to navigate tools, ethics, and integration
Speakers: Llewyn Paine
Link:

Summary

As culture and technology evolve, researchers must learn to co-exist with AI. Understanding its benefits, limitations, and ethical challenges is essential. This guide offers practical insights on top AI tools, advice on when and how to integrate AI into workflows, and a balanced view of the good, the bad and the ugly of the current AI landscape.

Key Insights

  • AI is shifting the authoritative voice in research from humans to generative models, challenging researchers’ traditional value.

  • Framing research value by sheer output risks commoditizing the field and losing long-term business impact.

  • Measuring research impact through concrete business outcomes is harder but essential for sustaining researcher relevance.

  • Research integrity is at risk if AI adoption isn't rigorously tested, as stakeholders expect researchers to model best practices.

  • AI tools can introduce biases especially affecting marginalized groups, impacting fairness in user research.

  • User researchers’ specialized skills in navigating complex trade-offs uniquely position them to integrate AI responsibly.

  • The lack of standardized methods for using AI in social science research makes learning from engineers and scientists critical.

  • Human data collection remains irreplaceable due to AI's weaknesses in understanding niche cultures and novel insights.

  • AI enables researchers to extend beyond study execution into prototyping, impact tracing, and direct product decision influence.

  • The rise of AI-powered 'triple threat' researchers combines research, product management, and design augmented by AI capabilities.

Notable Quotes

"Right now, there’s a fear of being replaced by AI, and while a lot of that’s overblown, researchers have a legitimate reason to feel uncomfortable."

"The choices we make about AI are about much more than speed versus quality — they define how we value research and our relationships."

"Volume is a losing game because we cannot outcompete full AI automation for sheer speed and quantity of insights."

"If we want our value to be defined as anything other than studies run, we must learn how to trace and articulate the actual business impact."

"If we are haphazard about AI adoption and trust marketers over rigorous testing, it looks like all our hand wringing about research accuracy was just gatekeeping."

"Generative AI is non-deterministic and doesn’t understand meaning, so baseline error rates exist in all tools we use."

"Prompting is a specialized skill, just like statistics or in-depth interviewing; the wrong wording can bias your results."

"More than a third of content on social media sites is AI generated, creating a training problem for new AI models."

"AI can empower researchers to solve pain points — like anticipating trends or critiquing discussion guides — amplifying impact rather than replacing work."

"We should think of AI as something that amplifies what we already are: if we think research is about impact, AI will help us understand and track it."

Ask the Rosenbot
Sean Fitzell
Craft of User Research: Building Out Jobs to be Done Maps
2021 • Advancing Research 2021
Gold
Karen McGrane
AI for Information Architects: Are the robots coming for our jobs?
2024 • Rosenfeld Community
Anupama Dhareshwar
From blueprint to bot: Designing resilient AI-powered services
2025 • Advancing Service Design 2025
Conference
Christian Rohrer
Research Operations at Scale
2017 • DesignOps Summit 2017
Gold
Jorge Arango
[Demo] How to re-categorize content at scale using LLMs
2024 • Designing with AI 2024
Gold
Matteo Gratton
Can Data and Ethics Live Together?
2021 • DesignOps Summit 2021
Gold
Ben Davies
Expert Panel: The Principles of Research Repository Design
2022 • Advancing Research 2022
Gold
Christian Crumlish
The Pygmalion Effect: In Which a Vibe Coding Experiment Becomes a Million Lines…
2025 • Rosenfeld Community
Jeff Gothelf
Innovation Studios: the Engines of Enterprise Experimentation
2015 • Enterprise UX 2015
Gold
Nick Cochran
Growing in Enterprise Design through Making Connections
2019 • Enterprise Community
Michelle Morrison
Practice What You Preach
2024 • DesignOps Summit 2020
Gold
Marina Martin
Lives on the Line: The Stakes of UX at the Scale of Government
2018 • Enterprise Experience 2018
Gold
Lija Hogan
Practical Principles of Inclusive Research
2023 • Advancing Research 2023
Gold
Jemma Ahmed
Redefining the research toolkit: Expanding methodologies for a changing world
2025 • Advancing Research 2025
Gold
Jon Fukuda
The Big Question about Innovation: A Panel Discussion
2024 • DesignOps Summit 2024
Gold
Dr. Karl Jeffries
The Science of Creativity for DesignOps
2024 • DesignOps Summit 2020
Gold

More Videos

Luca Rager

"Instead of overrides, we swap out entire theme files to customize components for different product teams."

Luca Rager

Empowering Gaming at Scale: How Xbox Builds Powerful, Automated, and Distributed Design Systems with Sketch

September 30, 2021

Sheryl Cababa

"Ethics is not just about compliance; it’s about doing the right thing holistically."

Sheryl Cababa Ethan Marcotte Milena Pribic

Day 2 Panel

June 5, 2024

Megan Blocker

"If you see a problem or opportunity, you should do something about it."

Megan Blocker

Getting to the “So What?”: How Management Consulting Practices Can Transform Your Approach to Research

March 26, 2024

Amy Parness

"People don’t come to the site just to read about ESG goals; they want customer-facing information they can apply."

Amy Parness

Scaling Sustainability: Complementary strategies that drive long-term success

May 29, 2025

Ilana Lipsett

"The internet as the self-regulating market is a failed experiment."

Ilana Lipsett

Anticipating Risk, Regulating Tech: A Playbook for Ethical Technology Governance

December 10, 2021

John Calhoun

"Hold your destination in mind. Have an intention to succeed. Break up the long tracks and brace for the hard parts."

John Calhoun

Have we Reached Our Peak? Spotting the Next Mountain For DesignOps to Climb

October 1, 2021

Ryan Matthew

"Even the most mature organizations tend to be on the nascent end of design ops maturity, and that leads to duplicated work and inconsistent user experiences."

Ryan Matthew Alex Kurchev

DesignOps without Boundaries: Building More with What You Have

September 10, 2025

Jackie Ajoux

"I like to think of what I do as doing UX for the UXers – applying design thinking to organizational problems."

Jackie Ajoux

Leveling-Up: A Single-Player’s Guide to the DesignOps Team-of-One

January 8, 2024

Eniola Oluwole

"We stopped talking about patterns and consistency as it made me the ‘patterns guy’—instead, I shifted language to what different roles cared about."

Eniola Oluwole

Lessons From the DesignOps Journey of the World's Largest Travel Site

October 24, 2019