Managing the Human Engagement Risks of AI
This video is featured in the Designing with AI 2025 playlist and 1 more.
Summary
AI is fast transforming the way companies operate and design products and services. New design practices like “vibe coding,” or “prompt engineering,” are accelerating product launches at speed and scale. But with the rush to market; how can teams maintain their commitment to human-centered design, minimize the risks to consumers and keep up with the increase in developer velocity? We showcase various tactics companies are using to manage the risk of implementing AI into their products while keeping their ever tightening GTM timelines.
Key Insights
-
•
Generative AI does not think or understand; it excels at pattern recognition and filling in blanks like a souped-up mad lib.
-
•
AI design requires a paradigm shift from building products to designing dynamic behaviors within multi-agent ecosystems.
-
•
UX designers must move beyond traditional user-device interaction design to address machine learning's stochastic and contextual outputs.
-
•
Generative AI models have no moral code and can generate harmful outputs without human-guided guardrails and feedback loops.
-
•
Red teaming, adversarial testing, and safety classifiers are essential tools to proactively identify and mitigate AI risks.
-
•
Data and model cards documenting dataset provenance and model training are crucial for auditing bias and risk in AI systems.
-
•
Automation bias can cause humans to overtrust AI outputs, requiring design strategies that clearly signal uncertainty and keep users in control.
-
•
Machine learning models can unlearn harmful behaviors given correct feedback, but timely monitoring is critical before embedded biases solidify.
-
•
Mindful AI design entails co-design, simulations of multiple futures, and iterative testing with diverse users to understand AI behavior in context.
-
•
UX professionals hold unique responsibility and agency to embed ethical, equitable practices in AI design and deployment.
Notable Quotes
"Generative AI is a souped up mad lib, its job is to fill in the blank."
"Good UX and good design need not only to make a comeback but lead the way in this era of AI."
"In the post software era, we have to think about how products will behave, not just how they are built."
"The user and machine can both be in control; technology affordance changes over time and outcomes are dynamic."
"Generative AI has no moral code; you have to discipline it so it can learn right from wrong."
"There is no unbiased creation of machine learning or artificial intelligence; the idea of agnostic algorithms is a myth."
"You have to build guardrails and test models with red teaming to ensure they behave ethically and safely."
"Minimize anthropomorphism in AI design so users know they are engaging with machines, not humans."
"Machine learning models can unlearn behaviors, but we need robust feedback loops to detect when they go awry."
"The more that you engage with AI technology, the better confidence you will have in designing with it."
Or choose a question:
More Videos
"DeepFakes can look too realistic and might introduce new privacy risks if donor faces come from real people."
Llewyn Paine[Demo] Deploying AI doppelgangers to de-identify user research recordings
June 5, 2024
"Safety is your number one concern, privacy is your number two, and usability is number three."
Daniel J. RosenbergDigital Medicine Design
September 26, 2019
"We use a structured rubric that evaluates expected impact and how research will be used to inform decisions."
Chris Geison Cristen Torrey Eric MahlstedtWhat is Research Strategy?: A Panel of Research Leaders Discuss this Emergent Question
March 4, 2021
"Jobs to be done gives us a normalized, structured, and stable unit of analysis to then add emotion, brand, and price on top."
Jim KalbachJobs To Be Done
February 25, 2021
"Data quality involves accuracy, completeness, uniqueness, timeliness, and representativeness—not just error reduction."
Caroline JarrettGarbage in, garbage out? Measuring error rates to get ready for AI
January 8, 2026
"Can you testify right now that all your digital properties are accessible to disabled users?"
Suzan Bednarz Hilary SunderlandAccessibilityOps for All
January 8, 2024
"Over-enforced design system adoption bred resentment and low morale, despite good adoption metrics."
Rachael Greene Alison DavisBuilding a Design Ops Practice that Really Works (Most of the Time)
October 2, 2025
"You have to deliberately quiet that noise in your head and focus on humans in a world of humans."
Steve Portigal Susan Simon-Daniels Tamara Hale Randolph Duke IIWar Stories LIVE! Q&A-Discussion
March 30, 2020
"We tend to do anything but inquire, even though inquiry is most effective for sustainable change."
Laura WeissThere is No Playbook: Leader as Coach During Challenging Times
April 26, 2024
Latest Books All books
Dig deeper with the Rosenbot
Why is prior authorization such a persistent problem and how can patient-centered solutions be designed around it?
In what ways can AI tools accelerate value delivery in design operations, and what pitfalls should be avoided?
What HIPAA-compliant platforms are recommended for quickly developing healthcare apps?