Broad Strokes: Connecting Design, Research, and AI to the World Around Us
Summary
AI-based modeling plays an ever-increasing role in our daily lives. Our personal data and preferences are being used to build models that serve up anything from movie recommendations to medical services with a precision designed to make life decisions easier. But, what happens when those models are not precise enough or are unusable because of incomplete data, incorrect interpretations, or questionable ethics? What's more, how do we reconcile the dynamic and iterative nature of AI models (which are developed by a few) with the expansive nature of the human experience (which is defined by many) so that we optimize and scale UX for AI applications equitably? This talk will push our understanding of the sociotechnical systems that surround AI technologies, including those developing the models (i.e., model creators) and those for whom the models are created (i.e., model customers). What considerations should we keep top of mind? How might the research practice evolve to help us critique and deliver inclusive and responsible AI systems?
Key Insights
-
•
AI systems in healthcare often embed racial biases that exacerbate disparities, particularly impacting Black and Hispanic populations.
-
•
Algorithms based on past healthcare spending can unfairly dictate future levels of care, disadvantaging poorer patients.
-
•
AI hiring tools can perpetuate bias by training on historically discriminatory data, reducing trust among candidates.
-
•
Dark patterns in UI design, when combined with AI, may manipulate users into unintended decisions, especially vulnerable groups.
-
•
Deepfakes, created via AI neural networks, pose serious risks of misinformation, reputational harm, and psychological damage.
-
•
Facial recognition software shows significant racial and gender bias, performing worst on darker-skinned females as shown by Joy Buolamwini.
-
•
Biases in AI extend beyond technical factors to include systemic and human biases embedded in organizations and data collection.
-
•
Ethical failures like the Tuskegee Syphilis Study highlight the urgency of integrating ethics and do-no-harm principles into AI research and design.
-
•
Women and minorities remain vastly underrepresented in technical AI roles, limiting the diversity of perspectives in AI development.
-
•
Emerging regulations like GDPR and CPRA hold organizations accountable for ethical data use, reinforcing the responsibility of AI practitioners.
Notable Quotes
"Imagine going to the hospital for an emergency and the doctor doesn’t believe you’re in pain because AI says so."
"Algorithms that adjust care based on race don’t actually help and embed disparities instead."
"Dark patterns are not new in UX, but AI amplifies their impact by deceiving users in new ways."
"Deepfakes can disrupt democratic elections and inflict lasting psychological harm on victims."
"Biases out of sight are biases out of mind — facial recognition misidentifies Black faces leading to false arrests."
"All commercially available facial recognition software perform worse on darker females."
"Doing no harm is a core tenet of user experience research, whether or not you are trained in IRB processes."
"Only 12% of AI researchers globally are women, and 6% of professional software developers are women of color."
"Policy like the California Privacy Rights Act requires us to explain data use and comply with user deletion requests."
"The best transformation happens by asking the right questions, even when the problem feels overwhelming."
Or choose a question:
More Videos
"Sometimes it’s easier to say what DesignOps isn’t than what it is."
Dave Malouf Patrizia Bertini Jon FukudaThe Past, Present, and Future of DesignOps: a 2-part DesignOps Community Call (Part 2)
April 28, 2022
"The seventh generation principle calls for long-term decision making that transcends self-interest."
Jayne Engle Tanya Chung-Tiam-FookCivic Design for the Next Seven Generations—A Discussion on Sacred Civics
August 25, 2022
"We need more storytelling and positive use cases to help people see how AI affects their specific roles."
Frances Yllana Jorge Arango Maria Taylor Briana ThomasThe Big Question about Impact: A Panel Discussion
September 24, 2024
"For every 10 men promoted from entry to middle management, only 7.2 women are promoted—showing the broken rung problem."
Joi FreemanA New Vantage Point: Building a Pipeline for Multifaceted Research(ers)
March 30, 2020
"Stress makes us stupid because our creative thinking gets cloudy when under pressure."
Laura WeissTurn Down the Heat: 3 Ways to Handle Conflict in the Moment
November 20, 2024
"Integrate additional innovation skills like data analytics and behavioral economics to amplify the power of design."
Stephanie WadeBuilding and Sustaining Design in Government
December 8, 2021
"We need to start thinking about fluidity in identity because there is no 'this' anymore; people are multiracial, fluid in sexual identity, neurodiversity, and more."
Dave Malouf Meredith Black Farid SabitovThe Past, Present, and Future of DesignOps: a 2-part DesignOps Community Call (Part 1)
February 17, 2022
"Science cultures like CERN have a huge headwind against adopting design measurement tools because of skepticism."
Panel Discussion: Communicating the Value of DesignOps
November 7, 2018
"Data entry is the enemy; automation is the key to making research repositories successful."
Sofia QuinteroThe Product Philosophy Behind EnjoyHQ
March 10, 2021
Latest Books All books
Dig deeper with the Rosenbot
What strategies motivate cross-functional teams to input their research findings into a centralized repository?
How can researchers establish themselves as thought leaders promoting empathy across teams?
What are the core UX research tools essential for managing participant recruitment and study logistics in large companies?