Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Dark Metrics: Illuminating the Negative Impact of Digital Health Design
Gold
Friday, March 12, 2021 • Advancing Research 2021
Share the love for this talk
Dark Metrics: Illuminating the Negative Impact of Digital Health Design
Speakers: Raven Veal
Link:

Summary

Traditional design metrics and KPIs are often geared towards measuring product success. Dark metrics challenge this paradigm by proactively measuring the unintended yet harmful psychological, social, and physical effects of our technologies. The examples within digital health are plentiful. From accelerating burnout among clinicians to widening racial disparities in quality of care, we can only reach the height of our most courageous solutions when we expose our deepest failures.

Key Insights

  • Traditional product metrics, like the Google HEART framework, often miss broader impacts on users, focusing narrowly on product success rather than holistic well-being.

  • Dark Metrics is a framework designed to measure negative unintended effects in digital health across four dimensions: disempowerment, exclusion, addiction, and distraction.

  • Disempowerment occurs when technology removes users’ autonomy, such as opaque black-box AI systems that undermine clinician or patient decision-making.

  • Exclusion can be subtle, as algorithms that proxy biased variables like healthcare cost can reproduce racial disparities without explicit intent.

  • Racial equity in design can be assessed using heuristics or rubrics co-created by diverse teams, as demonstrated by Raven’s IBM colleagues Dre Barbara, Sherees Cooper, and Morgan Foreman.

  • Addiction to technology is an overused concept in consumer health, but distinguishing healthy from excessive use requires linking engagement data to well-being measures.

  • Distraction from core tasks is common in clinical environments when new tech disrupts workflows, evidenced by studies with ER staff and clinical trial recruitment tools.

  • Ethics frameworks like the Institute for the Future’s Ethical OS help anticipate risks like surveillance, bias, and data control, which inform Dark Metrics design principles.

  • Engaging diverse stakeholders and including co-creation early in research helps uncover biases and unintended consequences before launch.

  • Addressing negative impacts requires transparency with clients and a strong ethical posture, even when business priorities may conflict with user protection.

Notable Quotes

"The traditional product metrics focus narrowly on the product or near-term impact but fail to capture what success means for the whole person."

"Within IBM Watson, we prefer the term augmented intelligence rather than artificial intelligence to emphasize support, not takeover."

"An AI algorithm that didn’t explicitly consider race still produced racial disparities by using healthcare costs as a proxy."

"I am a Black person, but I do not have every Black experience. Not having experienced something is not proof that it doesn’t exist."

"The difference between technology and slavery is that slaves are fully aware they are not free."

"Doctors want to help people, not be on a machine all day; many health technologies are more distracting than helpful."

"We can assess distraction by observing time spent on screens versus with patients, and self-reported mental effort and stress."

"It’s important to ask, before any new release, what are all the things that could possibly go wrong?"

"Our jobs are to protect users from harm. If clients don’t care about side effects, it may be necessary to draw a line and walk away."

"Storytelling is highly effective in helping stakeholders understand the complete performance of products, including the darker sides."

Ask the Rosenbot
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Christian Rohrer
Research Operations at Scale
2017 • DesignOps Summit 2017
Gold
Laura Klein
Human vs. machine: Testing AI’s ability to synthesize and analyze research
2026 • Advancing Research 2026
Conference
Iulia Cornigeanu
QuantQual Book Club: Small Data
2024 • QuantQual Interest Group
Sara Asche Anderson
Not Your Ordinary Re-Brand: Design's Path to Driving Customer Obsession at Best Buy
2024 • Enterprise Experience 2020
Gold
Steve Baty
Discussion
2016 • Enterprise UX 2016
Gold
Sarah Kinkade
Design Management Models in the Face of Transformation
2022 • Design at Scale 2022
Gold
Kate Towsey
The State of ResearchOps: More Than Just Theory
2019 • DesignOps Community
Amber Knabl
Empowering innovation: The critical role of inclusive product development in the AI era
2024 • Designing with AI 2024
Gold
Anna Nguyen
Why Our Voice of the Customer is Better Than Yours
2022 • Advancing Research 2022
Gold
David Cronin
Discussion
2015 • Enterprise UX 2015
Gold
Milan Guenther
A Shared Language for Co-Creating Ambitious Endeavours
2023 • Enterprise UX 2023
Gold
Joi Freeman
A New Vantage Point: Building a Pipeline for Multifaceted Research(ers)
2020 • Advancing Research 2020
Gold
Dem Gerolemou
Climate technology fundamentals
2024 • Climate UX Interest Group
David Cronin
The GE Design System and Thoughts about Craft at Scale
2015 • Enterprise UX 2015
Gold
Kyle Godbey
Non-linear service design for complex adaptive systems
2025 • Rosenfeld Community

More Videos

Daniel Gloyd

"Design research often acts like scientists with clipboards rather than humans in conversation."

Daniel Gloyd

Designing From the Inside Out: How Method Acting Can Inspire Design Research

February 12, 2026

Greg Petroff

"How can I guide my community to design the form and behavior of light on the edge of becoming this?"

Greg Petroff

Exit Interview #1: Greg Petroff: From Silicon Valley Executive to Sonoma County Possibilitarian

September 24, 2025

Dave Hora

"You can create a strategic capability profile of your team by mapping skills and desired growth across members."

Dave Hora

A Research Skills Evolution

March 10, 2021

John Taschek

"Success is about relationships: one person stopping you in the hallway to say hi is more valuable than many formal meetings."

John Taschek

Making People the X-Factor in the Enterprise

June 15, 2018

Christopher Taylor Edwards

"Pairing is two people with different roles doing an activity together simultaneously, like a driver and a navigator."

Christopher Taylor Edwards Valerie Roske

Design as a Team Practice, A Practical Guide to Cross-functional Collaboration

September 30, 2021

Changying (Z) Zheng

"Operations work best when practical tools are put in peoples' hands, not just theory."

Changying (Z) Zheng

Practical DesignOps: From Ideas to Tools That Teams Actually Use

September 25, 2025

Sam Proulx

"The question who are most people is confusing when applied to assistive technology users in usability scales."

Sam Proulx

SUS: A System Unusable for Twenty Percent of the Population

September 29, 2021

Amy Parness

"We created new design system components and icons so sustainability info could be inserted consistently across touchpoints."

Amy Parness

Scaling Sustainability: Complementary strategies that drive long-term success

May 29, 2025

Kayla Farrell

"Real estate data such as price per square foot varies widely by region, making system mapping challenging."

Kayla Farrell Chelsey Glasson Sean Fitzell Jared LeClerc

What It's Like To Be a User Researcher at Compass

March 12, 2021