Rosenverse
Dark Metrics: Illuminating the Negative Impact of Digital Health Design

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Dark Metrics: Illuminating the Negative Impact of Digital Health Design

Gold
Friday, March 12, 2021 • Advancing Research 2021
Share the love for this talk
Dark Metrics: Illuminating the Negative Impact of Digital Health Design
Speakers: Raven Veal
Link:

Summary

Traditional design metrics and KPIs are often geared towards measuring product success. Dark metrics challenge this paradigm by proactively measuring the unintended yet harmful psychological, social, and physical effects of our technologies. The examples within digital health are plentiful. From accelerating burnout among clinicians to widening racial disparities in quality of care, we can only reach the height of our most courageous solutions when we expose our deepest failures.

Key Insights

  • Traditional product metrics, like the Google HEART framework, often miss broader impacts on users, focusing narrowly on product success rather than holistic well-being.

  • Dark Metrics is a framework designed to measure negative unintended effects in digital health across four dimensions: disempowerment, exclusion, addiction, and distraction.

  • Disempowerment occurs when technology removes users’ autonomy, such as opaque black-box AI systems that undermine clinician or patient decision-making.

  • Exclusion can be subtle, as algorithms that proxy biased variables like healthcare cost can reproduce racial disparities without explicit intent.

  • Racial equity in design can be assessed using heuristics or rubrics co-created by diverse teams, as demonstrated by Raven’s IBM colleagues Dre Barbara, Sherees Cooper, and Morgan Foreman.

  • Addiction to technology is an overused concept in consumer health, but distinguishing healthy from excessive use requires linking engagement data to well-being measures.

  • Distraction from core tasks is common in clinical environments when new tech disrupts workflows, evidenced by studies with ER staff and clinical trial recruitment tools.

  • Ethics frameworks like the Institute for the Future’s Ethical OS help anticipate risks like surveillance, bias, and data control, which inform Dark Metrics design principles.

  • Engaging diverse stakeholders and including co-creation early in research helps uncover biases and unintended consequences before launch.

  • Addressing negative impacts requires transparency with clients and a strong ethical posture, even when business priorities may conflict with user protection.

Notable Quotes

"The traditional product metrics focus narrowly on the product or near-term impact but fail to capture what success means for the whole person."

"Within IBM Watson, we prefer the term augmented intelligence rather than artificial intelligence to emphasize support, not takeover."

"An AI algorithm that didn’t explicitly consider race still produced racial disparities by using healthcare costs as a proxy."

"I am a Black person, but I do not have every Black experience. Not having experienced something is not proof that it doesn’t exist."

"The difference between technology and slavery is that slaves are fully aware they are not free."

"Doctors want to help people, not be on a machine all day; many health technologies are more distracting than helpful."

"We can assess distraction by observing time spent on screens versus with patients, and self-reported mental effort and stress."

"It’s important to ask, before any new release, what are all the things that could possibly go wrong?"

"Our jobs are to protect users from harm. If clients don’t care about side effects, it may be necessary to draw a line and walk away."

"Storytelling is highly effective in helping stakeholders understand the complete performance of products, including the darker sides."

Ask the Rosenbot
Daniel Gloyd
Transcendent Experience (ZX) Design: A Design Framework for Human Connection
2026 • Rosenfeld Community
Patrick Boehler
Fishing for Real Needs: Reimagining Journalism Needs with AI
2025 • Designing with AI 2025
Gold
Mark Templeton
Creating a Legacy: the ultimate experience
2017 • Enterprise Experience 2017
Gold
Séamus Byrne
Aligning Teams with Choreography
2024 • Enterprise Experience 2020
Gold
Sam Proulx
Understanding Screen Readers on Mobile: How And Why to Learn from Native Users
2023 • Advancing Research 2023
Gold
Dan Saffer
Why AI projects fail (and what we can do about it)
2025 • Rosenfeld Community
Uday Gajendar
Theme Four Intro
2023 • Enterprise UX 2023
Gold
Changying (Z) Zheng
Practical DesignOps: From Ideas to Tools That Teams Actually Use
2025 • Rosenfeld Community
Ariel Kennan
Building a Design Culture
2017 • Enterprise Experience 2017
Gold
Mark Boulton
Ops without Designers
2018 • DesignOps Summit 2018
Gold
Paul Ford
New work, new words: A glossary for AI
2026 • Designing with AI 2026
Conference
Isaac Heyveld
Expand DesignOps Leadership as a Chief of Staff
2022 • DesignOps Summit 2022
Gold
Louis Rosenfeld
GenAI for UXers: A Rosenbot Demo and Discussion
2025 • DesignOps Summit 2025
Gold
Laurent Christoph
Scale the impact of DesignOps in 3D: Diligence, Decision, Discipline
2025 • DesignOps Community
Indi Young
Thinking styles: Mend hidden cracks in your market
2025 • Rosenfeld Community
Robert Fabricant
A Genuine Conversation about the Future of UX Research
2024 • Advancing Research Community

More Videos

Natalia Radywyl

"Nearly a third of the 65 mental health commission recommendations call for co-design, which is pretty exciting."

Natalia Radywyl

Co-Designing New Power in Australia's Public Sector

November 16, 2022

Aditi Ruiz

"Synthesizing user feedback in a way that’s easy for product managers to consume is one of the hot topics right now."

Aditi Ruiz Christian Crumlish Farid Sabitov

Pulse Check: Empathy Mapping Your Product Manager, Pt. 2

December 6, 2022

Sheryl Cababa

"Optimizing something for ease of use does not mean best for the user or humanity."

Sheryl Cababa

Living in the Clouds: Adopting a Systems Thinking Mindset

June 6, 2023

Alba Villamil

"Attributing design differences to cultural values overlooks user workflows, constraints, and needs."

Alba Villamil

Stereotyped by Design: Pitfalls in Cross-Cultural User Research

March 30, 2020

Scott Jensen

"Both of our perspectives changed, and now we’re seeing that come to fruition through the product we’re building."

Scott Jensen Sarah Delaney Carmen Liu

Short Take #2: UX/Product Lessons from Your Industry Peers

December 6, 2022

Randolph Duke II

"I put down my question guide and just talked with Carol."

Randolph Duke II

War Stories LIVE! Randy Duke II

March 30, 2020

Briana Thomas

"Dogs and spam started filling the slides. That was a sign this meeting wasn’t working anymore."

Briana Thomas Christina Rodriguez

When Design Ops Comes in H.O.T. : A Tale of a Transformed Design Org

September 30, 2021

Saara Kamppari-Miller

"Look for the commonality in our definition of inclusion rather than the differences."

Saara Kamppari-Miller

Theme Three Intro

September 9, 2022

Sam Proulx

"Accessibility is a journey and a conversation—thanks for starting yours by being here today."

Sam Proulx

Online Shopping: Designing an Accessible Experience

October 3, 2023