Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Dark Metrics: Illuminating the Negative Impact of Digital Health Design
Gold
Friday, March 12, 2021 • Advancing Research 2021
Share the love for this talk
Dark Metrics: Illuminating the Negative Impact of Digital Health Design
Speakers: Raven Veal
Link:

Summary

Traditional design metrics and KPIs are often geared towards measuring product success. Dark metrics challenge this paradigm by proactively measuring the unintended yet harmful psychological, social, and physical effects of our technologies. The examples within digital health are plentiful. From accelerating burnout among clinicians to widening racial disparities in quality of care, we can only reach the height of our most courageous solutions when we expose our deepest failures.

Key Insights

  • Traditional product metrics, like the Google HEART framework, often miss broader impacts on users, focusing narrowly on product success rather than holistic well-being.

  • Dark Metrics is a framework designed to measure negative unintended effects in digital health across four dimensions: disempowerment, exclusion, addiction, and distraction.

  • Disempowerment occurs when technology removes users’ autonomy, such as opaque black-box AI systems that undermine clinician or patient decision-making.

  • Exclusion can be subtle, as algorithms that proxy biased variables like healthcare cost can reproduce racial disparities without explicit intent.

  • Racial equity in design can be assessed using heuristics or rubrics co-created by diverse teams, as demonstrated by Raven’s IBM colleagues Dre Barbara, Sherees Cooper, and Morgan Foreman.

  • Addiction to technology is an overused concept in consumer health, but distinguishing healthy from excessive use requires linking engagement data to well-being measures.

  • Distraction from core tasks is common in clinical environments when new tech disrupts workflows, evidenced by studies with ER staff and clinical trial recruitment tools.

  • Ethics frameworks like the Institute for the Future’s Ethical OS help anticipate risks like surveillance, bias, and data control, which inform Dark Metrics design principles.

  • Engaging diverse stakeholders and including co-creation early in research helps uncover biases and unintended consequences before launch.

  • Addressing negative impacts requires transparency with clients and a strong ethical posture, even when business priorities may conflict with user protection.

Notable Quotes

"The traditional product metrics focus narrowly on the product or near-term impact but fail to capture what success means for the whole person."

"Within IBM Watson, we prefer the term augmented intelligence rather than artificial intelligence to emphasize support, not takeover."

"An AI algorithm that didn’t explicitly consider race still produced racial disparities by using healthcare costs as a proxy."

"I am a Black person, but I do not have every Black experience. Not having experienced something is not proof that it doesn’t exist."

"The difference between technology and slavery is that slaves are fully aware they are not free."

"Doctors want to help people, not be on a machine all day; many health technologies are more distracting than helpful."

"We can assess distraction by observing time spent on screens versus with patients, and self-reported mental effort and stress."

"It’s important to ask, before any new release, what are all the things that could possibly go wrong?"

"Our jobs are to protect users from harm. If clients don’t care about side effects, it may be necessary to draw a line and walk away."

"Storytelling is highly effective in helping stakeholders understand the complete performance of products, including the darker sides."

Ask the Rosenbot
Tricia Wang
Spatial Collapse: Designing for Emergent Culture
2024 • Enterprise Experience 2020
Gold
Abbey Smalley
Scaling UX Past the Size of Your Team
2024 • Enterprise Experience 2020
Gold
Louis Rosenfeld
Discussion: What Operations can teach DesignOps
2017 • DesignOps Summit 2017
Gold
Kristin Skinner
Theme 1 Intro
2021 • DesignOps Summit 2021
Gold
Savina Hawkins
Harnessing AI in UXR: Practical Strategies for Positive Impact
2024 • Advancing Research 2024
Gold
Maria Giudice
Remaking the Making Company: Moving from Product to Experience
2016 • Enterprise UX 2016
Gold
Marissa Cui
Climate Design Product Showcase
2024 • Climate UX Interest Group
Yunyan Li
UX Best Practices
2021 • Design at Scale 2021
Gold
Discussion
2017 • Enterprise Experience 2017
Gold
Daniel J. Rosenberg
Designing with and for Artificial Intelligence
2022 • Enterprise Community
Marjorie Stainback
Transforming Strategic Research Capacity through Democratization
2019 • DesignOps Summit 2019
Gold
Wendy Johansson
Be a Product Boss!
2022 • Design in Product 2022
Gold
Sean Baker
Weaving Knowledge Management into the Fabric of Our Design Practice
2025 • DesignOps Summit 2025
Gold
Theresa Slate
Why Changing Hearts & Minds Doesn’t Work When Promoting DE&I Efforts, but Checklists Do
2023 • DesignOps Summit 2023
Gold
Matt Duignan
Atomizing Research: Trend or Trap
2020 • Advancing Research 2020
Gold
Samuel Proulx
From Standards to Innovation: Why Inclusive Design Wins
2025 • Advancing Service Design 2025
Conference

More Videos

Doug Powell

"We have over 140,000 IBMers who’ve completed design thinking online training."

Doug Powell

Closing Keynote: Design at Scale

November 8, 2018

Mila Kuznetsova

"Make sure children understand they can stop the session at any time, and confirm they really get that."

Mila Kuznetsova Lucy Denton

How Lessons Learned from Our Youngest Users Can Help Us Evolve our Practices

March 9, 2022

Sarah Gallimore

"We created a Wizard of Oz smoke and mirrors prototype—a simple text message thread between a parent and this service—just to get enough buy-in."

Sarah Gallimore

Inspire Progress with Artifacts from the Future

November 18, 2022

Lada Gorlenko

"You can’t really advocate for UX or move an organization forward without interacting with your team and business colleagues."

Lada Gorlenko Sharbani Dhar Sébastien Malo Rob Mitzel Ivana Ng Michal Anne Rogondino

Theme 1: Discussion

January 8, 2024

Alnie Figueroa

"Our goal is to be AI fluent, not necessarily AI experts."

Alnie Figueroa

The Future of Design Operations: Transforming Our Craft

September 10, 2025

Landon Barnes

"Statistical significance tells you if a change is likely not random, but it does not tell you if the change matters to your business."

Landon Barnes

Are My Research Findings Actually Meaningful?

March 10, 2022

Emily Eagle

"When you’re talking, it’s easier to learn something new, but if you’re shy and don’t talk, you don’t learn anything."

Emily Eagle

Can't Rewind: Radio and Retail

June 3, 2019

Malini Rao

"The curse of hyper-focus means you lose peripheral vision and miss the larger context and customer needs."

Malini Rao

Lessons Learned from a 4-year Product Re-platforming Journey

June 9, 2021

Asia Hoe

"Collaborate and involve design early and often to better align on user goals and reduce friction."

Asia Hoe

Partnering with Product: A Journey from Junior to Senior Design

November 29, 2023