Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Dark Metrics: Illuminating the Negative Impact of Digital Health Design
Gold
Friday, March 12, 2021 • Advancing Research 2021
Share the love for this talk
Dark Metrics: Illuminating the Negative Impact of Digital Health Design
Speakers: Raven Veal
Link:

Summary

Traditional design metrics and KPIs are often geared towards measuring product success. Dark metrics challenge this paradigm by proactively measuring the unintended yet harmful psychological, social, and physical effects of our technologies. The examples within digital health are plentiful. From accelerating burnout among clinicians to widening racial disparities in quality of care, we can only reach the height of our most courageous solutions when we expose our deepest failures.

Key Insights

  • Traditional product metrics, like the Google HEART framework, often miss broader impacts on users, focusing narrowly on product success rather than holistic well-being.

  • Dark Metrics is a framework designed to measure negative unintended effects in digital health across four dimensions: disempowerment, exclusion, addiction, and distraction.

  • Disempowerment occurs when technology removes users’ autonomy, such as opaque black-box AI systems that undermine clinician or patient decision-making.

  • Exclusion can be subtle, as algorithms that proxy biased variables like healthcare cost can reproduce racial disparities without explicit intent.

  • Racial equity in design can be assessed using heuristics or rubrics co-created by diverse teams, as demonstrated by Raven’s IBM colleagues Dre Barbara, Sherees Cooper, and Morgan Foreman.

  • Addiction to technology is an overused concept in consumer health, but distinguishing healthy from excessive use requires linking engagement data to well-being measures.

  • Distraction from core tasks is common in clinical environments when new tech disrupts workflows, evidenced by studies with ER staff and clinical trial recruitment tools.

  • Ethics frameworks like the Institute for the Future’s Ethical OS help anticipate risks like surveillance, bias, and data control, which inform Dark Metrics design principles.

  • Engaging diverse stakeholders and including co-creation early in research helps uncover biases and unintended consequences before launch.

  • Addressing negative impacts requires transparency with clients and a strong ethical posture, even when business priorities may conflict with user protection.

Notable Quotes

"The traditional product metrics focus narrowly on the product or near-term impact but fail to capture what success means for the whole person."

"Within IBM Watson, we prefer the term augmented intelligence rather than artificial intelligence to emphasize support, not takeover."

"An AI algorithm that didn’t explicitly consider race still produced racial disparities by using healthcare costs as a proxy."

"I am a Black person, but I do not have every Black experience. Not having experienced something is not proof that it doesn’t exist."

"The difference between technology and slavery is that slaves are fully aware they are not free."

"Doctors want to help people, not be on a machine all day; many health technologies are more distracting than helpful."

"We can assess distraction by observing time spent on screens versus with patients, and self-reported mental effort and stress."

"It’s important to ask, before any new release, what are all the things that could possibly go wrong?"

"Our jobs are to protect users from harm. If clients don’t care about side effects, it may be necessary to draw a line and walk away."

"Storytelling is highly effective in helping stakeholders understand the complete performance of products, including the darker sides."

Ask the Rosenbot
Husani Oakley
Theme Three Intro
2023 • Enterprise UX 2023
Gold
John Calhoun
Have we Reached Our Peak? Spotting the Next Mountain For DesignOps to Climb
2021 • DesignOps Summit 2021
Gold
Aditi Ruiz
A PM State of Mind: Empathy Mapping Your Product Manager, Pt. 1
2022 • Design in Product 2022
Gold
Emilia Åström
Unlock Your Team’s Intelligence with Collaboration Design
2022 • Design at Scale 2022
Gold
Ana Maria Montero Barrantes
The Authentic UX Talent Show
2024 • Enterprise Experience 2020
Gold
Anupama Dhareshwar
From blueprint to bot: Designing resilient AI-powered services
2025 • Advancing Service Design 2025
Gold
Susan Simon-Daniels
War Stories LIVE! Susan Simon-Daniels
2020 • Advancing Research 2020
Gold
Robin Beers
How to create actionable insight in the face of politics and silos [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Victor Udoewa
Radical Participatory Research: Decolonizing Participatory Processes
2022 • Advancing Research 2022
Gold
Briana Thomas
The Quiet Force: Uncovering Hidden Leadership in High-Impact Design Teams
2024 • DesignOps Summit 2024
Gold
Greg Petroff
Everything is About to Change: Software as Material
2016 • Enterprise UX 2016
Gold
Kaaren Hanson
Stop Talking, Start Doing
2017 • Enterprise Experience 2017
Gold
Trisha Causley
[Demo] Complexity in disguise: Crafting experiences for generative AI features
2024 • Designing with AI 2024
Gold
Dave Hora
Advice for Establishing Research
2022 • Advancing Research Community
Carl Turner
You Can Do This: Understand and Solve Organizational Problems to Jumpstart a Dead Project
2023 • Advancing Research 2023
Gold
Dave Hora
Research in the Face of Complexity: New Sensibility for New Situations
2025 • Rosenfeld Community

More Videos

Carol Scott

"People who experience trauma often feel powerless; trauma-informed design should empower and give voice and choice instead."

Carol Scott Melissa Eggleston

Avoid Harming Your Team and Users: Promoting Care and Brand Reputation with Trauma-Informed UX Practices

February 5, 2025

Lija Hogan

"If you press the exact same button today and tomorrow you get two different results, that’s non-deterministic UX."

Lija Hogan Milan Mijatovic Sam Proulx Louis Rosenfeld

Three Years Out: Perspectives on the Near-Term Future of User Research

March 15, 2024

Sam Proulx

"Every app built into my phone is perfect when it comes to accessibility – Apple and Android set a strong example."

Sam Proulx

Mobile Accessibility and You

June 9, 2022

Ana Ferreira

"We document everything from product specs to implementation and design specs to prevent knowledge silos."

Ana Ferreira

Designing Distributed: Leading Doist’s Fully Remote Design Team in Six Countries

January 8, 2024

Jorge Arango

"Working with these tools requires new workflows and checkpoints to validate and tweak the AI's output."

Jorge Arango

Scale Smart: AI-Powered Content Organization Strategies

September 24, 2024

Chris Moses

"We built an app to let reviewers asynchronously capture design audit findings, significantly reducing scheduling friction."

Chris Moses

Stretching the Definition of DesignOps with Product Development

November 7, 2018

Trisha Causley

"People get stuck during the sessions when asked to describe their brand voice in just one or two words."

Trisha Causley

[Demo] Complexity in disguise: Crafting experiences for generative AI features

June 5, 2024

Clara Kliman-Silver

"Novice users can try things out thanks to automation, but experts can focus on the challenging and creative pieces."

Clara Kliman-Silver

UX Futures: The Role of Artificial Intelligence in Design

June 7, 2023

Michael Polivka

"Mutual benefit and agreement on the definition of success upfront are critical to long-term team effectiveness."

Michael Polivka

Scaling Design through Relationship Maps

November 7, 2017