Measuring What Matters
Summary
Key Insights
-
•
IBM’s CIO design team represents just 1% of the IT org yet supports UX for 385,000 employees across the whole company.
-
•
Over half of IBMers have been with the company less than five years, adding urgency to delivering great onboarding and daily experiences.
-
•
About 30% of IBM employees work remotely or offsite, increasing reliance on intuitive, well-integrated digital tools.
-
•
The team follows a mission to lead with design, engineer with excellence, deepen agile practices, and secure the enterprise.
-
•
Kristen’s team moved from reactive user research to building proactive, predictive capabilities by aggregating decades of experience data.
-
•
Measurement shifted from isolated metrics like NPS to a holistic scorecard including ease of use, capability, and goal completion.
-
•
A voice of the employee continuous feedback tool replaced large, infrequent surveys to capture real-time user sentiment.
-
•
Data-driven decisions led to major redesigns of IBM’s employee directory, help system, and corporate intranet, each achieving significant user appreciation.
-
•
Measurement can be challenging due to resistance and attempts to game survey responses, underscoring the need for data integrity.
-
•
Embracing design ops chaos and continuous iteration is key to maturing UX capability in a large enterprise organization like IBM.
Notable Quotes
"This circle represents the size of IBM, 385,000 IBMers. The CIO team is only 3% of that, and our design team is just 1% of the CIO."
"We have about 5,000 new IBMers arriving every month, with high expectations and low tolerance for poor experiences."
"The state of IT is a daily reflection of what the company thinks and feels about its employees."
"You can't just jump to the top of the pyramid like Maslow’s hierarchy—you have to consider all the underlying layers supporting the experience."
"Fletcher Prevend took us to the next level by reporting design and user research directly into the CIO office—a huge differentiator."
"Every minute spent struggling with an IT system is a minute not spent delivering value for IBM and our clients."
"Before, we never really looked holistically at decades of UX data—now we use it to be proactive and avoid duplicative studies."
"We want to send fewer mailers but make them more impactful, using A/B testing to optimize open and click rates."
"Don’t try to control the data to fit your narrative—the goal is to let insights guide your decisions, not justify them."
"We’re the voice of the employee—we’re arbiters of truth, defenders of experience, and sometimes validators against manipulation."
Or choose a question:
More Videos
"Ownership of VOC is a hot potato, so we built a coalition with a facilitator to keep it collaborative and effective."
Shipra KayanHow we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up
September 30, 2021
"Language gives us the words to talk about warmth, and that’s the birth of these ideas spreading within organizations."
Daniel GloydTranscendent Experience (ZX) Design: A Design Framework for Human Connection
August 6, 2026
"System fonts load faster, improve legibility, and outperform fancy brand fonts in conversion."
Erin WeigelUX Lessons from running more than 1,200 A/B Tests
July 10, 2024
"Some old fashioned solutions bring new problems for Istanbul."
Onur Kocan Ayhan EnsiciUnderstanding the Strategy for Civic Design in a Complex City: Istanbul
November 16, 2022
"The best we do should be open to improvement and evolution to better serve clients' target audiences."
Megan CamposWhat Did I Miss? The Hidden Costs of Deprioritizing Diversity in User Research
March 12, 2021
"Everything everyone knows is only a model. It’s important to get our models out there where they can be challenged."
Jennifer FraserWhat would Emmy Noether Do? Math, Models and Mulling in UX Research
March 29, 2023
"It’s about helping your colleagues be more productive and make better decisions to help customers."
Sofia QuinteroBeyond Tools: The Messy Business of Implementing Research Repositories
March 10, 2022
"No one knows yet exactly what bringing truth to power means, and that’s the beauty of it."
Jemma AhmedTheme 2 Intro
March 26, 2024
"If you really want to build culture and see how people cross collaborate, try to make lunch with your entire team every single week."
Jennifer KanyamibwaCreating the Blueprint: Growing and Building Design Teams
November 8, 2018
