Are My Research Findings Actually Meaningful?
Summary
Key Insights
-
•
Research must be tied to business imperatives and KPIs to deliver value, not done arbitrarily.
-
•
Qualitative research is crucial to interpret quantitative metrics like NPS meaningfully.
-
•
Statistical significance depends on sample size, variability, and magnitude of change.
-
•
A statistically significant change may not always be meaningful to the business context.
-
•
Meaningfulness is determined by alignment with organizational goals and market conditions.
-
•
Increasing sample size alone can make every change appear significant, which is misleading.
-
•
Driver modeling helps prioritize initiatives by linking key metrics to business outcomes.
-
•
Benchmarking against competitors and cross-industry experiences anchors understanding of CX scores.
-
•
Effective communication involves framing findings differently for executives, managers, and individual contributors.
-
•
Frequent measurement without changes made leads to wasted effort and misinterpretation.
Notable Quotes
"Research is not as easy as putting on a shoe and just doing it; it requires careful consideration and alignment with imperatives."
"Every board member can interpret an NPS score differently depending on their role and responsibilities."
"Statistical significance tells you if a change is likely not random, but it does not tell you if the change matters to your business."
"If you measure an infinite number of customers, every change would be statistically significant, but not every change would be meaningful."
"Meaningfulness is linked directly to what your organization cares about in terms of goals and priorities."
"Driver modeling tells you which research initiatives matter most based on their impact on key outcome metrics."
"Customers benchmark your company to their last best experience, often outside of your industry."
"When executives start questioning your research details, that’s a good sign they’re engaged and trusting the process."
"You shouldn’t send another survey unless you’ve implemented changes and allowed time for behavior to adapt."
"Mixed methods research, combining qualitative and quantitative, gives the fullest understanding of customer experience."
Or choose a question:
More Videos
"We don’t limit applicants by their major or background; it doesn’t matter if they studied physics or marketing."
Victor M. GonzalezPracticing Learners and Learning Practitioners
March 10, 2021
"Stories are great and build empathy, but how representative are those stories of the broader customer base?"
Ross SmithBreaking Barriers with Empathy
June 9, 2017
"We made it a priority from day one to support data exports so customers can leave without wasting weeks migrating data."
Sofia QuinteroThe Product Philosophy Behind EnjoyHQ
March 10, 2021
"We built a coalition of about 30 research advocates in a single year. That’s powerful."
Dagmara KukawkaTiny team, moonshot impact: Democratizing research across continents
March 10, 2026
"If you skip iterative testing, you risk building features nobody really wants."
Prayag Narula Hannah HudsonEmpowering Designers to do Good Research
March 11, 2022
"Sometimes democratized research happens in silence, with little coordination, causing duplicated efforts."
Kathleen AsjesResearch Democratization: the Good, the Bad and the Ugly
March 10, 2022
"Repair is any action that prevents negativity from spiraling out of control and helps bring people back to connection."
Alla WeinbergCross-Functional Relationship Design
December 6, 2022
"The evolution in AI UIs is ongoing and rapid, and we’re still figuring out what the final form of AI-native applications looks like."
Luke WroblewskiDesigning AI Applications
March 5, 2026
"The research librarian equals consistent taxonomy; without that, you can’t build accessible and searchable knowledge."
Sarah Sgarlato Pierini Kevin NewtonFrom Passion to Execution: A Story of Evolving Research Maturity at LinkedIn
September 9, 2022
