Are My Research Findings Actually Meaningful?
Summary
You should not be doing research for the sake of doing research. Research takes time and needs to be well throughout. More importantly, you need to determine if your findings are actually meaningful to the organization. In this session we will look at the idea of statistical significance and meaningfulness when reporting research findings.
Key Insights
-
•
Research must be tied to business imperatives and KPIs to deliver value, not done arbitrarily.
-
•
Qualitative research is crucial to interpret quantitative metrics like NPS meaningfully.
-
•
Statistical significance depends on sample size, variability, and magnitude of change.
-
•
A statistically significant change may not always be meaningful to the business context.
-
•
Meaningfulness is determined by alignment with organizational goals and market conditions.
-
•
Increasing sample size alone can make every change appear significant, which is misleading.
-
•
Driver modeling helps prioritize initiatives by linking key metrics to business outcomes.
-
•
Benchmarking against competitors and cross-industry experiences anchors understanding of CX scores.
-
•
Effective communication involves framing findings differently for executives, managers, and individual contributors.
-
•
Frequent measurement without changes made leads to wasted effort and misinterpretation.
Notable Quotes
"Research is not as easy as putting on a shoe and just doing it; it requires careful consideration and alignment with imperatives."
"Every board member can interpret an NPS score differently depending on their role and responsibilities."
"Statistical significance tells you if a change is likely not random, but it does not tell you if the change matters to your business."
"If you measure an infinite number of customers, every change would be statistically significant, but not every change would be meaningful."
"Meaningfulness is linked directly to what your organization cares about in terms of goals and priorities."
"Driver modeling tells you which research initiatives matter most based on their impact on key outcome metrics."
"Customers benchmark your company to their last best experience, often outside of your industry."
"When executives start questioning your research details, that’s a good sign they’re engaged and trusting the process."
"You shouldn’t send another survey unless you’ve implemented changes and allowed time for behavior to adapt."
"Mixed methods research, combining qualitative and quantitative, gives the fullest understanding of customer experience."
Or choose a question:
More Videos
"Aim for a two for one: how can you make your goals and the goals of others align?"
Jess GrecoClaiming your power: Practical tools for amplifying your unique voice
March 13, 2025
"Large language models are far better at figuring out what you mean than simply giving an answer."
Josh Clark Veronika KindredSentient Scenes and Radically Adaptive Experiences
June 11, 2025
"There are over five hundred tools that call themselves survey software on G2, making it one of the largest software categories."
Aleksandra Korczynska Caroline Jarrett Justyna ParmeeSurvey Tools
March 12, 2026
"Moving partners from passive to resistant helped reveal their fears and motivations so we could tailor our conversations to them."
Anat Fintzi Rachel MinnicksDelivering at Scale: Making Traction with Resistant Partners
June 9, 2022
"Information architects use nodes and links to create environments for understanding."
Peter MorvilleThe Architecture of Understanding
May 13, 2015
"I didn’t want to be an AI guy. That was never my goal."
Paul FordNew work, new words: A glossary for AI
June 9, 2026
"When I started, there were a few practitioners in the US, a small community, and now I see this growing network and community of practice."
Ariel KennanTheme Two Intro
November 17, 2022
"Don’t forget cars are fun—get behind the wheel and enjoy a road trip once in a while; UX is meant to enhance that experience."
James RamptonThe Basics of Automotive UX & Why Phones Are a Part of That Future
July 25, 2024
"No research method is perfect; use a mix and triangulate results depending on whether you want to predict or explain."
Michaela MoraAdvanced Concept Testing Approaches To Guide Product Development and Business Decisions
March 11, 2022
