Summary
You should not be doing research for the sake of doing research. Research takes time and needs to be well throughout. More importantly, you need to determine if your findings are actually meaningful to the organization. In this session we will look at the idea of statistical significance and meaningfulness when reporting research findings.
Key Insights
-
•
Effective CX research requires a clear alignment with business imperatives and KPIs.
-
•
Statistical significance alone does not guarantee that findings are meaningful to the business.
-
•
Qualitative research provides context and meaning to quantitative data, enriching understanding.
-
•
Competitive benchmarking is essential for assessing relative performance against industry standards.
-
•
Increased sample size doesn’t automatically ensure significance; the context and distribution of data matter.
-
•
Understanding the relationship between statistical significance and meaningfulness is crucial in research interpretation.
-
•
Not all metrics have the same impact on business objectives; prioritizing findings is key.
-
•
Driver modeling is a valuable technique to identify effective perception metrics that influence outcomes.
-
•
CX research should not only measure but also interpret changes in customer sentiment meaningfully.
-
•
Engaging stakeholders by relating findings to their specific interests enhances research impact.
Notable Quotes
"Research is not easy to do; it requires thoughtful planning and execution."
"You don't just want to be doing research for the sake of doing research; it has to have an outcome."
"Qualitative research is extremely important; it gives a clear picture of what the numbers mean."
"Statistical significance does not always correlate with meaningfulness; you need to dive deeper into the data."
"You should not measure more often unless you've made changes that have had time to be adopted."
"Seeing data without context can lead to misinterpretation; it's about understanding what the scores tell you."
"The findings must be relatable to what executives care about, like increasing revenue or reducing costs."
"You should focus on results that are meaningful to your business, not just statistically significant findings."
"Benchmarking your results across different industries is crucial because customers compare experiences beyond your sector."
"Utilizing driver modeling can help determine which metrics are most impactful on your business results."
More Videos
"Designing with AI is about speeding up processes, not replacing creativity."
Noz UrbinaRapid AI-powered UX (RAUX): A framework for empowering human designers
May 1, 2025
"There are guidelines that we rely on to help us stay together."
Jim KalbachJazz Improvisation as a Model for Team Collaboration
June 4, 2019
"What would you do if info sets said you can't use this tool?"
Holly ColeUnderstanding Experiences: When you have to do more than work
November 8, 2018
"We can achieve pixel parity by working with the same source of truth."
Jack BeharHow to Build Prototypes that Behave like an End-Product
December 6, 2022
"It is very important for us to keep the vibes positive and make sure you feel good about being here."
Bria AlexanderOpening Remarks
October 3, 2023
"Organizations must build a culture of safe experimentation and reward curiosity."
Erika FlowersAI-Readiness: Preparing NASA for a Data-Driven, Agile Future
June 10, 2025
"Equipped with the right tools, we can truly meet students where they are."
Kristin SkinnerFive Years of DesignOps
September 29, 2021
"Big ears mean listening to other people more than you're listening to yourself."
Jim KalbachJazz Improvisation as a Model for Team Collaboration
November 6, 2017
"Think in systems to scale creative initiatives effectively."
Surya VankaUnleashing Swarm Creativity to Solve Enterprise Challenges
June 10, 2021
