Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Summary
Let’s face it, many of us feel daunted by statistics. But we also know that colleagues and clients ask whether our research has “statistically significant” results. Erin’s book Design for Impact helps you to test your hypotheses about improving design, and she guides you through deciding on your effect sizes to help you get those statistically significant results. Caroline’s book Surveys That Work talks about “significance in practice” and she’s not all that convinced about whether it’s worth aiming for statistical significance. Watch this lively session where Erin and Caroline compared and contrasted their ideas and approaches - helped by your questions and contributions.
Key Insights
-
•
Statistics can be intimidating due to its mathematical language and cultural perceptions in the design community.
-
•
Understanding the difference between statistical significance and practical significance is crucial in making informed decisions.
-
•
Effect sizes help determine the meaningfulness of any changes in research outcomes, highlighting the importance of measuring bigger impacts rather than just statistical significance.
-
•
Moving fast is not always synonymous with progress in product design; thoughtful research leads to better decisions.
-
•
Developing an intuitive understanding of user behavior takes time and experimentation, which helps make better guesses about outcomes.
-
•
The use of statistics in research should not be seen as a barrier but rather as a vital tool to enhance decision-making processes.
-
•
Bad survey questions can yield misleading results, illustrating the need for precision in formulation.
-
•
Collaborating with engineers requires integrating their logic into your statistical methods to promote understanding and adoption of research practices.
-
•
Research should be holistic – integrating qualitative and quantitative data enhances overall insights.
-
•
Practicing basic statistical concepts, like averages, helps build comfort and familiarity with numbers.
Notable Quotes
"Statistics can feel hard because we're using math to understand the world around us."
"We want to make progress fast, but we must ensure we're heading in the right direction."
"Fast is not a virtue in and of itself; it's about acting with intention."
"You can have something that's significant in practice but not statistically significant – and vice versa."
"The real question is whether a change is meaningful and helpful, not just if it’s statistically significant."
"Statistical significance is a number that tells us whether the change happened by chance or due to our actions."
"Not all numbers need to be precise; they just need to provide a useful understanding of the situation."
"If you want to ship faster, know what you're aiming for and if that aim is worth pursuing."
"You get better results from asking one person a useful question than asking 10,000 people a silly one."
"The magic of statistics comes from how it can help predict what's true and what's real."
More Videos
"The power of followership is pretty amazing."
John MaedaMaking Sense of Enterprise UX
June 9, 2016
"Instead of doing it by feature, what if we did it by thinking style?"
Indi YoungThinking styles: Mend hidden cracks in your market
January 8, 2025
"It doesn't matter what the insight is if no one does anything with it."
Chris ChapoData Science and Design: A Tale of Two Tribes
May 13, 2015
"Public displays foster engagement and trust among teams."
Elizabeth ChurchillExploring Cadence: You, Your Team, and Your Enterprise
June 8, 2017
"Our work here is not done. This is a journey."
Rusha SopariwalaRemote, Together: Craft and Collaboration Across Disciplines, Borders, Time Zones, and a Design Org of 170+
June 9, 2022
"We mostly have something called BYOE, or bring your own ethics, in the design industry."
George AyeThat Quiet Little Voice: When Design and Ethics Collide
November 16, 2022
"Our mental, physical, and emotional bandwidth is like a battery that fluctuates between being fully charged and completely depleted."
Ariba JahanTeam Resiliency Through a Pandemic
January 8, 2024
"This isn’t just about design; this is about enabling clients to succeed."
Julie Gitlin Esther RaiceDesign as an Agent of Digital Transformation at JPMC
June 9, 2021
"We can decide how to improve the trust of our users based on the way we are transparent with them."
Matteo GrattonCan Data and Ethics Live Together?
October 1, 2021
