Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Summary
Let’s face it, many of us feel daunted by statistics. But we also know that colleagues and clients ask whether our research has “statistically significant” results. Erin’s book Design for Impact helps you to test your hypotheses about improving design, and she guides you through deciding on your effect sizes to help you get those statistically significant results. Caroline’s book Surveys That Work talks about “significance in practice” and she’s not all that convinced about whether it’s worth aiming for statistical significance. Watch this lively session where Erin and Caroline compared and contrasted their ideas and approaches - helped by your questions and contributions.
Key Insights
-
•
Statistics can be intimidating due to its mathematical language and cultural perceptions in the design community.
-
•
Understanding the difference between statistical significance and practical significance is crucial in making informed decisions.
-
•
Effect sizes help determine the meaningfulness of any changes in research outcomes, highlighting the importance of measuring bigger impacts rather than just statistical significance.
-
•
Moving fast is not always synonymous with progress in product design; thoughtful research leads to better decisions.
-
•
Developing an intuitive understanding of user behavior takes time and experimentation, which helps make better guesses about outcomes.
-
•
The use of statistics in research should not be seen as a barrier but rather as a vital tool to enhance decision-making processes.
-
•
Bad survey questions can yield misleading results, illustrating the need for precision in formulation.
-
•
Collaborating with engineers requires integrating their logic into your statistical methods to promote understanding and adoption of research practices.
-
•
Research should be holistic – integrating qualitative and quantitative data enhances overall insights.
-
•
Practicing basic statistical concepts, like averages, helps build comfort and familiarity with numbers.
Notable Quotes
"Statistics can feel hard because we're using math to understand the world around us."
"We want to make progress fast, but we must ensure we're heading in the right direction."
"Fast is not a virtue in and of itself; it's about acting with intention."
"You can have something that's significant in practice but not statistically significant – and vice versa."
"The real question is whether a change is meaningful and helpful, not just if it’s statistically significant."
"Statistical significance is a number that tells us whether the change happened by chance or due to our actions."
"Not all numbers need to be precise; they just need to provide a useful understanding of the situation."
"If you want to ship faster, know what you're aiming for and if that aim is worth pursuing."
"You get better results from asking one person a useful question than asking 10,000 people a silly one."
"The magic of statistics comes from how it can help predict what's true and what's real."
More Videos
"The very innovation you may need might already be written."
Jacqui FreyFlow and Superfluidity for Design Orgs
November 7, 2018
"The words we use should enable clear paths of understanding so recipients know their role."
Laura Gatewood Laine ProkayBeyond Buzzwords: Adding Heart to Effective Slack Communication
September 23, 2024
"Confirmation bias makes it harder to be open-minded when interpreting research data."
Johanna KollmannInsights-Driven Product Strategy: Get your Research to Count
December 6, 2022
"Take advantage of our sponsor sessions because they are really high quality and free."
Bria AlexanderDay 3 Welcome
September 25, 2024
"When you explore lots of different ideas, you're going to understand the context better."
Billy CarlsonIdeation tips for Product Managers
December 6, 2022
"The most underutilized asset in a retail environment is the store employees themselves."
Catherine DubutBridging Physical and Digital Spaces: Approaches to Retail Service Design
March 18, 2021
"The V2Mom framework helps us align our personal and company values effectively."
Kim Holt Emma Wylds Pearl Koppenhaver Maisee XiongA Salesforce Panel Discussion on Values-Driven DesignOps
September 8, 2022
"Those voices in a community of practice are not equally prominent."
Adrian HowardSturgeon’s Biases
September 25, 2024
"We learned to better work more synchronously as opposed to having a meeting for everything."
Asia HoePartnering with Product: A Journey from Junior to Senior Design
November 29, 2023
