Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Summary
Quantitative instruments are frequently sought because 1) they can be quickly fielded to lots and lots of people, and 2) when carefully sampled, they can be generalizable to the population of users/customers. However, because many times the focus is on speed to launch because decision-makers need results quickly, there is not much depth given to their development, nor an investigation of the validity evidence. In the session, I will share a framework that centers validity and is necessarily a mixed methods approach to research. I will also share ideas on how to scale the research over time so that findings and insights are able to be iteratively delivered to stakeholders, while also iteratively informing one another in a qual-quant research dance that brings more trustworthy, user-centered evidence to decision-makers. Finally, I will share ideas for a course I am developing for supporting qualitative researchers to become more mixed in their approach.
Key Insights
-
•
A mixed-methods approach enhances validity and trust in survey research.
-
•
Validity is crucial in both qualitative and quantitative research to ensure accurate interpretations.
-
•
Engaging stakeholders early in the survey design process leads to better alignment with their needs.
-
•
Cognitive interviews can uncover participant understanding and response processes, refining survey items.
-
•
Iterative testing of surveys allows for continuous improvement and insight generation over time.
-
•
It's important to address both the ethical implications of survey questions and their potential impact on participants.
-
•
Researcher collaboration between quant and qual experts strengthens the validity argument.
-
•
Surveys should be treated as products requiring careful design and testing before deployment.
-
•
Identifying and understanding subpopulations can reveal biases and improve survey accuracy.
-
•
Validity assessments should be continuous, reflecting changes in user bases and research goals.
Notable Quotes
"I believe that when we have these multiple points of information, it does help build trust and improves our decision-making processes."
"Validity is the degree to which evidence and theory support the interpretations of test scores for proposed uses."
"When you have broader research goals that overlap, they can support you in answering important questions."
"These surveys are typically used to answer questions that support multiple stakeholder interests."
"If you're only doing qualitative research, you should still consider how you can contribute to the validity argument."
"Observing how people interpret questions can provide significant insights into survey design."
"Test content requires spending time talking with stakeholders about desired outcomes."
"Reliability and validity must work together to create meaningful surveys."
"Ethics in survey research should be top of mind throughout all stages of the process."
"We are the users of this product, and we need to iteratively test it."
More Videos
"Innovators can listen to what is unsaid."
John MaedaMaking Sense of Enterprise UX
June 9, 2016
"There are a lot of variability in how people approach their intent or purpose."
Indi YoungThinking styles: Mend hidden cracks in your market
January 8, 2025
"It doesn't matter what the insight is if no one does anything with it."
Chris ChapoData Science and Design: A Tale of Two Tribes
May 13, 2015
"The social dynamics of technology adoption play a critical role."
Elizabeth ChurchillExploring Cadence: You, Your Team, and Your Enterprise
June 8, 2017
"We had to put together very intentional and deliberate rituals."
Rusha SopariwalaRemote, Together: Craft and Collaboration Across Disciplines, Borders, Time Zones, and a Design Org of 170+
June 9, 2022
"The most common way people give up their power is by thinking they don't have any."
George AyeThat Quiet Little Voice: When Design and Ethics Collide
November 16, 2022
"Every once in a while, it's okay to turn your video off."
Ariba JahanTeam Resiliency Through a Pandemic
January 8, 2024
"We have more demand for design work than capacity right now, which is a good problem to have."
Julie Gitlin Esther RaiceDesign as an Agent of Digital Transformation at JPMC
June 9, 2021
"The more data we have, the more difficult it is to find what we actually need."
Matteo GrattonCan Data and Ethics Live Together?
October 1, 2021
