Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Integrating Qualitative and Quantitative Research from Discovery to Live
Friday, December 16, 2022 • QuantQual Interest Group
Share the love for this talk
Integrating Qualitative and Quantitative Research from Discovery to Live
Speakers: Mackenzie Cockram , Sara Branco Cunha and Ian Franklin
Link:

Summary

Over the last three years Southampton University in the UK has been doing a complete website redesign following an Agile process; user research and performance analytics have been an integral part of the Agile process from the very start. The website has a variety of products aimed at potential students and research collaborators. Performance analytics informed the business case and objectives and qualitative research in Discovery uncovered the user needs to improve the user experience. During alpha qualitative research informed the design of the early prototypes and in beta analytics and user research integrated qual and quant in a variety of metrics around performance and the user experience. The same metrics have been taken forward and enhanced in Live to ensure continuous improvement which sits alongside the new product roadmap. The presentation will outline the integration of qual and quant and give examples of what has been done, the metrics, and how they are informing the user experience and business objectives.

Key Insights

  • The University of Southampton faced over 4 million URLs due to decentralized content, causing poor user experience and high bounce rates.

  • Replacing 1000+ web content authors with a skilled specialist team improved content quality, consistency, and sustainability.

  • Integrating quantitative analytics with qualitative user research enabled a comprehensive understanding of user journeys and needs.

  • Measuring over 30 distinct user journeys allowed targeted benchmarking and direct evaluation of site performance.

  • Adopting user-centered design was a cultural shift from the university’s previous waterfall approach.

  • The beta phase attracted over 4,000 users, providing rich data to validate design choices before full launch.

  • Heat maps and recordings revealed user behaviors like 'click rage' where users clicked on non-clickable elements.

  • Microsoft reaction cards helped capture user sentiment efficiently, complementing free-text feedback.

  • Continuous improvement is enabled by automated data analysis using R scripts and a process to triage issues for design sprints or backlog.

  • Stakeholder involvement in observing real user sessions is crucial to overcoming resistance and building empathy for user-centered design.

Notable Quotes

"We are not collecting vanity analytics, the data is doing real work on informing design."

"85% of people were leaving after viewing only one page, showing severe bounce issues."

"The new site had a 19% dropout rate compared to 32% on the old site, showing clear improvement."

"User journeys are the bedrock for deciding which data to collect and measure."

"Some stakeholders still prefer their beliefs to data, even in academia."

"Heat maps showed users clicking on areas they thought were clickable but weren’t, resulting in click rage."

"We brought all the user journeys together with UX designers, content designers, product owners — it wasn’t just analytics and research in isolation."

"Having the qualitative and quantitative data together lets us tell a comprehensive story to stakeholders."

"We started analytics at the beginning so we could benchmark and compare old and new site performance."

"Agile can feel like a cult, but the data helps show it’s working and not just a belief system."

Ask the Rosenbot
Dan Hill
Strategic design, slowdown, and the infrastructures of everyday life
2022 • Enterprise Community
JD Buckley
Communicating the ROI of UX within a large enterprise and out on the streets
2018 • Enterprise Experience 2018
Gold
Kayla Farrell
What It's Like To Be a User Researcher at Compass
2021 • Advancing Research 2021
Gold
Zariah Cameron
ReDesigning Wellbeing for Equitable Care in the Workplace
2024 • DesignOps Summit 2024
Gold
Llewyn Paine
Day 1 Using AI in UX with Impact
2025 • Designing with AI 2025
Gold
Dan Hill
Designing for the infrastructures of everyday life
2024 • Designing with AI 2024
Gold
Taiye Akin-Akinyosoye
Amplifying voices and enhancing user research through group interviews
2025 • Advancing Research 2025
Gold
Sarah Fathallah
A Typology of Participation in Participatory Research
2023 • Advancing Research 2023
Gold
John Devanney
The Design Management Office
2017 • DesignOps Summit 2017
Gold
Mark Interrante
Collaboration Flows in Product Development
2017 • Enterprise Experience 2017
Gold
Marc Fonteijn
First Insights from the 2025 Service Design Salary(+) Report
2024 • Advancing Service Design 2024
Gold
Ryan Matthew
Bridging Design and Code: AI-Powered Design System Integration
2025 • Rosenfeld Community
Rachel Posman
A Closer Look at Team Ops and Product Ops (Two Sides of the DesignOps Coin)
2020 • DesignOps Community
Sydney Lawson
Anatomy of a Strong User Panel
2026 • Advancing Research 2026
Conference
Bria Alexander
Opening Remarks
2023 • DesignOps Summit 2023
Gold
Johanna Kollmann
Insights-Driven Product Strategy: Get your Research to Count
2022 • Design in Product 2022
Gold

More Videos

Chelsey Glasson

"Working in tech was a scenic detour on a long-term career marathon, with more detours ahead but now I have the tools to navigate them."

Chelsey Glasson

Exit Interview #3: Same as It Ever Was: What Leaving Tech Taught Me About Change

December 17, 2025

Laura Klein

"AI cannot notice body language, tone, or environmental context during a research session."

Laura Klein

Human vs. machine: Testing AI’s ability to synthesize and analyze research

March 11, 2026

Dr. Jamika D. Burge

"Biases out of sight are biases out of mind — facial recognition misidentifies Black faces leading to false arrests."

Dr. Jamika D. Burge

Broad Strokes: Connecting Design, Research, and AI to the World Around Us

June 7, 2023

Iram Shah

"Intelligence is the pursuit of knowledge; wisdom is the pursuit of truth."

Iram Shah

Closing Keynote: The View from the Top

June 4, 2019

Alicia D. Johnson

"Every emergency manager has an origin story, mine was the Yellowstone Fire in 1988 when I was seven."

Alicia D. Johnson

Disasters and the 21st Century

December 10, 2021

Lija Hogan

"If you press the exact same button today and tomorrow you get two different results, that’s non-deterministic UX."

Lija Hogan Milan Mijatovic Sam Proulx Louis Rosenfeld

Three Years Out: Perspectives on the Near-Term Future of User Research

March 15, 2024

Sofía Delsordo

"Good design needs better PR and the government has to be actively involved in the process."

Sofía Delsordo Kassim Vera

Public Policy for Jalisco's Designers to Make Design Matter

December 8, 2021

Ron Bronson

"It’s less about interaction and more about integrity: How can we build trust in experiences?"

Ron Bronson

Design, Consequences & Everyday Life

November 18, 2022

Richard Buchanan

"Are you thinking about what comes after the smartphone? Yes, and there are hundreds of people working on that."

Richard Buchanan

Creativity and Principles in the Flourishing Enterprise

June 15, 2018