Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Integrating Qualitative and Quantitative Research from Discovery to Live

Friday, December 16, 2022 • QuantQual Interest Group
Share the love for this talk
Integrating Qualitative and Quantitative Research from Discovery to Live
Speakers: Mackenzie Cockram , Sara Branco Cunha and Ian Franklin
Link:

Summary

Over the last three years Southampton University in the UK has been doing a complete website redesign following an Agile process; user research and performance analytics have been an integral part of the Agile process from the very start. The website has a variety of products aimed at potential students and research collaborators. Performance analytics informed the business case and objectives and qualitative research in Discovery uncovered the user needs to improve the user experience. During alpha qualitative research informed the design of the early prototypes and in beta analytics and user research integrated qual and quant in a variety of metrics around performance and the user experience. The same metrics have been taken forward and enhanced in Live to ensure continuous improvement which sits alongside the new product roadmap. The presentation will outline the integration of qual and quant and give examples of what has been done, the metrics, and how they are informing the user experience and business objectives.

Key Insights

  • The University of Southampton faced over 4 million URLs due to decentralized content, causing poor user experience and high bounce rates.

  • Replacing 1000+ web content authors with a skilled specialist team improved content quality, consistency, and sustainability.

  • Integrating quantitative analytics with qualitative user research enabled a comprehensive understanding of user journeys and needs.

  • Measuring over 30 distinct user journeys allowed targeted benchmarking and direct evaluation of site performance.

  • Adopting user-centered design was a cultural shift from the university’s previous waterfall approach.

  • The beta phase attracted over 4,000 users, providing rich data to validate design choices before full launch.

  • Heat maps and recordings revealed user behaviors like 'click rage' where users clicked on non-clickable elements.

  • Microsoft reaction cards helped capture user sentiment efficiently, complementing free-text feedback.

  • Continuous improvement is enabled by automated data analysis using R scripts and a process to triage issues for design sprints or backlog.

  • Stakeholder involvement in observing real user sessions is crucial to overcoming resistance and building empathy for user-centered design.

Notable Quotes

"We are not collecting vanity analytics, the data is doing real work on informing design."

"85% of people were leaving after viewing only one page, showing severe bounce issues."

"The new site had a 19% dropout rate compared to 32% on the old site, showing clear improvement."

"User journeys are the bedrock for deciding which data to collect and measure."

"Some stakeholders still prefer their beliefs to data, even in academia."

"Heat maps showed users clicking on areas they thought were clickable but weren’t, resulting in click rage."

"We brought all the user journeys together with UX designers, content designers, product owners — it wasn’t just analytics and research in isolation."

"Having the qualitative and quantitative data together lets us tell a comprehensive story to stakeholders."

"We started analytics at the beginning so we could benchmark and compare old and new site performance."

"Agile can feel like a cult, but the data helps show it’s working and not just a belief system."

Ask the Rosenbot
Joerg Beringer
Scaling User Research with AI: Continuous Discovery of User Needs in Minutes
2025 • DesignOps Summit 2025
Gold
Laine Riley Prokay
How DesignOps can Drive Inclusive Career Ladders for All
2021 • DesignOps Summit 2021
Gold
Mary-Lynne Williams
Exit Interview #4: From Product Design Leadership to Sound Healing
2026 • Rosenfeld Community
Dominique Ward
The Most Exciting Time for DesignOps is Now
2022 • DesignOps Summit 2022
Gold
Lada Gorlenko
Theme 3: Introduction
2021 • Design at Scale 2021
Gold
Cara Maritz
The Art of Extrapolation
2023 • Advancing Research 2023
Gold
B. Pagels-Minor
Breaking the Tension: The Power of Enabling Your Employees to Show Up Authentically
2022 • Design at Scale 2022
Gold
John Maeda
Making Sense of Enterprise UX
2016 • Enterprise UX 2016
Gold
Laura Klein
Unique challenges of innovation in enterprises
2020 • Enterprise Community
Neil Barrie
Widening the Aperture: The Case for Taking a Broader Lens to the Dialogue between Products and Culture
2024 • Advancing Research 2024
Gold
Erin Weigel
Failure Friday #6: 90% of Everything I Do is Either Broken or Pointless
2025 • Rosenfeld Community
Sam Proulx
Understanding Screen Readers on Mobile: How And Why to Learn from Native Users
2023 • Enterprise UX 2023
Gold
Dalia El-Shimy
So You've Got a Seat at the Table. Now What?
2020 • Advancing Research 2020
Gold
Dave Malouf
The Past, Present, and Future of DesignOps: a 2-part DesignOps Community Call (Part 1)
2022 • DesignOps Community
Christopher Geison
Theme 1 Intro
2024 • Advancing Research 2024
Gold
Nicole Aleong
What UX research can learn from other research practices [Advancing Research Community Workshop Series]
2023 • Advancing Research Community

More Videos

Victor Udoewa

"Songs become methodologies—the way you identify who is at the door is through the song they sing."

Victor Udoewa

Beyond Methods and Diversity: The Roots of Inclusion

March 26, 2024

Peter Morville

"Changing culture is hard because we’re willing to change actions but resistant to changing beliefs."

Peter Morville

The Architecture of Understanding

May 13, 2015

Sam Ladner

"Some of the most mundane objects can signal change."

Sam Ladner

How Research Can Drive Strategic Foresight

March 9, 2022

Michelle Bejian Lotia

"We created research indexes with search blocks that update over time to help people focus on the most recent insights."

Michelle Bejian Lotia Anne-Marie Morell

Rolling Out a Repository: How Zapier Centralizes Insights from Across their Organization

March 28, 2023

Leisa Reichelt

"My kids know everything. If you listen to conversations with them, they say I know, Ma, I know."

Leisa Reichelt

Opening Keynote: Operating in Context

November 7, 2018

Sarah Gallimore

"Every parent we talked to had some variation of the question: this exists, can I use it right now? Where was this when I needed to apply?"

Sarah Gallimore

Inspire Progress with Artifacts from the Future

November 18, 2022

Joerg Beringer

"Product managers appreciate the jobs to be done desired outcome statements because they are not overly detailed but actionable."

Joerg Beringer Thomas Geis

Scaling User Research with AI: Continuous Discovery of User Needs in Minutes

September 10, 2025

Peter Merholz

"It’s not enough to have great people; you need to create the space that brings out the best in them."

Peter Merholz

Customer-Centered Design Organizations

June 8, 2017

Dalia El-Shimy

"VPs want to know what you really think. They want to see your insights distilled into a clear position of what needs to be done next."

Dalia El-Shimy

So You've Got a Seat at the Table. Now What?

March 31, 2020