Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Scale Smart: AI-Powered Content Organization Strategies
Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Scale Smart: AI-Powered Content Organization Strategies
Speakers: Jorge Arango
Link:

Summary

Keeping large content repositories organized is an ongoing challenge. There's always new stuff coming in, and taxonomies evolve over time. Resource-strapped teams seldom have opportunities to re-categorize older content. It's a task well-suited for generative AI. Large language models have powerful capabilities that can help teams keep content organized at scale. Using LLMs in this capacity can lead to better user experiences and free team members to focus on more valuable efforts. This presentation explores two approaches for using LLMs to organize content at scale: 1) re-categorizing content using existing categories and 2) developing new categories from existing content. Both will be shown as proofs of concept alongside feasible next steps.

Key Insights

  • Maintaining organized large content repositories is often deprioritized despite evolving product portfolios, leading to reduced findability.

  • Search alone is insufficient for large repositories because users often lack context about what content exists.

  • Evolving taxonomies require periodic retagging of older content to keep it relevant and findable.

  • AI, specifically large language models like GPT-4, can automate taxonomy maintenance tasks significantly faster than humans.

  • Human oversight is critical to review AI-suggested tags to prevent hallucinations and ensure quality.

  • Developing new taxonomies requires analyzing content holistically, which exceeds typical LLM context windows.

  • Embedding databases help chunk and relate content statistically for clustering similar topics at scale.

  • Graph Retrieval Augmented Generation (Graph RAG) integrates knowledge graphs with LLMs to increase precision by leveraging semantic relationships.

  • RAG enables querying domain-specific content unseen by generic LLM training data, supporting bespoke AI assistants.

  • AI tools enable higher speed and new methods for content work but require new workflows and ongoing experimentation.

Notable Quotes

"Keeping large content repositories organized is an ongoing challenge that often gets deprioritized."

"Search alone doesn’t cut it, particularly when people don’t know what they’re looking for or lack context."

"If organizations don’t keep content organized, it becomes less usable and less useful over time."

"Large language models can help organize content faster and at much larger scale than people can."

"I used GPT-4 to retag 1,200 blog posts, reducing an estimated 10–12 hours of tedious manual work to about a third of that time."

"I made sure the LLM would only use terms from my predefined taxonomy, but it still introduced some of its own tags."

"Working with these tools requires new workflows and checkpoints to validate and tweak the AI's output."

"Graph RAG uses knowledge graphs instead of just plain text snippets, which really improves precision."

"I see these AI tools as augmenting human work, letting us focus on what matters most, not replacing humans."

"The best way to understand these technologies is to learn them hands-on, especially if your work involves language or taxonomies."

Ask the Rosenbot
Dan Willis
Enterprise Storytelling Sessions
2016 • Enterprise UX 2016
Gold
Heidi Trost
To Protect People, You Have to Protect Information: A Human-Centered Design Approach to Cybersecurity
2025 • Rosenfeld Community
Tutti Taygerly
Make Space to Lead
2021 • Design at Scale 2021
Gold
Megan Kierstead
You Are a Badass at UX: Overcoming Imposter Syndrome
2021 • Advancing Research 2021
Gold
Erika Kincaid
Connecting the Dots: How to Foster Collaboration and Build a Strong Design Review Culture
2022 • Design at Scale 2022
Gold
Bob Baxley
Theme 4: Discussion
2024 • Enterprise Experience 2020
Gold
Jorge Arango
[Demo] How to re-categorize content at scale using LLMs
2024 • Designing with AI 2024
Gold
Greg Petroff
Design is the Differentiator: Bringing New Design Innovations to a Very Antiquated and Very Large Industry
2021 • Design at Scale 2021
Gold
Dave Malouf
Panel: Design Systems and Documentation
2017 • DesignOps Summit 2017
Gold
Jemma Ahmed
Democratization: Working with it, not against it [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Greg Petroff
The Compass Mission
2021 • Advancing Research 2021
Gold
Johanna Kollmann
Insights-Driven Product Strategy: Get your Research to Count
2022 • Design in Product 2022
Gold
Sha Hwang
The Lost Year
2021 • Design at Scale 2021
Gold
Christopher Geison
Theme 1 Intro
2024 • Advancing Research 2024
Gold
Ned Dwyer
The Future of DesignOps is Tool Consolidation
2024 • DesignOps Summit 2024
Gold

More Videos

Doug Powell

"Good design is good business."

Doug Powell

Closing Keynote: Design at Scale

November 8, 2018

Mila Kuznetsova

"Middle schoolers might ask you tough questions about release forms and how their data will be used."

Mila Kuznetsova Lucy Denton

How Lessons Learned from Our Youngest Users Can Help Us Evolve our Practices

March 9, 2022

Sarah Gallimore

"We created a Wizard of Oz smoke and mirrors prototype—a simple text message thread between a parent and this service—just to get enough buy-in."

Sarah Gallimore

Inspire Progress with Artifacts from the Future

November 18, 2022

Lada Gorlenko

"Field usability testing showed stakeholders immediately how much value UX can add by catching issues before deployment."

Lada Gorlenko Sharbani Dhar Sébastien Malo Rob Mitzel Ivana Ng Michal Anne Rogondino

Theme 1: Discussion

January 8, 2024

Alnie Figueroa

"Who deals with chaos, ambiguity, and disruption best? It’s us, design operations humans."

Alnie Figueroa

The Future of Design Operations: Transforming Our Craft

September 10, 2025

Landon Barnes

"Driver modeling tells you which research initiatives matter most based on their impact on key outcome metrics."

Landon Barnes

Are My Research Findings Actually Meaningful?

March 10, 2022

Emily Eagle

"When you’re talking, it’s easier to learn something new, but if you’re shy and don’t talk, you don’t learn anything."

Emily Eagle

Can't Rewind: Radio and Retail

June 3, 2019

Malini Rao

"Re-platforming is transformative not just for the product but also for the people, teams, and organization."

Malini Rao

Lessons Learned from a 4-year Product Re-platforming Journey

June 9, 2021

Asia Hoe

"Accessibility isn’t just a checklist; it changes moment to moment for any individual."

Asia Hoe

Partnering with Product: A Journey from Junior to Senior Design

November 29, 2023