This video is featured in the AI and UX playlist and 1 more.
Summary
As a data visualization designer and developer the challenge I often face is what to do with unstructured data. One case study I can show is exploring survey results where the multiple-choice questions are straightforward to analyze but interesting open-ended questions like “What do your colleagues not understand about data visualization?” are much harder to crack. Latent Scope is an open-source tool I built that streamlines a process of embedding text, mapping it to 2D, clustering the data points on the map and summarizing those clusters with an LLM. Once the process is done on a dataset structure emerges from the unstructured text, allowing us to get a sense of patterns in the survey answers. Themes like “the time it takes” to develop data visualization pop out, as do “the importance of good design.” While people don’t use the same language to describe these themes, they show up as clusters in the tool thanks to the power of embedding models. https://github.com/enjalot/latent-scope
Key Insights
-
•
AI embeddings transform unstructured data like text or sketches into high-dimensional vectors capturing hidden semantic patterns.
-
•
Dimensionality reduction algorithms map these high-dimensional embeddings into 2D clusters, making patterns visually accessible.
-
•
Free text survey responses, often too large to analyze manually, can be organized and explored effectively through embedding and clustering.
-
•
Google Cloud user journey data revealed surprising usage patterns, like enterprise and beginner tools being combined unexpectedly.
-
•
Simple rule-based filters for harmful prompts in generative AI image models can be bypassed by subtle prompt variations; embedding-based classifiers are more effective.
-
•
Latent Scope is an open source tool enabling non-technical users to embed, cluster, label, and visualize unstructured text data locally.
-
•
Local runs of embedding and clustering models on modest hardware are practical, preserving data privacy and sensitive user information.
-
•
Embedding models trained on multilingual data can cluster semantically similar texts across diverse languages in one shared space.
-
•
Breaking long text into smaller chunks can make embeddings more manageable and improve similarity comparisons.
-
•
Hands-on exploration of embedding models via platforms like Hugging Face helps users internalize concepts of semantic similarity and pattern discovery.
Notable Quotes
"People just don't get that the design and the process is fundamental."
"Similar inputs will produce similar high dimensional numbers."
"Dimensionality reduction algorithms take data points in high dimensional space and put them close together in 2D if they're similar."
"We found patterns that product teams didn’t expect or even want to look for."
"Simple word-list based filters can easily be tricked by misspellings or slight variations in prompts."
"What if you didn’t know there were important questions you should be asking in your data?"
"Latent Scope lets you quickly explore hundreds or thousands of free text responses to find clusters and patterns."
"You don’t need special hardware; these open source models can run locally on an M1 MacBook or a gaming machine."
"Multilingual embedding models can cluster similar meanings across languages in a shared latent space."
"Downloading and playing with local open source models gives a different experience than using faceless APIs mediated through interfaces."
Or choose a question:
More Videos
"This conference is more than a snapshot; it’s a part of a year-round conversation and community."
Louis RosenfeldWelcome / Housekeeping
June 6, 2023
"The cybernetic feedback loop is sensing, comparing to a goal, and acting to correct course."
Paul Pangaro, PhDSystems Disciplines: Table Stakes for 21st Century Organizations
June 6, 2023
"Service design has a critical role in mapping human and agent interactions in an AI and automation world."
Dharani PereraThe mandala of service design: unlocking alignment and action through service design
November 20, 2025
"Not every feature change needs a fully interactive prototype or pixel-perfect mock-up."
Tiffany ChengDesigning in a Pandemic: Integrating Speed and Rigor
June 9, 2022
"Perception maps visually represent how consumers perceive brands relative to competitors along key attributes."
Ricardo MartinsUnlocking the power of advanced quantitative methods
March 12, 2025
"We became addicted to our loop of over-optimization, driving traffic to certain features but frustrating users."
William Newton Jenny ChangHow to Lead With Data, and Without Data
June 7, 2023
"If you walk in someone else’s shoes, then you’ve taken their shoes."
Sahibzada MayedThe Politics of Radical Research: A Manifesto
March 27, 2023
"Design research—they need each other. They are much, much better together."
Prayag NarulaHow to Empower Your Designers to Do Good Research – And Why You Want To
June 10, 2022
"Component-driven prototyping means designing with coded UI components your development team already created."
Jack BeharHow to Build Prototypes that Behave like an End-Product
December 6, 2022
