This video is featured in the Designing with AI 2025 playlist.
Summary
We built a novel math tutoring app for 11-year-olds in the UK. Since this was our first AI project, we expected lots of technical issues. Those happened and on top of it, we ended up questioning the value of designers in AI-driven products.
Key Insights
-
•
Designing AI products demands a strong understanding of AI’s inner workings to anticipate challenges and create meaningful alternatives.
-
•
AI model updates can unpredictably impact performance, requiring flexible design solutions and rapid iteration.
-
•
The Boost 11 Plus app used photography and step-by-step AI tutoring to assist stressed students and their families effectively.
-
•
Strict SLAs, like 24-hour design comment turnarounds, help teams manage rapid changes and keep alignment in AI projects.
-
•
Design principles such as avoiding gamification and clarifying the app is a calculator, not a person, were crucial to maintaining ethical clarity.
-
•
Designers face loss of traditional precision and predictability when designing with large language models due to their inherent opacity.
-
•
Rapid AI advances demand constant adaptation; designers must create multiple design alternatives to navigate ambiguity.
-
•
The plastics analogy illustrates how shiny new technologies can deliver powerful capabilities but carry unforeseen long-term harms and ethical risks.
-
•
AI’s environmental cost, like energy use for generating content, is significant and often overlooked in design and deployment choices.
-
•
AI systems can diminish critical cognitive skills like memory and critical thinking, highlighting the need for mindful usage and design.
Notable Quotes
"The app reads the question and prepares tutoring material, then helps solve it step by step or through chat if further questions remain."
"We realized designing for AI is very different from other computational mediums we used before."
"Understanding the lifecycle of AI code is essential because AI is essentially code running in constant evolution."
"With LLMs, precision is not possible; no one really knows why it decided what to say."
"AI is like plastics on the surface — it gives amazing powers but has hidden damages we must not repeat."
"Design is an excellent way to navigate chaos, turning ambiguity into direction through alternatives."
"We used design principles like look but don't copy, this is a calculator not a person, and no gamification to guide tough decisions."
"AI companies focus on the shiniest thing that comes without caring about stability or reliability."
"It costs about as much as a bottle of water in energy to generate a single email from an LLM system."
"AI users have reduced critical thinking skills within just two years of using AI-assisted tools."
Or choose a question:
More Videos
"By limiting our classes to six or seven students, we could give better quality feedback."
Marjorie Stainback Kelsey KingmanTransforming Strategic Research Capacity through Democratization
October 24, 2019
"User-centered design methods can be used at every stage of policy making, from intention setting to evaluation."
Alexandra SchmidtWhy Ethics Can't Save Tech
November 18, 2022
"These systems aren’t stable like old software; they change as data and models evolve."
Carol SmithOperationalizing Responsible, Human-Centered AI
June 7, 2023
"Stakeholder communication is crucial because without it, we risk activity without impact."
Fisayo Osilaja[Demo] The AI edge: From researcher to strategist
June 4, 2024
"People coming from agencies sometimes need help dropping bad habits like working at 2am or stressful environments."
Allison SandersOperating with Purpose
January 8, 2024
"The curse of hyper-focus means you lose peripheral vision and miss the larger context and customer needs."
Malini RaoLessons Learned from a 4-year Product Re-platforming Journey
June 9, 2021
"Your short term memory is the context window; too big, and the model has trouble accessing everything."
Peter Van DijckHands on AI #3: Claude Code for UX people
October 22, 2025
"Talking about thinking styles is just as easy and fast a shortcut as talking about demographics, but more useful."
Indi YoungThinking styles: Mend hidden cracks in your market
January 8, 2025
"Having a repo isn’t the same as having a strategy for socializing and evangelizing the research."
Brigette Metzler Dana ChrisfieldResearch Repositories: A global project by the ResearchOps Community
August 27, 2020
Latest Books All books
Dig deeper with the Rosenbot
How can program or project managers support design ops activities when no dedicated ops role exists?
In what ways does Kenneth see the technology industry retreating from ethical responsibility recently?
How does Kenneth justify the impact of his academic path on the UX community despite leaving direct practice?