Tools are moments. Capabilities compound.
Summary
As design organizations race to adopt AI, many still measure readiness through tool fluency — who can use ChatGPT, Claude, or Figma Make the fastest. But tools, as we’ve learned, are ephemeral. The deeper challenge is defining the durable human capabilities that remain valuable as AI fundamentally reshapes design work. This talk explores a new approach to AI readiness for design organizations: moving beyond tool proficiency to identify the underlying skills, competencies, and capabilities that scale across changing technologies. We’ll examine the distinction between skills and capabilities, the process of building an AI capability framework for design, and how we’re mapping designers against those competencies to establish a baseline and path forward. This session offers a practical, human-centered model for building resilient, AI-augmented design organizations — grounded in durable capability, not temporary tooling trends.
Key Insights
-
•
Measuring AI fluency by tool usage is insufficient and often misleading; focusing on human judgment and observable behaviors offers deeper understanding.
-
•
Vanessa distinguishes between skills (tool-specific, ephemeral), competencies (applied judgment using multiple skills), and capabilities (umbrella outcomes across competencies).
-
•
A capability framework with six core areas and 21 competencies was built collaboratively with hundreds of Amazon design practitioners across roles.
-
•
Creative exploration, quality and responsibility, insight and synthesis, systems and architecture, learning and adaptability, and AI-human collaboration are the framework’s six key capabilities.
-
•
Gamified assessments with about 30 questions allow designers to baseline their fluency, receive diagnostics, and get an actionable growth map.
-
•
Frequency of AI tool use doesn’t equate to fluency; depth of skill integration and confidence in directing AI matter much more.
-
•
Patterns show common gaps in quality and responsibility and systems thinking, which are areas for targeted learning and development investments.
-
•
Role assumptions are challenged—design program managers aren’t always stronger in systems thinking, for example, revealing real-world variance and learning needs.
-
•
Providing a shared vocabulary around AI fluency helps practitioners articulate and connect their evolving AI skills beyond specific tools.
-
•
Baseline assessments need to be challenging enough to create a growth ceiling, balancing accuracy and aspirational goal setting without skewing results.
Notable Quotes
"I actually don’t need to feel ready to learn, and I can be uncomfortable or scared and still learn."
"It feels like we’ve forgotten to ask what the humans were actually building while chasing the tools."
"The difference is putting the human and their agency back into the mix with the question: What can you do with AI?"
"Fluency is how deeply you can work with AI—the judgment and direction you bring, not just how often you use it."
"The skills are tied to the tool and ephemeral; capabilities are what belong to us and travel with us as tools change."
"We wanted to create something that’s not just a survey but a robust, gamified experience to baseline current skills and provide a growth map."
"We’re focusing on observable behaviors and outcomes, not specific tools or time saved, to measure AI fluency."
"Everyone is learning and evolving right now; they just need language and frameworks to name what they’re growing."
"If the bar is too low, everyone looks very capable, and that doesn’t help growth or trust in the mapping."
"AI is the instrument. You all are the musicians. Build the capabilities that make quality music."
Or choose a question:
More Videos
"The climate crisis is a symptom of us being disconnected from so many things around us."
Aiyana Bodi James Christie Marc O'Brien Louis RosenfeldThree Key Climate Initiatives and How You Can Help
September 11, 2024
"Participatory sessions provide a unique chance to connect with users who might otherwise be hard to reach."
Nidhi Singh Rathore Amber DavisEmbracing participation to unlock deeper truths in commercial research
March 12, 2025
"Pilots don’t need to be perfect, they just need enough moving parts to gather useful feedback quickly."
Kim Fellman CohenMeasuring the Designer Experience
October 23, 2019
"Spotify introduced weekly mini subscriptions in the Philippines inspired by the local 'tenny' culture of small purchases."
Chui Chui TanGlobal insights: Embracing international and intercultural research for innovation
March 12, 2025
"Ethical judgment is the ability to maximize benefits while minimizing harm to the communities we work with."
Sarah Fathallah Alba VillamilBeyond insights: Rethinking the role of researchers as stewards of organizational wisdom
March 13, 2025
"Coming in too hot means running into a burning building like a firefighter, ignoring past work, and rushing without listening."
Maria GiudiceBecoming a Changemaker by Leading with Design
March 29, 2023
"Conversational analysis is not about replacing human understanding, it’s about amplifying it."
Deanna MitchellDesigning with culture: Unlocking impactful insights for Product and UX
March 12, 2025
"Insight summaries act like atomic units that connect research across studies and can be routed to owning teams."
Jake BurghardtFinding More Inroads into Research Impact
February 20, 2026
"You have to deliberately quiet that noise in your head and focus on humans in a world of humans."
Steve Portigal Susan Simon-Daniels Tamara Hale Randolph Duke IIWar Stories LIVE! Q&A-Discussion
March 30, 2020
