Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Summary
Did you know that you're probably designing multimodal experiences? Most designers today are working in a multimodal environment, but haven't been trained to make the most of the many capabilities the latest generations of devices provide. Your customers have a small universe of devices, and they now come with the ability to handle far more than traditional haptic input like keyboards and mice. Gestural input, from swipes to hand gestures in video calls to stylus input, is becoming more common. Audio input and output are becoming more important in a world where digital assistants are poised to make a second surge powered by large language model AI. Visual output has become much more nuanced, and sometimes spans multiple devices. How do you wrangle all of this, optimize for great experiences, and still keep the human at the center? By becoming more consciously aware of the different inputs and outputs we're working with - and the many ways these inputs and outputs include and exclude our customers - we can build more resilient, more inclusive, and more powerful next-generation experiences.
Key Insights
-
•
Multimodal design is often an unrecognized practice in the design community, but most designers already incorporate aspects of it.
-
•
Human communication is inherently multimodal, and the future of design must mirror this complexity.
-
•
Understanding client context, including environmental factors and user relationships, is essential for creating effective designs.
-
•
Designing for multiple devices requires careful attention to transitions between modalities, which can significantly impact user experience.
-
•
Proactive communication during transitions enhances user satisfaction and prevents confusion during device handoffs.
-
•
Not all users interact with devices in the same way; tailoring experiences to different contexts is key to engagement.
-
•
Designs must accommodate both high and low information density, as well as proximity to users, to ensure seamless interaction.
-
•
Incorporating user research, including diary studies and guided tours, can reveal nuanced insights into multimodal usage.
-
•
Stakeholder buy-in for multimodal design should be linked to business goals like accessibility improvements and enhanced user engagement.
-
•
Emerging technologies will continue to shape the landscape of multimodal design, allowing for even richer interactions.
Notable Quotes
"The future is multimodal because humans are multimodal."
"Multimodal design is the design practice you probably don't realize you're doing."
"Understanding your customer's context is critical to your success."
"If a customer has come with a goal and you put something in their way, that's when you lose them."
"Proactive communication is critical to customer comfort and preventing tickets and freakouts."
"If we fully understand both design spaces, we truly empower ourselves to create resilient experiences."
"A mode in this context is a type of communication—primarily using senses."
"The practice adds an additional layer of design rigor on top of our existing modality-specific designs."
"Customers are swimming in devices now, and even websites are cross device."
"Design Beyond Devices gives you a complete toolkit for working at the cutting edge."
More Videos
"Executive sponsorship is essential for scaling design thinking across the organization."
Julie Gitlin Esther RaiceDesign as an Agent of Digital Transformation at JPMC
June 9, 2021
"The acknowledgment and validation of experiences can go a long way in cultivating trust and safety among teams."
Rachael Dietkus, LCSW Uday Gajendar Dr. Dawn Emerick Dawn E. Shedrick, LCSWLeading through the long tail of trauma
July 13, 2022
"Trust in the iterative process of writing; chapter one might get written last."
Louis RosenfeldCoffee with Lou: Should You Write a (UX) Book?
March 7, 2024
"What if I told you that Evan actually spent less than 15 minutes on the essay?"
Sarah GallimoreInspire Progress with Artifacts from the Future
November 18, 2022
"You don't have to get there all at once."
John DonmoyerShipping your code generation experiments to production
June 11, 2025
"Optimization through Slack can help us be gentler to each other."
Bria AlexanderTheme 1 Intro
September 23, 2024
"The art of flow diagrams and information architecture artifacts is vital."
JJ KercherA Roadmap for Maturing Design in the Enterprise
June 15, 2018
"We need each other and we need each other's support"
Bria AlexanderTheme Two Intro
September 8, 2022
"Using handwriting as an interface minimizes distractions from AI augmentations."
Lukas Moro“Feels Like Paper!”: Interfacing AI through Paper
June 11, 2025
