Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
Gold
Thursday, May 14, 2015 • Enterprise UX 2015
Share the love for this talk
Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
Speakers: Bill Scott
Link:

Summary

Bill recounts his journey from Netflix, where he witnessed the shift from DVDs to streaming, to PayPal, where he led transformative engineering reforms. At Netflix, he realized that designing software as a throwaway experimentation layer rather than permanent code was key, with multiple concurrent experiments driving user-focused learning. He stresses that engineering should enable learning rather than just code stability. At PayPal, Bill faced legacy technical debt and cultural inertia (organizational antibodies) but pushed for a culture of rapid iteration, collaboration, and customer immersion. He implemented a new technology stack based on Node.js and GitHub, democratized innovation through an internal open source model, and emphasized the need to give Agile a 'brain' by embedding continuous user feedback deeply into the backlog and process. Bill highlights the importance of shared vocabulary between disciplines, collaboration, and continuous customer feedback to keep teams aligned and focused on solving real user problems rather than defending solutions. Drawing on examples from Netflix, PayPal, and Meetup, he underscores that successful teams embrace failure in small increments, enable rapid prototyping, and design for volatility.

Key Insights

  • At Netflix, 95% of the UI layer was thrown away within a year, reframing UI as an experimentation layer rather than durable software.

  • Engineering's primary goal should be enabling learning, partnering closely with design and product teams.

  • Different parts of a software stack have different risk profiles; applying the concept of shearing layers helps accept more risk on the user interface layer to enable faster learning.

  • Organizations contain 'antibodies'—cultural and organizational forces resistant to change—that must be understood and navigated to drive transformation.

  • PayPal's transformation was accelerated by a top-down mandate combined with intense cross-functional collaboration and frequent user testing.

  • Using internal GitHub and open source paradigms democratizes code access and innovation, allowing anyone to contribute and experiment.

  • Prototyping should be considered a first-class engineering activity, not a separate or lesser process.

  • Agile methods lack an inherent ‘brain’; embedding continuous customer feedback and real user context into the backlog gives Agile teams direction and purpose.

  • Engineering teams that share vocabulary and deeply collaborate with designers and product managers produce better outcomes.

  • Embracing small incremental failures enables faster learning and avoids the risk of large-scale failures that can paralyze organizations.

Notable Quotes

"I started thinking of the UI layer as the experimentation layer."

"Engineering’s number one goal should be to enable learning."

"You have to design for throw away ability because the majority of experience gets thrown away."

"Organizations contain antibodies—cultural forces that resist change."

"If you don’t soak teams in real customer context, they do dumb things not because they’re dumb, but because they lack context."

"Features eventually become barnacles that are impossible to scrape off the boat."

"Agile needs a brain, and that brain is the continuous customer feedback loop."

"Prototyping isn’t a second-class citizen, it should be a first-class citizen."

"Democratizing code with an internal open source model accelerates innovation."

"I know I will fail, but I will fail in small increments rather than bet everything and fail big."

Ask the Rosenbot
Sara Asche Anderson
Not Your Ordinary Re-Brand: Design's Path to Driving Customer Obsession at Best Buy
2024 • Enterprise Experience 2020
Gold
Jemma Ahmed
Redefining the research toolkit: Expanding methodologies for a changing world
2025 • Advancing Research 2025
Gold
Christopher Noessel
AI of the now: Designing for Agents
2024 • Rosenfeld Community
Sandra Camacho
Creating More Bias-Proof Designs
2025 • Rosenfeld Community
Marc Fonteijn
Increase your confidence, influence, and impact (through a Professional Community)
2024 • Advancing Service Design 2024
Gold
Erin Weigel
Get Your Whole Team Testing to Design for Impact
2024 • Rosenfeld Community
Brenna Fallon
Learning Over Outcomes
2019 • DesignOps Summit 2019
Gold
Jemma Ahmed
Theme 2 Intro
2024 • Advancing Research 2024
Gold
Dave Malouf
The Past, Present, and Future of DesignOps: a 2-part DesignOps Community Call (Part 2)
2022 • DesignOps Community
Joi Freeman
A New Vantage Point: Building a Pipeline for Multifaceted Research(ers)
2020 • Advancing Research 2020
Gold
Jeff Gothelf
Who does what by how much?
2025 • Advancing Service Design 2025
Conference
Megan Kierstead
You Are a Badass at UX: Overcoming Imposter Syndrome
2021 • Advancing Research 2021
Gold
Bria Alexander
Day 3 Welcome
2024 • DesignOps Summit 2024
Gold
Chris Geison
What is Research Strategy?: A Panel of Research Leaders Discuss this Emergent Question
2021 • Advancing Research Community
Kristen Guth, Ph.D.
Out of the FOG: A Non-traditional Research Approach to Alignment
2023 • Advancing Research 2023
Gold
Dean Broadley
Not Black Enough to be White
2024 • DesignOps Summit 2020
Gold

More Videos

Molly Fargotstein

"When you market UX research, you prove and show your value, make your company smarter by your insights, and create advocates."

Molly Fargotstein

Multipurpose Communication & UX Research Marketing

September 12, 2019

Dave Malouf

"Design turns a 10 from an engineering team into an 11 or a 12; it turns up the volume much more."

Dave Malouf

Closing Keynote: Amplify. Not Optimize.

October 24, 2019

Jorge Arango

"Notes are preeminent things for thinking with because they expand our cognitive abilities."

Jorge Arango

AI as Thought Partner: How to Use LLMs to Transform Your Notes (3rd of 3 seminars)

May 3, 2024

Daniel J. Rosenberg

"The AI is looking through material and that material’s changing every day."

Daniel J. Rosenberg

Designing with and for Artificial Intelligence

August 11, 2022

Alexis Lucio

"No design system will ever be fully accessible because new features and permutations are always being introduced."

Alexis Lucio

Scaling Accessibility Through Design Systems

June 9, 2022

Peter Boersma

"I took roughly three months to get the first shareable version of the roadmap, including onboarding and interviews."

Peter Boersma

How to Define and Maintain a DesignOps Roadmap

October 3, 2023

Ben Reason

"Designers act as mediators and neutral parties to get stuck projects unstuck through design diplomacy."

Ben Reason Aline Horta Majid Iqbal Fabiano Leoni

Making the system visible: The fastest path to better decisions

November 20, 2025

Peter Van Dijck

"We devote 30 to 50 percent of the project time just on evaluation with real users and experts to ensure quality and safety."

Peter Van Dijck Louis Rosenfeld

Coffee with Lou #4: Taking a Peek Under the Rosenbot's Hood

June 14, 2024

Taylor Klassman

"Democratizing research is okay as long as we remain experts in rigorous methods and question questionable research kindly."

Taylor Klassman

Shaping the Next Era of UX Research: Collaborative Forum

March 11, 2025