Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
Gold
Thursday, May 14, 2015 • Enterprise UX 2015
Share the love for this talk
Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
Speakers: Bill Scott
Link:

Summary

Bill recounts his journey from Netflix, where he witnessed the shift from DVDs to streaming, to PayPal, where he led transformative engineering reforms. At Netflix, he realized that designing software as a throwaway experimentation layer rather than permanent code was key, with multiple concurrent experiments driving user-focused learning. He stresses that engineering should enable learning rather than just code stability. At PayPal, Bill faced legacy technical debt and cultural inertia (organizational antibodies) but pushed for a culture of rapid iteration, collaboration, and customer immersion. He implemented a new technology stack based on Node.js and GitHub, democratized innovation through an internal open source model, and emphasized the need to give Agile a 'brain' by embedding continuous user feedback deeply into the backlog and process. Bill highlights the importance of shared vocabulary between disciplines, collaboration, and continuous customer feedback to keep teams aligned and focused on solving real user problems rather than defending solutions. Drawing on examples from Netflix, PayPal, and Meetup, he underscores that successful teams embrace failure in small increments, enable rapid prototyping, and design for volatility.

Key Insights

  • At Netflix, 95% of the UI layer was thrown away within a year, reframing UI as an experimentation layer rather than durable software.

  • Engineering's primary goal should be enabling learning, partnering closely with design and product teams.

  • Different parts of a software stack have different risk profiles; applying the concept of shearing layers helps accept more risk on the user interface layer to enable faster learning.

  • Organizations contain 'antibodies'—cultural and organizational forces resistant to change—that must be understood and navigated to drive transformation.

  • PayPal's transformation was accelerated by a top-down mandate combined with intense cross-functional collaboration and frequent user testing.

  • Using internal GitHub and open source paradigms democratizes code access and innovation, allowing anyone to contribute and experiment.

  • Prototyping should be considered a first-class engineering activity, not a separate or lesser process.

  • Agile methods lack an inherent ‘brain’; embedding continuous customer feedback and real user context into the backlog gives Agile teams direction and purpose.

  • Engineering teams that share vocabulary and deeply collaborate with designers and product managers produce better outcomes.

  • Embracing small incremental failures enables faster learning and avoids the risk of large-scale failures that can paralyze organizations.

Notable Quotes

"I started thinking of the UI layer as the experimentation layer."

"Engineering’s number one goal should be to enable learning."

"You have to design for throw away ability because the majority of experience gets thrown away."

"Organizations contain antibodies—cultural forces that resist change."

"If you don’t soak teams in real customer context, they do dumb things not because they’re dumb, but because they lack context."

"Features eventually become barnacles that are impossible to scrape off the boat."

"Agile needs a brain, and that brain is the continuous customer feedback loop."

"Prototyping isn’t a second-class citizen, it should be a first-class citizen."

"Democratizing code with an internal open source model accelerates innovation."

"I know I will fail, but I will fail in small increments rather than bet everything and fail big."

Ask the Rosenbot
Theresa Neil
Designing for Wellness: Specializing in Healthcare
2024 • Rosenfeld Community
Jack Moffett
SAFe or Sorry?
2019 • Enterprise Community
Erin May
Distributed, Democratized, Decentralized: Finding a Research Model to Support Your Org
2022 • Advancing Research 2022
Gold
Ovetta Sampson
Managing the Human Engagement Risks of AI
2025 • Designing with AI 2025
Gold
Husani Oakley
Bias Towards Action: Building Teams that Build Work
2018 • Enterprise Experience 2018
Gold
Brian T. O’Neill
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
2022 • QuantQual Interest Group
Landon Barnes
Are My Research Findings Actually Meaningful?
2022 • Advancing Research 2022
Gold
Theresa Marwah
How Atlassian is Operationalizing Respect in Research
2020 • Advancing Research Community
DesignOps and The Great Talent War of 2021
2021 • DesignOps Community
Kaaren Hanson
Stop Talking, Start Doing
2017 • Enterprise Experience 2017
Gold
Johnny Michaelsen
Measure Behaviors, Not Results
2026 • Rosenfeld Community
Christopher Taylor Edwards
Design as a Team Practice, A Practical Guide to Cross-functional Collaboration
2021 • DesignOps Summit 2021
Gold
Kate Towsey
Shaping the future of research ops: Expanding roles and strategies for a more integrated research ecosystem
2025 • Advancing Research 2025
Gold
Samuel Proulx
From Standards to Innovation: Why Inclusive Design Wins
2025 • Advancing Service Design 2025
Gold
Erika Kincaid
Connecting the Dots: How to Foster Collaboration and Build a Strong Design Review Culture
2022 • Design at Scale 2022
Gold
Kit Unger
Theme 2: Discussion
2024 • Enterprise Experience 2020
Gold

More Videos

Louis Rosenfeld

"Building a portfolio of talks, refreshed over time, lets you keep evolving and sharing your ideas in different formats."

Louis Rosenfeld Jemma Ahmed Christian Crumlish Uday Gajendar Chris Geison

Coffee with Lou #3: What Makes for a Successful UX Conference Presentation?

May 2, 2024

Jilanna Wilson

"We share the inconvenience of meeting times across time zones, alternating so no one team always bears it."

Jilanna Wilson

Distributed Design Operations Management

October 23, 2019

Matt Webb

"Computers got ten years better overnight with the arrival of GPT-3 in November 2022."

Matt Webb

Context Window: Five Futures for AI

June 11, 2025

Sam Proulx

"If your company has customers with disabilities but no way to collect their feedback, you’re missing critical insights."

Sam Proulx

Accessibility: An Opportunity to Innovate

September 8, 2022

Theresa Neil

"Don’t bluff regulated healthcare UX experience; everyone will know, and it’s harmful to your team and clients."

Theresa Neil

Designing for Wellness: Specializing in Healthcare

May 22, 2024

Peter Van Dijck

"One of the biggest problems in AI building is evolving your prompts and having a fast feedback loop."

Peter Van Dijck

Hands-on AI #2: Understanding evals: LLM as a Judge

October 15, 2025

Rachel Posman

"Design ops and product ops are way more alike than dissimilar; they differ mainly in the orgs they serve."

Rachel Posman John Calhoun

"Ask Me Anything" with Rachel Posman and John Calhoun, Authors of the Upcoming Rosenfeld Book, The Design Conductors

September 25, 2024

Tricia Wang

"It's not about going out to communities but changing your team and leadership to genuinely reflect those communities."

Tricia Wang

The most popular design thinking strategy is BS

January 27, 2022

Sarah Auslander

"It’s a marathon not a sprint. Pace yourself and take care of yourself and the other runners around you."

Sarah Auslander Betsy Ramaccia Gordon Ross

Insights Panel

November 18, 2022