Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
Gold
Thursday, May 14, 2015 • Enterprise UX 2015
Share the love for this talk
Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
Speakers: Bill Scott
Link:

Summary

Bill recounts his journey from Netflix, where he witnessed the shift from DVDs to streaming, to PayPal, where he led transformative engineering reforms. At Netflix, he realized that designing software as a throwaway experimentation layer rather than permanent code was key, with multiple concurrent experiments driving user-focused learning. He stresses that engineering should enable learning rather than just code stability. At PayPal, Bill faced legacy technical debt and cultural inertia (organizational antibodies) but pushed for a culture of rapid iteration, collaboration, and customer immersion. He implemented a new technology stack based on Node.js and GitHub, democratized innovation through an internal open source model, and emphasized the need to give Agile a 'brain' by embedding continuous user feedback deeply into the backlog and process. Bill highlights the importance of shared vocabulary between disciplines, collaboration, and continuous customer feedback to keep teams aligned and focused on solving real user problems rather than defending solutions. Drawing on examples from Netflix, PayPal, and Meetup, he underscores that successful teams embrace failure in small increments, enable rapid prototyping, and design for volatility.

Key Insights

  • At Netflix, 95% of the UI layer was thrown away within a year, reframing UI as an experimentation layer rather than durable software.

  • Engineering's primary goal should be enabling learning, partnering closely with design and product teams.

  • Different parts of a software stack have different risk profiles; applying the concept of shearing layers helps accept more risk on the user interface layer to enable faster learning.

  • Organizations contain 'antibodies'—cultural and organizational forces resistant to change—that must be understood and navigated to drive transformation.

  • PayPal's transformation was accelerated by a top-down mandate combined with intense cross-functional collaboration and frequent user testing.

  • Using internal GitHub and open source paradigms democratizes code access and innovation, allowing anyone to contribute and experiment.

  • Prototyping should be considered a first-class engineering activity, not a separate or lesser process.

  • Agile methods lack an inherent ‘brain’; embedding continuous customer feedback and real user context into the backlog gives Agile teams direction and purpose.

  • Engineering teams that share vocabulary and deeply collaborate with designers and product managers produce better outcomes.

  • Embracing small incremental failures enables faster learning and avoids the risk of large-scale failures that can paralyze organizations.

Notable Quotes

"I started thinking of the UI layer as the experimentation layer."

"Engineering’s number one goal should be to enable learning."

"You have to design for throw away ability because the majority of experience gets thrown away."

"Organizations contain antibodies—cultural forces that resist change."

"If you don’t soak teams in real customer context, they do dumb things not because they’re dumb, but because they lack context."

"Features eventually become barnacles that are impossible to scrape off the boat."

"Agile needs a brain, and that brain is the continuous customer feedback loop."

"Prototyping isn’t a second-class citizen, it should be a first-class citizen."

"Democratizing code with an internal open source model accelerates innovation."

"I know I will fail, but I will fail in small increments rather than bet everything and fail big."

Ask the Rosenbot
Peter Van Dijck
Building impactful AI products for design and product leaders, Part 1: The new product journey
2025 • Rosenfeld Community
Carol Massa
Designing Health: Integrating Service Design, Technology, and Strategy to Transform Patient and Clinician Experiences
2024 • Advancing Service Design 2024
Gold
Bria Alexander
Opening Remarks
2023 • Advancing Research 2023
Gold
Yolanda Rankin
Black Feminist Epistemology as a Critical Framework for Equitable Design
2021 • Advancing Research 2021
Gold
Rachel Posman
A Closer Look at Team Ops and Product Ops (Two Sides of the DesignOps Coin)
2020 • DesignOps Community
Nicole Wright
Democratizing Research at HoneyBook
2022 • Advancing Research 2022
Gold
Maria Giudice
Becoming a Changemaker by Leading with Design
2023 • Advancing Research 2023
Gold
JD Buckley
COMMUNICATE: Discussion
2018 • Enterprise Experience 2018
Gold
Joerg Beringer
Scaling User Research with AI: Continuous Discovery of User Needs in Minutes
2025 • Designing with AI 2025
Gold
Robin Beers
Research as a Catalyst for Organizational Transformation
2021 • Advancing Research 2021
Gold
John Calhoun
Two Sides of the DesignOps Coin: Teams Ops and Product Ops
2024 • DesignOps Summit 2020
Gold
Aurobinda Pradhan
Introduction to Collaborative DesignOps using Cubyts
2022 • DesignOps Summit 2022
Gold
Ren Pope
Building Experiences for Knowledge Systems
2023 • Enterprise UX 2023
Gold
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Changying (Z) Zheng
Navigating Innovation with Integrity
2024 • DesignOps Summit 2024
Gold
Sarah Flamion
Complex Problem? Add Clarity by Combining Research and Systems Thinking
2020 • Advancing Research 2020
Gold

More Videos

Ben Reason

"Scaffolding is about hacking and doing things differently, not lowering the bar but delivering value continuously."

Ben Reason Aline Horta Majid Iqbal Fabiano Leoni

Making the system visible: The fastest path to better decisions

November 20, 2025

Sam Proulx

"Mobile can be moved and angled, allowing users to customize their experience in ways a desktop cannot."

Sam Proulx

Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World

March 10, 2022

Anna Nguyen

"We realized that everyone kind of has a different definition of the voice of customer and that we needed to define our own to make it relevant for us."

Anna Nguyen Emily Brogan

Why Our Voice of the Customer is Better Than Yours

March 10, 2022

Abby Covert

"I want my collaboration tool to be like the Nest thermostat of my workday, knowing when I’m heads down and when I’m open to communication."

Abby Covert Tomer Sharon

Panel: Collaboration Tools

November 6, 2017

Phil Hesketh

"Not all accessibility needs are obvious – even someone with vision changes related to aging develops needs over time."

Phil Hesketh

Designing Accessible Research Workflows

September 29, 2021

Irina Tikhonova

"Employees creating content improved rapidly when centralized guidelines and editing were applied."

Irina Tikhonova Kari Dietrich

Small Wins, Big Impact: Leveraging and Elevating User Engagement

December 9, 2021

Tim Frick

"Accessibility is a line in the sand for us; if you don’t want an accessible website, maybe don’t hire us."

Tim Frick

The journey of building a sustainable design practice

April 23, 2025

Catherine Dubut

"It almost feels like cheating because your users are showing and telling you what they need. It's almost too easy."

Catherine Dubut

Bridging Physical and Digital Spaces: Approaches to Retail Service Design

March 18, 2021

Justin Entzminger

"We recruited locally and from historically Black colleges and universities to build a diverse and talented innovation team."

Justin Entzminger Terrance Smith Tracy M. Colunga Mai-Ling Garcia

Risk and Reward: How to Diversify the Field of Civic Innovators and Designers

November 17, 2022