Lean-Scrum: The Next Generation of Scrum


Scrum is the most popular Agile approach today, so much so that many confuse Agile with Scrum.  But Scrum is only one approach, and, in fact, there is more than one approach to Scrum. This article discusses how Net Objectives and others are now doing Scrum – Scrum within the context of Lean-Principles and Kanban practices.  This is not the same as adding some Lean into the standard Scrum approach, rather it represents a new mindset that has proven to be more effective than Classic Scrum[1] in many situations.

The Roots of Scrum

Scrum started out as a framework for performing software development.  Its roots are in Lean manufacturing, being inspired by the article “The New New Product Development Game” by Takeuchi and Nonaka.   The concept was to provide a development team with a mandate of what to build but let them figure out how to get there.  The team would be “cross-functional” meaning they would have all of the skills needed to create the software.  They would self-organize with the intention of creating code and demonstrating it on a monthly basis The Scrum Master’s role was to coach the team in continuous improvement, act as a liaison to management to remove impediments and to keep the Scrum board up to date so both the team and management would be able to see the progress, or lack of it, being made. The product owner role was to provide guidance for what functionality was required.

Scrum established the following as effective practices:

  • Build working code on a regular cadence in order to reduce technical risk and validate the direction the team is headed
  • Provide visibility to management about the progress the team is doing
  • Create a framework for learning so the team can improve its methods
  • Let the team decide how to do their work so that they can avoid unnecessary practices

Scrum represented a breakthrough in software development methods.

The Mindset of Lean-Scrum

With the adoption of Lean Software Development and Kanban a few years later, a new paradigm for Scrum emerged.  Around 2005 I claimed that Scrum could be thought of as a partial implementation of Lean-Thinking.  That time-boxing created smallish batch sizes, cross-functional teams improved flow and self-organization was an example of decentralizing decision making – all Lean concepts.  My thinking was if Scrum was an implicit implementation of Lean then using Lean principles to guide Scrum explicitly could accelerate the adoption of Scrum by new teams. Our evidence was that new Scrum teams seemed to fall into similar traps of behavior, behavior that Lean-Thinking told you explicitly to avoid.

Lean-Thinking is too broad a subject to detail here, but its main precepts are:

  • View software development from a systems-thinking perspective. Understand that the behavior of people developing software is affected by the systems within which they are. This does not dilute the importance of people.  It actually emphasizes our respect for them, saying we shouldn’t attribute bad motivations for failure, but rather look to see how to improve systems to prevent it.
  • Attend to the flow of work. In particular, remove delays in the workflow.  Delays cause problems and waste.  There is a science to flow that can be used to remove these delays, thereby increasing efficiency and shortening time to the delivery of value.
  • Create visibility of the work flow.  This enables better team communication within a team, between teams and even the development teams and their management.  It also promotes continuous improvement.
  • Ensure management can see the workflow processes the team uses.  This enables management to learn how to best work with the team – both to reduce impediments and not to create any
  • Manage work-in-process (WIP) levels by instituting a pull system at the work item level.
  • Work on small batches to achieve efficiency of work both by avoiding delays and improving feedback loops.
  • Take an attitude of removing waste by improving the process

Lean-Scrum is manifesting Scrum within these concepts.  That is, our foundation is these principles, and Scrum is an approach we have chosen to take for the situation we find ourselves in.

How Lean Extends Scrum

The Lean-Scrum approach has been gaining a lot of ground because most people find it more effective. Some of the reasons for this are:

  • It makes explicit much of what developers either intuitively know or figure out after a few sprints
  • It enables teams new to Scrum to build on top of practices that others have established
  • It  can help avoid “Scrum-but” (i.e., the label many classic Scrum minded coaches put on people who say “we do Scrum but we don’t do ….”  Scrum-but often happens because some aspects of Scrum are difficult to implement.  This is actually as intended, the supposition being if the practices were implemented, the team’s performance would improve. This is true.  But Lean-Thinking often provides insights into why these practices are essential, thereby providing the additional motivation to implement them, or alternative practices that achieve the same outcome – thereby achieving what’s necessary even if not being pure Scrum

The Practices of Lean-Scrum

Lean-Software Development and Kanban provide a number of practices that are useful for virtually all Scrum teams. I will list these here and then go through each in detail:

  • Make workflow policies explicit
  • Include management
  • Create visibility of the workflow inside the team
  • Manage work in process (WIP) levels
  • Acceptance Test-Driven Development
  • Use Minimum Business Increments (MBIs) to drive releases
  • Decompose requirements into small stories (1-3 days to complete)

Make Workflow Policies Explicit

Scrum has always had a mandate for some workflow policies to be explicit:

  • What’s going to be worked on during the Sprint (work the team is willing to commit to)
  • What it takes for an item to be considered completed (it must be tested)
  • What to do if someone outside the team attempts to add additional work into the Sprint (abort the Sprint)

Over the last few years, additional policies have become a de-facto part of Scrum:

  • Don’t let something into the sprint unless you know how to get it out of the sprint
  • Measure team velocity and use that to assist in the selection of what will be in a sprint

However, Lean-Scrum suggests that workflow policies be explicit inside the Sprint as well[2].  This means to have the teams discuss how they do their work and to establish workflow policies amongst themselves.  These policies are meant to encourage team collaboration and understanding of how the team works.  It is not intended to establish policies that the team must follow.  Explicit means discussed, not prescriptive, set or even documented – just discussed.  Having a common understanding of workflow also means that making changes can be better understood.

Include Management

Nowhere is the disparity between Classic Scrum and Lean-Scrum greater than in how management is held.  While the Chicken and Pigs story[3] is fortunately fading into the past, it belies an attitude by many in the Agile community that the team is more important than management and usually more committed to Agile. True, the team is where the product is built, but proper management is required when several teams are involved.  The popular Scrum-of-Scrums approach suggested by Scrum has a poor track-record.  Peer-to-peer organization, while exiting to many, does not have a track record of success without management involvement. 

Only one of management’s responsibilities is to remove impediments from the teams.  Another is to help achieve proper systems within which teams can work effectively.  The focus needs to be on the bigger picture of delivering business value to the customer, not merely accomplishing team iterations.  This almost always requires management involvement.

Create visibility of the Workflow Inside the Team

Ken Schwaber, one of the co-creators of Scrum, describes the team’s work as a black-box process.  Because software development is complex, he asserts that the process must not be well-defined.  I have heard others that the team must protect itself from management interference by not exposing their methods. This is clearly part of the “chicken and pig” mentality Lean-Scrum is saying to shed. The Lean community does not agree with either of these perspectives. Lean suggests that well-defined and explicit policies assist in understanding, communication, and continuous learning.  It is a cornerstone of Deming’s Plan-Do-Check-Act (PDCA) cycle that is the foundation for Lean’s incremental learning approach (Kaizen).   Visibility of a team’s process is essential for an understanding of all interested parties.  It enables management to see what is happening and provides insights for them to both help the team and avoid hurting the team. 

Measure Work in Process Levels (WIP)

This is a fundamental practice of Lean and Kanban.  Too much WIP causes task switching and delays. Task switching happens as people work on too many things.  But the delays this creates is even worse.  People have to wait on others that are busy working on some other unrelated story.  These delays literally increase the amount of work because when the work is resumed it takes a while to recreate where the person was. The greater this delay the greater this amount of work.  I call this extra work “induced work” because it is created by delay[4]

WIP can be managed in in several ways.  At a minimum, team members can agree to always consider if they can help out another team member finish a story instead of starting a new one.  Pairing in this way keeps the number of stories open to a minimum.  If explicit policies have been created, teams can take things a step further by creating explicit WIP limits for each step of work.  Other methods include creating sub-teams dynamically to work on features that require specialized skills.

Acceptance Test-Driven Development (ATDD)[5]

Lean suggests the role of test and quality assurance is not to test quality in, but to improve the process that builds the code. One implication of this is to ensure an understanding of what is being built from the very beginning.  ATDD, and the related Behavioral Driven Development (BDD) is the practice of creating acceptance tests prior to writing any code.  An acceptance test is an explicit description of what it takes for a piece of code to be considered done.  The key to ATDD is to have the product owner and the development team discuss the suggested requirement and create specific tests that will validate it was built correctly.  The power of ATDD is not merely that it helps the development team have a target to build to, the real power is how the conversation about requirements changes when the people involved in specifying, building and validating the code talk together.  This three way conversation can drive out hidden assumptions that can either be done via the conversation that takes place when the tests are specified, or when the product owner eventually sees the code.

There are actually different degrees of following ATDD[6]:

  • The development team creates tests which are validated by the product owner
  • The development team creates the tests with the product owners
  • The tests are implemented
  • The tests are automated

ATDD enables quick feedback loops, and assists in the creation of small stories – described next.

Use Minimum Business Increments (MBIs) to drive releases

The focus on Agile should be to release business value quickly, not on team iterations.  This means that product owners must start with the minimum business value that can be released and then decompose those into stories.  By keeping the end release in mind during development one can keep scope focused.  MBIs go by many other names – MMFs, MVFs, MVP, …  We prefer minimum business increment to remind everyone that we are driving towards creating value to the business by adding value to the customer.

Decompose requirements into small stories (1-3 days to complete)

Scrum merely states that stories need to be finished within a sprint.  However, Lean suggests that small batch sizes are critical.  Scrum has three different types of batch – the release, the sprint and the story.  While Scrum gives some explicit attention to release and sprint size, it not discuss story size.  Our experience is that small stories, taking about 1-3 days to complete, are significantly better than having larger stories.  There are several reasons for this are that small stories

  • facilitate the creation of acceptance tests
  • enable better collaboration within a team
  • speed up feedback loops
  • increase discipline
  • encourage developers and testers to work together (I know these roles are not supposed to exist in Scrum but they often do)

Unfortunately, creating smaller stories is often difficult, especially in the development of complicated products (e.g., engineering type applications).   Many developers have challenges believing their work can be broken up into stories taking only a few days to implement.  We have found, however, that it is possible if one uses a particular story template, now popularized by Cucumber (a tool used for testing):

 “Given a <initial state>, and <event occurs>, then <asserted final state>”

This template enables one to pull off specific cases of arbitrary size, essentially splitting up stories as small as desired.   Doing this helps in analysis because it creates a method of creating smaller cases to investigate than what at first may appear to be a bewildering set of related cases.

A Cautionary Tale

Lean-Scrum didn’t just spring up capriciously.  It was created because many of us who knew Lean saw teams struggling with Scrum and knew Lean-Thinking could help. We extended Scrum because doing so improved the speed and quality at which teams adopted Scrum. Nearly a decade later, many who resisted implementing Scrum within the context of Lean are starting to acknowledge the value in doing so.  I caution the reader to attend to the difference between implementing Scrum from within a Lean-Thinking background and adding a few Lean practices to the classic Scrum framework.  There is a difference.

In Summation

Scrum is a great team process.  Doing Scrum within the context of Lean makes it better. In particular, Scrum teams need to understand the Lean principles on which Scrum is based and the Lean and Kanban practices that can be incorporated into Scrum. 

[1] Our term for the original Scrum mindset. This may include Lean and/or Kanban practices but does not start with  the principles of Lean.

[2] Note: This runs counter to Ken Schwaber’s mandate that the team’s process be black box and not explicit.  The concern is that if one creates explicit policies that teams will start blindly following them.  Our experience is that this is not a valid concern in Lean-Scrum

[3] The Chicken and Pigs story  is an old joke that goes like this:  A chicken and a pig were talking and the chicken told the pig – “we should start a restaurant and serve breakfast.”  The pig asked – “and what would be on our menu?”  The chicken responded “ham and eggs.”  After a moment’s consideration, the pig said “I don’t think so, while you’d be interested, I’d be committed.” Scrum mandates that only people committed to the Sprint commitment are allowed to attend the daily standup.  This story is often used to describe management as “chickens” and not allowed to attend. But this implies that management is not committed to the Sprint.  While you may or may not want management to be at the daily standups (in many cases it is why to preclude them) to pre-suppose management is less committed than the team shows a disrespectful attitude towards management.