Mindsets: Waterfall, 1st & 2nd Generation Agile

April 28, 2012 — Posted by Al Shalloway

“We can't solve problems by using the same kind of thinking we used when we created them.” Albert Einstein

I had mentioned in my last blog that I’d be discussing how to pick which practices you should be doing.  However, in thinking about this, I realized that there is one other step you must first get clear on.  This is being clear about the mindset you have and whether it works well for what you are trying to accomplish.  Mindsets are often difficult for people to analyze and/or challenge. Mindsets typically show up as the way it is so people don’t challenge them.  Given most people in the Agile community now appreciate the difference between Waterfall and Agile, I think it could be useful to create a table with how those mindsets differ along with how 1st and 2nd generation mindsets often differ.  I’ve written a couple of earlier blogs on this that you might find of interest: and The Importance of Mindset and The Importance of Mindsets – Part II. Of course, these comparisons are not literally correct. People typically haven't talked about mindsets.  I am just trying to expose different ways of thinking.

I want to be clear why I am comparing mindsets here. It is not an attempt to show one mindset is more correct or useful than another.  Certainly, there is not a best mindset for all circumstances.  However, many of us believe our mindset reflects reality.  It does not - it is just how we frame reality.  When we can see that there are alternative mindsets, it helps defeat this myth.  We are then free to look to see if our mindset is serving us or limiting us.  We then have more freedom to change our mindset to something that works better for us in the situation we find ourselves in.  Unfortunately, few people take the time to reflect on their own mindset but tend to defend it in an attempt to prove its worth. This is because many people identify themselves with their mindset.  This is also unfortunate as it severely limits their ability to reflect on their mindset as doing so feels like a reflection on their own self-worth.

Where 1st and 2nd Generation Methods Are Similar to Each Other

 

Waterfall

1st Generation Agile

2nd Generation Agile

Requirements

Requirements can and should be detailed in advance before work on implementing them begins.

Customers will not understand their requirements fully.  Even if they do, developers will not be able to understand them perfectly.  It is better to incrementally discover the requirements by building part of the system and learning from it.

Same as 1st Generation

Delivery of value

Best to deliver in full, complete chunks

Deliver value incrementally.

Same as 1st Generation.  Lean, Kanban and Lean-start up put some extra emphasis on Minimal marketable features, Minimal viable products, etc.

Main Areas Where 1st and 2nd Generation Methods Differ

How To Start a Transition from Waterfall

N/A

Start at the team. Have impediments to the team dictate how improve the rest of the organization.  If necessary, define new roles and practices and follow them until the team "gets is."  Remove impediments to these practices.

Take an holistic view of the entire value stream. Look at what is the root cause of the impediments to flow and work on that.  Start with what you do now, initially, respect current roles, responsibilities, and job titles.  Change once visibility and understanding are present.

How to Change Culture N/A Change culture by insisting on certain practices until the team gets better with them. This was extreme (pun intended) during the initial phases in XP where teams were warned that if one didn't do all 12 practices it wouldn't stick.  Scrum has continued this by chastising those who don't do Scrum properly as Scrum-butters.  The challenge is that even when the practices are followed things often don't stick when impediments due to otherh parts of the organization, those not following the practices, are encountered. Change culture by starting where you are, create an understanding, and attempt to create a culture of continuous improvement. Side note: this is not as satisfying at first in that you can't point to big changes and show how much success you are having.  However, it is much more likely to still be growing and improving a year later
Respecting People We may respect them personally, but don't provide much respect professionally.  We have management tell them what to do. Respect means professionally and personally.  Let them figure out how to self-organize their work. Respect in same way as first generation Agile, but also respect the human condition.  That is, people don't like too much change.  Accept how people learn and create an environment in which they can grow.  Just saying do it, just doesn't do it.  Acknowledge it isn't a technical problem, it's a people problem.
What Is Agile and What Is Agile Based On? N/A Agile is based on values and principles. Agile is based on values, principles, an understanding of human behavior and product development flow.
Approach to learning People can be told what to do and trained in basic skills needed. People learn best by doing.  Understanding will come after people have done the work for a while.  Retrospectives at the end of iterations is frequent enough for re-thinking and improving one's process. People learn best by doing while also being presented with the theory of why the suggested practices work. Learning what to do may be complicated, getting people on board with it is complex - hence a combination of practice and theory is essential.  A continuous learning model, a la Rother's Toyota Kata, is ideal.

Organization and focus of Requirements

Top-down, providing full detail

Requirements focus on the customer.  They are gathered in the form of stories and decomposed into manageable chunks of work.  These stories are put together into releases. This is a somewhat small pieces put together to a release approach.

Requirements focus on the customer within the value proposition of the business. Because of the product portfolio view of 2nd generation methods, it is realized that just focusing on customer value may have us continue creating value for this customer set when we should switch to building value for another set.  Requirements are created from a business capability perspective using Minimal Marketable Features or Minimal Viable Products, etc., to create a release plan within which stories are developed.

How to achieve improvement

Focus on management telling workers the best way to do the work

Focus on the people doing the work figuring out the best way to do it and telling management how management can help them.

Have management create an environment for people to do their best and have people self-organize within that environment

The relationship between people and process

Use process to tell people what to do

People over process

People and process (i.e., process is important, but must be created by and serve people)

Management

Management dictates process to teams

Somewhat ignored initially other than making it clear management should not interrupt the team. Teams do provide visibility of the progress and results of their work to the team, during and at the end of  the iteration. While many in the Scrum community have finally abandoned the chickens and pigs story, the fact that it ever arose in it indicates the change in attitude about management.  It is inconceivable that the story could ever have occurred in Lean/Kanban as close working of management and teams is one of their tenets.

Teams recognize that management is essential to the creation of valuable software.   In addition to the visibility provided by 1st generation methods, 2nd generation methods make their workflow visible.  This happens through the explicit policies of the teams’ workflow.

Level of workflow definition

Workflow is defined and stable.  Teams are intended to follow the workflow.

It is not possible to explicitly define workflow except on entry to the team’s work and on exit from the team’s workflow.

Workflow is defined from the input queue of the team until it is delivered

Scope of methods

Look at the entire value stream.

Focus on the team

Look at the entire value stream.

Team Structure

Teams are fungible and can be put together as needed to maximize utilization.

Organize people as cross-functional teams and keep them fixed.

Cross-functional teams are most effective, but are not necessary.

Workload

Planned completely ahead an given to teams

Pulled by team in batches.  Manage workload implicitly via time-boxes called iterations.

Pulled by team in small pieces. Manage workload explicitly via work in progress limits.

Other Differences Between 1st and 2nd Generation Methods

Time-Boxing

Not used

Time-boxing used for managing iterations.

Time-boxing for managing flow discouraged, but can be done. Cadence used to coordinate activities, reviews, data gathering.

Conformance

It is important to conform to the specification

Conformance is not something to strive for.

Conformance to the best workflow you know (and changing it when you learn) is a good thing.

Technical – Design

Design the system up front

Little to no design is needed.  Complete design via emergent design is possible.

 

Author: 

Share this:

About the author | Al Shalloway

Al Shalloway is the founder and CEO of Net Objectives. With over 40 years of experience, Alan is an industry thought leader in Lean, Kanban, product portfolio management, SAFe, Scrum and agile design.


Comments

I agree. i think i said this in the opening as a matter of fact.  I also agree Agile isn't for everyone.  I actually call Agile #2 "Lean-Agile" because the lean aspect of it can be applied (albeit in different ways) to virtually every software development group I've seen.  The potential for improvemnt varies, of course.

One thing I have learned in life is the difference between a cynic (which I do not like) and a skeptic (which I do like).  A cynic knows something will fail before it does because they've previously seen something like it fail. A skeptic may have also seen something fail earlier, but they will now take this knowledge and use it to help what they want to have happen from failir. Skepticism opens up new possibilities, cynecism closes them.

 

We are destined to repeat them.

Thanks for clarifying. I am sceptical, born from experience. I've lived through BPR. TQM, Six Sigma, MBO, Lean, etc. and seen them all come and go as pasing fads. Not saying that there wasn't value in each, but the proponents of each all over promised and under delivered because they chose to make claims of universality.

So the next question is if Lean-Agile is indeed different from Agile, does it offer the same potential benefits? - my feeling is no, but yu could rightly describe that as cynicism :)

Lean-Agile as you describe it sounds a useful approach, but again I don't believe it is universal. Lean isn't universal. 2% of Lean adoptions are successful, the rest fail.

Some organisations will find the values in Lean-Agile appealing, some won't. This isn't the fault of Lean-Agile in the same way it isn't the fault of Agile or any other approach that relies on people and their values for success.

Paul.

I have no doubt that many people have taken up lean as a fad.  But that doesn' not mean lean _is_ a fad.  It continues to evolve and has added great value to many many organizations.  Your 2% number is incredibly mis-leading.  Lean's intent is a very high bar.  Many of what would be considered to be the 98% of failures represents significant improvement but not achieving Toyota's level of continuous improvement / learning. In any event, our clients have more like a 75% success rate with Lean-Agile adoption.  Perhaps instead of looking at what can go wrong, perhaps you should start looking at why some work and some don't.

I would suggest discussing particulars.  The gross generalizations you make here don't add much value.  I'd be interested in why you think things fail and what you could do.  Just saying that every new thing is over-promised and delivers under-value adds little value.   I'll likely respond to specifics, but may not bother with these continual over-generalizations coupled with the implication that if I believe something will make a difference with our clients (regardless of the massive evidence we have) them I am just promoting a fad.

BTW: All cynics think they are skeptics. 

Hi Alan,

It is not about name calling :) I agree with yu that the 98% lean failure rate needs to be supported by evidence. It turns out that like most things in our industry that the evidence is only anecdotal, but even this rebuffal of the 98% claim makes dismal reading:

http://softwareflow.wordpress.com/2010/11/25/debunked-leans-98-failure-r...

I have my own anecdotes of course because I experienced such failures first had :)

On to the subjects of fads. Lean is both a fad and not a fad at the same time, same with Agile, and no doubt the same with lean-Agile if it builds momentum. What makes something a fad is the claims of the proponents and how those claims are used to recruit adopters, and how false claims (over promising) leads to eventual disappointment followed by mass rejection:

http://hear-see-do.com/AM/projects/hartwick%20miller%20management%20fad%...

Fortunately these researchers have backed up their research with hard data. Hard to refute. So the thing to look out for when spotting a fad iis the claims made. Particularly the claim of universality. Not all proponents make the same claims.

Paul.

I agree with the fad part.  I think agile has turned into a fad, scrum in particuilar.  While there is value there, it is being overstated.  Kanban runs the risk of it being a fad, but that hasn't happened yet - and I'm doing what I can to avoid that.

Lean, as you say, is and isn't a fad. :)

Re the 98%, i'm not arguing with the number, i'm saying there are different bars.  If a company can use lean insights to have their software development organization become 50% more efficient, that's a good thing even if they don't become lean.  I would suggest that's achievable 75% plus of the time.   I make the distinction between this productivity improvement and Scrum's mantra of high-performing teams as what I am discussing is at the organizational level.  Achieving individual team improvements of 3-20 times is not tremendously difficult.  Unfortunately, that in and of itself does not translate to higher organizational productivity.

 

Yes I agree. Organisations do stupid things. Many could improve their software delivery efficiency by at least 50% if they simply did less (matching demand to capacity)

If this is what you mean by Lean then I totally agree. This isn't what Clifford Ransom, the originator of the 2-3% success claim was talking about though. He was talking about a noticeable improvement in financial results.

An organisation can become more efficient at doing the wrong things. Like I say I know this from first hand experience. The real challenge, especially for knowledge work is doing the right things and being effective. Assuming that financial performance is a good indicator of organisational effectiveness, then a 2-3% figure is nothing to crow about.

Tying this in with fads; then your 75% success claim really should be qualified by your definition of success, which you've now done.

Thanks,

Paul.

There are very few companies that are 'lean' a la toyota (or at least how they were).

But achieving a 50% improvement for a software development organization is nothing to sneeze at either.

Too many folks are still working on the wrong paradigm - worrying about utilization.  If the agile world would attend to the lean paradigm explicitly we'd see a much higher rate of improvement.  After flow was accepted, perhaps true continuous improvement a la lean management would be more possible.

Thanks for sticking with this Alan. In the UK many years ago British Telecom had a TV advert with the punch line, "it's good to talk".

Agile assumes that the organisation is past focusing on utilisation, and is looking to be as effective as possible in a fast moving and dynamic market.

Like you say, for many companies this is a false assumption, and their easiest path to "improvement" is getting past 100% utilisation by introducing slack.

This is contextual, it is not universally true. If all of us spoke more about when, where and why different ideas make sense, and stopped promoting our pet ideas as a universal cure for everyone, then the spectre of becoming yet another passing fad would be much less.

The only reason why I bang the Agile drum is that because once you've harvested all the low hanging fruit by introducing Lean "processes", true success still relies on people, values and inginuity. This is where Deming was going with his fourth pillar. It is also the area of focus for Nonaka with his study of organisational learning, knowledge management and tacit knowledge, leading to what we call Agile today.

When you have a deep understanding, all these ideas are related. When to focus on each becomes contextual.

Paul.

Pages

Free Email Updates!

Sign up for free email updates from
Net Objectives Thoughts Blog

Blog Authors

Al Shalloway
Business, Operations, Process, Sales, Agile Design and Patterns, Personal Development, Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Cory Foy
Change Management, Innovation Games, Team Agility, Transitioning to Agile
Jim Trott
Business and Strategy Development, Analysis and Design Methods, Change Management, Knowledge Management, Lean Implementation, Team Agility, Transitioning to Agile, Workflow, Technical Writing, Certifications, Coaching, Mentoring, Online Training, Professional Development, Agile, Lean-Agile, SAFe, Kanban
Ken Pugh
Software Design, Design Patterns, Technical Writing, TDD, ATDD, Coaching, Mentoring, Professional Development, Agile, Lean-Agile, SAFe, Scrum
Scott Bain
Analysis and Design Methods, Agile Design and Patterns, Software Design, Design Patterns, Technical Writing, TDD, Coaching, Mentoring, Online Training, Professional Development, Agile