Differences in Beliefs Results in Differences in Approach

July 5, 2009 — Posted by Al Shalloway

This blog is a synopsis of my thoughts resulting from a “conversation” Paul Oldfield and I had on the Lean-Agile Yahoo group recently.  I have added other things than were not in the original thread.  Thanks Paul, for a very interesting dialog. And thanks to Bob Marshall (twitter ID: flowchainsensei) for suggesting I write something like this up.

I’ve been thinking about the differences in results that I have seen in teams doing Agile (mostly Scrum) who know Lean and those who don’t.  One difference is their belief systems. I do believe that some belief systems are better than others (no pun intended).  While practices vary in effectiveness in different contexts some belief systems are virtually always more useful than others. In the area we (Net Objectives) typically works in (development organizations of at least 50, and usually between 300-1000) we have found Lean-Thinking to be extremely useful.  But it is the belief system of Lean itself that is more important than the Lean practices themselves.

I’ll describe four general areas where our belief systems are different than what we mostly see in the Agile community who haven’t embraced Lean thinking.  These are:

  1. To what extent should a process be defined?
  2. Should you use a small or large framework of thinking?
  3. What extent should management be involved in the process?
  4. Should you start with a team focus or an enterprise focus?

I’ll go through each of these and then summarize my thoughts.

To What Extent Should a Process Be Defined?

One of the common beliefs of all Agilists is that you must take advantage of feedback. The belief underneath this is that what we are doing cannot be exactly planned up front – so adjusting as we go is required.  A good belief.  Most Agilists would also agree that there be some sort of workflow – that is, you note when work has started, what stage it is in (test, coding, validation) and when it is finished. There is a fairly large discrepancy, however, between whether that workflow should be defined or not.   

Lean believes in having a well-defined workflow which you apply Plan Do Check Act (PDCA) against.  This workflow is not a static process, however.  Rather, it is team’s baseline for change.  In other words, the workflow incorporates the team’s understanding of the best way to do things. When the team sees a better way, they modify their workflow.  Typically, in Lean this workflow is designed to limit the team’s work in process (WIP) to the limit of the teams' capabilities (among other things).  It is also intended to generally shorten cycle time.  To incorporate risk issues.

My experience is that most Scrum practitioners do not believe in a well-defined workflow.  By well-defined, I mean explicitly describing how you go from one state in the workflow to another.  In other words, it’s not enough to say we have the states of: not started, started, tested, waiting for validation.  You must have the team understand what it takes pick something off the backlog; when do we test?; how much work do you allow to be in process? 

These are issues that I believe should be dealt with explicitly.  However, they are never put on the team from outside, but are figured out by the team.  This is a defined workflow. I have seen many Scrum teams get in trouble because there are not explicit rules about what it takes to start a story and too many stories get started and also don’t get finished by the end of the iteration as a result. I think many people in the Agile world don’t trust well-defined workflows because waterfall was a well-defined workflow.  But Waterfall was also a workflow that didn’t take much advantage of feedback – which is definitely bad.  So perhaps that is more of the reason Waterfall fails in most situations.
My belief as to why most Scrum practitioners don’t define their workflow is twofold.  First, the iconic leader of Scrum, Ken Schwaber, says not to define one.  This is based on the belief that you have choices between black box or deterministic processes - as described in Ken's book - Agile Software Development with Scrum (p 25): 

In a nutshell, there are two major approaches to controlling any process. The "defined" process control model requires that every piece of work be completely understood. Given a well-defined set of inputs, the same outputs are generated every time. A defined process can be started and allowed to run until completion, with the same results every time. ...

... the empirical model of process control, on the other hand, expects the unexpected. It provides and exercises control through frequent inspection and adaptation for processes that are imperfectly defined and generate unpredictable and unrepeatable outputs.

I don't agree with this thinking. An excerpt from our book Lean-Agile Software Development: Achieving Enterprise Agility discusses this (special thanks to Don Reinertsen for giving us an initial description of this):

One of the most frequent questions we get in our work in the field is “how can you predict what is going to happen if you are doing an Agile process?”  We believe that this question comes from a misunderstanding of some key issues that underlie process. 

Let’s start by cleaning up the terminology. We can view the output of a process as deterministic or non-deterministic (called stochastic). In a deterministic process the outputs are 100 percent determined by the inputs. In a stochastic process the output is a random variable—it has different values that occur with different probabilities.

Fully-determined systems do not exist, except in academia and thought experiments. Virtually all real world systems of manufacturing and development have stochastic outputs. That is, they are partially determined.

It is useful to make a distinction between whether a process is fully-determined and whether its output is fully-determined. Although many people have a tendency to assume a defined process will produce a deterministic output, this is not always true—a precisely defined process can still produce an output that is random. For example, the process for obtaining and summing the results of flips of a fair coin may be precisely defined, while its output is a random variable.

Well-defined systems can produce outputs that range on a continuum from deterministic to purely stochastic.  Just as we can structure a financial portfolio  to change the variance in its future value—by ranging from all cash to all equity—we can make design choices that affect amount of the variance in a systems output.

Thinking of system output as a random variable may be more useful than labeling it as either unpredictable or predictable. We could think of the output of a system as completely unpredictable, macroscopically predictable, or microscopically predictable.  It is unclear if anything falls into the first category—even a random number generator will produce uniformly distributed random numbers. It is the zone of what we would call “macroscopic” and “microscopic” predictability that is most interesting.

We can make this distinction using the coin-tossing analogy. When we toss a fair coin 1000 times, we cannot predict whether the outcome of the next coin toss will be a head or tail—we would call these individual outcomes “microscopically unpredictable.” There may be other microscopic outcomes that are fully determined since we have a fully-defined process. For example, we could define this process such that there is a zero percent chance that the coin will land on its edge and remain upright. (If coin land on its edge, then re-toss the coin.)
 Even when the outcome of an individual trial is “microscopically unpredictable,” it is still a random variable.  As such, it may have “macroscopic” or bulk properties that are highly predictable. For example, we can forecast the mean number of heads and its variance with great precision. Thus, just because the output of a process is stochastic, and described by a random variable, does not mean that it is “unpredictable.”  This is a rather important point because the derived random variables describing the “bulk properties” of a system are typically the most practical way to control a stochastic process.

The degree of feedback needed is another variable we should add to our duo of degree of predictability and degree of process definition.  In the software world, feedback is probably essential.  In other areas it may not be.  That is, in our world, feedback is probably the most cost-effective way to achieve our goal – but it is really an economic decision on how/when to use it.

Therefore, we consider it useful to think of processes as having the following:

  • A degree of process definition.
  • The randomness of its output.
  • The amount of feedback that the process uses.

It is important not to confuse process definition with the level of determinism or the amount of feedback required to keep things on track. The importance of this section is to understand that although we may not be able to microscopically predict the result of each story, we should be able to macroscopically predict the timing and cost of the business capabilities encompassed in our features.

The second reason I’ve seen comes from conversations I’ve had with many in the Scrum community.  It seems that a large segment of the community likes that the team comes from values more than principles – having a slim framework is good to them. This puts a lot of importance on the people – the “people over process” viewpoint of the Agile Manifesto. I have always felt that “people over process” was a good counter-reaction to the heavy process at the time of the Agile manifesto.  But I believe people and process (or as Reinertsen has said – “people times process” implying if either are small, the product is small) is better. I have heard John Shoock (a noted Lean expert and author of “Managing To Learn”) call it a balance between people and process. 

So what is the impact of these two different belief systems?  And where would one work better than another?  Let’s start this exploration by looking at what happens when there is no well-defined process but people just do what appears to be right versus one where you have a well-defined workflow and you look to see how it is working for you.

In the first case you have to ask how people are doing?  What's impeding them?  Are they having problems?  The discussion is a lot about the people (not surprising since many value people over process).  Now, if a person identifies themselves with their work, it should not be surprising that they may get defensive or alarmed at this.  We're talking about how she is doing, how she needs to change his actions.  I have had a lot of people express fear when they are about to start Scrum because they have to make promises and then they have to talk about how they are doing with these promises every day.  A bit intimidating if you take pride in and identify with your work (as many of us left-brained people do).

Now, how does it work with an explicitly defined workflow?  You talk about how the workflow is doing.  Oh, we aren't doing well - we have too much work in process, or something like that.  How can we fix that? How can we get less work started? How can we change the process?  Ironically, even though we are focused on the process, the conversation is more about the team here than the individual.  I say “ironic” because although Scrum is more team focused than Lean is, in practice it seems that in team retrospections, teams using Lean-Thinking (e.g., Kanban teams) are more team focused than teams not using Lean-Thinking.

We do ask how people are following the process - and if not - do we need to change it?  We do talk about people and how they are doing, but we do it in the context of how is the process working for us - not how a particular individual is doing.  The focus is more on the work, than on the people (other than how the work is helping people).  Experience reports at the Lean-Kanban conference in Miami illustrated how this led to much better continuous process improvement and team morale than what was typically seen with other Agile methods.

OK, so is one better than the other?  I would say again, you have to look at the context of your project.  If you have a small group of people with established high trust levels that understand that they are talking about the work and not the quality of the people (high respect too) and there is little/no fear, then the Scrum model will work quite well (at least at the team level).  People will adjust, people will not take insult, people will be coached.  Now if you don't have these high levels of trust and respect from management, this will still work if management doesn't need to see exactly what you are doing.  They only need to see this, by the way, if you need their help in changing how things work in your company.  If you don't - no worries.  If you do - then I think you may have impediments that will be difficult to solve.

On the other hand, if trust is low, respect mediocre or people tend to get defensive, then using Lean's methods will create a safer environment for people. You can get management involved more easily because you can talk about your workflow and ask for help in improving it.  You are not exposing weaknesses in your people.   You also will be better able to transfer the knowledge to other groups (which will only be an advantage when you have larger groups - so it's not always better).

The real question to me is not Lean versus Scrum but whether you believe in having a defined workflow or not.  I do.  Many others don’t. I am clear that many Scrum practitioners will tell you that Ken notwithstanding, a team can have an explicitly defined workflow if they want one.  However, to me, a process/framework (e.g., Scrum) that doesn’t say something about whether you should do something leaves something to be desired.  People who are not experts need guidance – so not talking about something will certainly reduce the opportunity to try something that would (in my opinion) be useful to them.

Of course, smart people will always need to make smart decisions. But having smart people focus on their process is a good thing. Also, it is important to understand that defined may not be as explicit as:

when X happens do Y

It could be: when selecting a new item from the product backlog, swarm on it so as to lower integration time. You may have to figure out how to do this but you'd have an explicit rule to follow. You could make it more defined by adding – pick team members so all needed skills are present the entire time you are working on it. Scrum would suggest handling this last one by have generalized team members. Unfortunately, this is an idealism that doesn't happen all of the time. I work in several areas (defense, health instruments, aviation) and it's beyond laughable to say that you don’t need roles and anyone can build anything.  For example, doing stress analysis requires a Ph.D. plus years and years of experience.  I have a Master of Science in Mathematics and seeing some of the things these people do is beyond belief (let alone understanding) to me.

Here's another related, key point, if you believe that software development works on top of certain laws (I do) then having a defined system makes sense. I will use these rules as much as I can. If you don't believe software lives on top of rules, then you probably can't do this. However, if you do believe software development lives on top of rules - then you had better learn what those rules are! I believe rules like utilization theory, risk management, JIT, Deming, etc., are rules that you should know because they will help you in all but the simplest situations. Again, if these rules are there and impact you, not talking about them leaves you in peril.

Should You Use a Small or Large Framework of Thinking?

Now, let’s talk about another different belief – the size of the mindset/toolset you bring to the party.   I like a thought model (Lean) that gives me a mindset (principles) that provide me an approach and tools to solve problems in virtually any situation. I have found that the perspective of systemic thinking coupled with the principles of fast-flexible-flow to be always useful and in most situations to be essential.  Lean also has several tools (such as value stream mapping) and practices (such as limiting work in process) that are often useful even though they typically need to be applied in different ways in different situations. If fast-flexible-flow doesn’t help me where I need it, I look for something else that does (e.g., Theory of Constraints, Real Options – just starting to get into these).  I keep building on my knowledge.  As I move forward, I end up creating a bigger and bigger knowledge base to make my teams (or my customers’ teams) more effective.  The foundation of Lean allows me to incorporate new knowledge into it. Essentially I am collecting patterns that work in different situations – but the patterns are all based on the same principles. 

There is a part of the Scrum community that prefers a simpler approach.  They prefer to have a small framework – and figure it out with “inspect and adapt”.  Scrum, in some contexts, works extremely well this way.  I like Scrum, but I don’t use it that way because I work with organizations larger than where I’ve found Scrum like this working well. Scrum provides little thought process or actions to use that are helpful beyond the team.  Using the Scrum mindset to figure things out beyond the team just seems like not being provided with the proper thinking to me.  I would rather use established principles and practices that enable me to avoid re-figuring things out that which others have already done.

There is no doubt that you can figure things out. But there are other reasons why that is not always the best tack.  In 1984 I wrote a system using automated acceptance testing on a project where the code was brittle and the requirements were continuously changing. Even with the brittle code (today I would know to work on that) we had great success. It is also a great personal embarrassment that I didn't use automated acceptance testing again for 15 years.  Why? Because I hadn't approached software as a system at the time and didn't realize automated acceptance testing was something I could use and get benefit from all the time. I had figured out that it would work there, so I did it.  But I didn’t really understand why automated acceptance testing was a great general tool so I didn’t think where else it could have been used. 

I have seen this behavior in design patterns as well.  I recollect thinking in 1997 when I first learned the bridge pattern that I could never have figured it out myself.  About a month later I realized some work I had done in 1985 (which was very impressive, if I say so myself) was built around the bridge.  But I didn’t know that’s what it was – I had just intuited a great solution.  I wonder how many other opportunities for using the bridge pattern were lost because it wasn’t in my consciousness and I wasn’t aware of the principles of design it followed.  I seriously doubt I am the only person who does this – find good solutions, don’t really understand why they were such good solutions and therefore missing opportunities to use them in other situations.  I am not satisfied with smart people intuiting great solutions from time to time. I want to bring to consciousness that which is useful.  This has the approach which led to good solutions happen more often.

Some people say this is an awful lot to learn. Well, yeah. We’re paid pretty well and should be acting like professionals.  If you do things right, it’s been shown a 10-to-1 improvement is readily achievable.  Then some proper training/education is worth the investment.
One of the reasons I helped start the Lean SSC and am excited about its possibilities is that we want to publicize the fact that certain principles exist and are key. Don Reinertsen's latest two books: Managing the Design Factory and the Principles of Product Development Flow are phenomenal books towards this. People can argue about things, but in order for us to be a true profession we need to accept which rules are out there and which ones we have to follow. A few years ago TDD was derided by many - now it is becoming to be accepted as necessary by most (I hope). There are many many things that are known by several that really should be being taken advantage by all.

What Extent Should Management Be Involved in the Process?

Many people in the Agile community seem to think all power should lie with the teams – after all, they are the ones doing the work.   I believe this may be an overreaction to overly bureaucratic management of the past.  Anarchy is not necessarily better than over-bearing managers applying micro-management and telling teams how to do things.  Both work in limited situations and fail in most others.  Respecting the team needs to be balanced with proper management and leadership.

I am not sure Scrum ever addresses this issue.  But the general attitude of many Scrum practitioners is to keep management out of the process.  The “Chickens and Pigs” metaphor is often used as a way to exclude management’s involvement.  There are many agile anarchists out there who are all too happy that management’s role has been diminished and spend little effort on improving management. 

While I don’t like bad management, I think good management is essential. Lean believes management has a key role in helping their staff continuously improve their process and to use what they believe to the best of their abilities.  Because of Lean’s insistence on a well-defined process that can be explicitly shown, management can see if the team is using their process, defining their process and what impediments they have in their process.  If the team gets caught up in their work the manager can nudge them along to pay attention to the important, albeit not at the moment, urgent things.

I am reminded of something I heard early in my career – “When you are up to your ass in alligators, it’s hard to remember you are there to drain the swamp.”  Management can remind the team their real job is to drain the swamp.

In Scrum, a lot of this role of management has moved to the Scrum Master.  That’s not bad as far as team impediments go, but it makes it difficult for higher management to see the impact of impediments that are being put on the team from the outside.  These are the people that will need to remove these impediments, but their visibility has been impaired.

Should You Start With a Team Focus or an Enterprise Focus?

The approach in the Scrum community has clearly been get a team using Scrum as a pilot.  Then get other teams using Scrum.  Spread it throughout the organization with teams learning it and teams doing Scrum-of-Scrums.  This is what I mean by a team focus.  Bottom-up.  Lean is driven by its principle of “optimize the whole.”  You can still do bottom-up if it works, but it gives you the additional possibility of bottom-up and top-down at the same time.  And, even if you are forced to do bottom-up because you don’t have management buy-in, you recognize the top-down issues and therefore are more likely to make a better case to management to get them more involved.  So, two approaches are available. While Scrum doesn’t say you can’t do bottom-up/top-down, it doesn’t help you out here. 

In fact, there are two reasons I believe a bottom up approach doesn’t work in many situations.  First of all, it can threaten mid-management. I have heard many stories from Scrum consultants about how they were brought in by some high-level executive to implement Scrum throughout the organization.  The top executive was excited, the teams were excited.  It seems everyone was excited except the mid-managers.  Why?  They had no role.  Why?  Because everything was being promoted around the team.  What happened?  Mid-management rebelled and the effort stalled or didn’t even get started.  I believe this attitude of starting with team and going up is coupled with the difference in belief of to what extent management should be involved (next different to be discussed). I find it ironic that instead of dealing with mid-management as an impediment (initially) that needs to be removed (in this case – enroll them in the transition) Scrum accommodates mid-management by somewhat ignoring them and trying to insulate the team from them.

he second reason bottom-up often doesn’t work well is that in many situations the main impediments to team productivity is not within the team.  The problem is often with how product enhancements are selected and how teams are formed.  This is a management issue.  One can say – well, remove the impediment and you’ll fix it.  But who can remove the impediment? When you are just starting, the team doesn’t exist yet and management is not getting involved.

Lean takes an attitude that both includes management and has the organizational view create the context for how the teams work.  It is not teams first and then expand that thinking to the organization.  It is how can teams work within the context of the organizational goals to maximize the value provided to the customers.

The combination of management leading teams with teams figuring out how to best do their jobs allows for a top-down/bottom-up implementation. Consider managing a group of firefighters in actions against an intensive blaze. Let’s say we have a crew of 100 fire fighters organized into 10 teams of 10. We would assign each of these teams to a different area of the forest fire. Scrum would suggest each team to do the best they can and to coordinate efforts with a Scrum of Scrums (probably with walkie-talkies in this instance). Lean would have a leader at the top giving objectives to the 10 teams and keeping them coordinated. The manager would expect that each team would organize themselves into the best way to fight the fire in their own situation – but that the objectives would be set from above (him). No micro-management here – rather managers setting vision, direction and coaching as needed. Note the difference if one of the right things to do is to let one section burn to work on others. The “Scrum” firefighters may feel like they were abandoned – their part of the forest got burned. But the “Lean” firefighters are more explicitly a larger group and will more easily identify with the bigger picture which was being managed.

Summary

Whether you affix these beliefs to Scrum or not, I have found most Scrum consultants tend to believe process should not be defined, a smaller knowledge base allows for greater creativity in solving your problems, starting with a team focus is good and management should mostly stay out of our way. I would say, except for possibly that last one re management, which at best Scrum is neutral on, Scrum suggests a different belief system than Lean one that works again an enterprise becoming Agile in most situations.  It is worthwhile to notice how these beliefs feed on each other as well. The lack of a common knowledge base makes it harder to have an explicitly defined process that both the team and managers can see.  This makes it harder for management to be involved. This makes it harder for management to remove impediments that only they can do.

I believe the perspective you come from based on your beliefs is very critical. Imagine we’re back in the 15th century and you believe the earth is flat.  You may incorporate many good sailing skills and techniques – but your overall approach will be based on this fundamental belief.  At times it might do you good, but in many situations it’ll be limiting.

It is very important for us to look at and examine the belief system from which we work.  If those beliefs are incorrect or work in only some situations, we will find ourselves at a disadvantage.  Some in the community do not like these types of comparisons.  I believe they are essential.  Our beliefs shape our actions.  If a process/framework/methodology has beliefs that only work well in certain situations, then it is worth knowing what those situations are.  If the process/framework/methodology doesn’t speak to a belief system, then people new to the process/framework/methodology run the danger of not having their belief systems improved.

Subscribe to our blog Net Objectives Thoughts Blog

Share this:

About the author | Al Shalloway

Al Shalloway is the founder and CEO of Net Objectives. With over 40 years of experience, Alan is an industry thought leader in Lean, Kanban, product portfolio management, SAFe, Scrum and agile design.



        

Blog Authors

Al Shalloway
Business, Operations, Process, Sales, Agile Design and Patterns, Personal Development, Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Cory Foy
Change Management, Innovation Games, Team Agility, Transitioning to Agile
Guy Beaver
Business and Strategy Development, Executive Management, Management, Operations, DevOps, Planning/Estimation, Change Management, Lean Implementation, Transitioning to Agile, Lean-Agile, Lean, SAFe, Kanban, Scrum
Israel Gat
Business and Strategy Development, DevOps, Lean Implementation, Agile, Lean, Kanban, Scrum
Jim Trott
Business and Strategy Development, Analysis and Design Methods, Change Management, Knowledge Management, Lean Implementation, Team Agility, Transitioning to Agile, Workflow, Technical Writing, Certifications, Coaching, Mentoring, Online Training, Professional Development, Agile, Lean-Agile, SAFe, Kanban
Ken Pugh
Agile Design and Patterns, Software Design, Design Patterns, C++, C#, Java, Technical Writing, TDD, ATDD, Certifications, Coaching, Mentoring, Professional Development, Agile, Lean-Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Marc Danziger
Business and Strategy Development, Change Management, Team Agility, Online Communities, Promotional Initiatives, Sales and Marketing Collateral
Max Guernsey
Analysis and Design Methods, Planning/Estimation, Database Agility, Design Patterns, TDD, TDD Databases, ATDD, Lean-Agile, Scrum
Scott Bain
Analysis and Design Methods, Agile Design and Patterns, Software Design, Design Patterns, Technical Writing, TDD, Coaching, Mentoring, Online Training, Professional Development, Agile
Steve Thomas
Business and Strategy Development, Change Management, Lean Implementation, Team Agility, Transitioning to Agile
Tom Grant
Business and Strategy Development, Executive Management, Management, DevOps, Analyst, Analysis and Design Methods, Planning/Estimation, Innovation Games, Lean Implementation, Agile, Lean-Agile, Lean, Kanban