I Was Wrong About Scrum

February 17, 2013 — Posted by Al Shalloway

If a person is familiar with a certain, single subject/has with them a certain, single instrument, they may have a confirmation bias to believe that it is the answer to/involved in everything.- proverb

Many of you know I am passionate about Agile at scale.  And many of you also know I don’t think that Scrum alone will achieve it.  I’ve been doing Scrum for well over a decade now and my attitude about Scrum has changed considerably over that time.  In reflecting on what I’ve thought Scrum can and can’t do, I realized that I’ve been wrong about Scrum quite a bit over this time and thought it’d be insightful to fess up.

There is a larger point to this blog than merely pointing out how some of us have mis-understood Scrum and what it can do.  It is to illustrate that when we are in a position of needing to make decisions we go with what we know.  We act as if our own knowledge and perspective are correct, even when we know they aren’t, because we don’t have time to research other opportunities at the moment the decision is called for. This is a variant of Maslow’s – “if all you have is a hammer, everything looks like a nail” – “if all you have is a hammer, everything looks like a hammer is the right tool for the job.”

I consider myself to be reasonably open to new ideas (then again, who doesn’t?).  But I have some demonstration of this by having been on the leading edge of XP, Design Patterns, Scrum, refactoring, Lean, Kanban, SAFe, ATDD, amongst other things.  Even so, looking back I can clearly see my myopia.  Hoping exposing mine will help you expose yours.

So here are some of the beliefs I had at one time about Scrum that I no longer believe are valid.

1) Agile Means Development Iterations.

When I was first trying to explain Agile to folks, it was clear to me that agile was about building software in stages in order to get feedback on how the team was doing quickly. That the essence of Agile was iterative software development.  While iterations are one way of achieving agility, Agile is really about the delivery of business value quickly and incrementally.  Iterations are one way of doing it, but not the only way.  This mis-identification is one of the reasons executives and business folks often seem lukewarm at best about Agile – they don’t care how business value is achieved, they just want it done quickly.

2) Getting the development team started is always the way to go.

One of my more embarrassing confessions is that when I first started Net Objectives, in my seminars, I always talked about the work required from basically after requirements were stated until after code was written and tested.  This was an incredibly myopic point of view.  However, this is where I lived (as a developer) so this was my world.  Clearly, at the time anyway, the developer was the most important person, the development team was the center of the software development universe.  It was therefore normal to think that any Agile initiative had to start with the developer team.

Over time, however, I have realized that most of the problems in software development occur before the development team is involved.  These include deciding which features to build, the lack of selecting minimal business increments (often called Minimal Marketable Features, MMFs) to build and deliver, poor prioritization and a lack of attending to lean-flow.   I see now that my myopia on the developer team led me think that a team-focused development framework/process was the correct one to begin Agile endeavors with.

For a blog on where you should start, see Scrum / Kanban Series: Where to Start. You might also look at How Successful Pilots Actually Often Hurt an Organization.

3) Management are chickens

In Ken Schwaber’s infamous chicken and pig story, he relates management as not being committed to the development of the project. For years this led to the implication that teams were committed to Scrum and management wasn’t. You’d often hear “protect the teams from management” or the like.  I hate to say that I used to tell the Chicken and Pigs story myself.  For those who’ve not heard the story:

There is an old joke in which a chicken and a pig are talking and the chicken says, "Let's start a restaurant." The pig replies, "Good idea, but what should we call it?" "How about 'Ham and Eggs,'" says the chicken. "No thanks," says the pig, "I'd be committed, you'd only be involved."

When projects are small, perhaps it’s ok to just try to keep management at bay.  But as you try to be more effective and as the need to become Agile throughout your organization increases, it is important to include management.  I think it is time to eliminate the Chicken and Pigs story from our Scrum methods.  While I hear many people tell me that this has already happened, I see it on most every major Scrum website today.

See Is “Chicken and Pigs” Counter-Productive? for more.

4) Use Scrum of Scrums to Scale

My former confidence in Scrum of Scrums was quite understandable given my myopic view of the importance of the development team and the belief that we had to run Scrum with self-organizing, self-managing teams. Scaling Scrum meant combining effective teams with each other in a self-organizing manner.  Teams-of-teams were just an extension of Scrum.  Unfortunately, we could never get this to work in organizations that weren’t already very effective across the organization (and since these types of companies didn’t need consultants, we never could get it to work).  After 2-3 years of failing with  Scrum-of-Scrums we started coming up with a combination of Lean leadership along with Scrum implementation.  Our early attempts with this included what we called the product coordination team and have since evolved to managing flow across teams by attending to lean-product development flow and shared backlogs.

See a recording of The Three Ways to Scale Agile and One That Doesn’t Work So Well on the Lean-Agile at Scale and Team Value Stream Series for more.

5) It is ok to start with a focus on process and ignore the technical issues

We started doing eXtreme Programming around 2000.  I had done virtually all of the technical practices at some point before XP was first described so they made total sense to me.  I had never put it together so I am not claiming I had been doing XP before that. But once described, it was very clear to me that XP was awesome.  Unfortunately, I couldn’t seem to get many others to buy into it.  TDD, paired programming, continuous integration appeared to be a lot of work and folks were already over-worked. 

I saw how the Scrum community was pretty much ignoring technical practices at the time and I was very excited about Scrum so I made the decision to go along with the crowd.  The idea was – get folks to do iterative development and they’ll be better off.  We can then get them to attend to better technical practices. 

I do believe we did this to a large extent, but in retrospect, I believe a lot of people never appreciated the need for technical practices because we focused so much on the team methods again – without attending to the technical issues. 

Since Scrum talked about process issues (release planning, Sprint planning, standups, burndowns, burnups, …) it kind of implied that technical issues weren’t so important.  I hate to admit but I was more concerned with them getting started and often did not focus enough on the full picture.   I, of course, wasn’t alone here.  I am glad the industry has mostly correctly itself here.

See Why Technical Debt Matters for more.

6) A big shift in roles and structure is often necessary and people must just adjust to it

When I first started doing Agile I knew only two methods: XP and Scrum.  Both required cross-functional teams and both used time-boxed iterations.  I mistakenly inferred this meant Agile required cross-functional teams and time-boxed iterations.  Therefore, if you didn’t have cross-functional teams, you needed to get them.  This is just what you had to do.  Ignore the psychological impact that you were now asking folks to abandon their familiar roles and jump into unfamiliar (uncomfortable) territory.  Being a professional meant that you did that.

Since that time, I’ve come to appreciate that people don’t really like to make major changes.  That having the opportunity of doing a smooth transition may be very useful in many different places.  That just because folks don’t want to make major jumps does not meanthat they are not motivated or disciplined.  This again relates to my early statement that if all you have is one way (in this case, of transitioning) then it looks like that’s always the way to do this. However, the evidence is otherwise.

Two books in particular, have demonstrated this to me - Switch: How to Change Things When Change is Hard and Managing Transitions: Making the Most of Change. Besides the fact that people don’t necessarily like to change too much, it’s been demonstrated it is not usually the best way to learn. One Small Step Can Change Your Life: The Kaizen Way and Toyota Kata: Managing People for Improvement, Adaptiveness and Superior Results demonstrate that people learn better in small steps.

The truth lies in between here, of course.  While always insisting on change is not a good course, always insisting on small changes has different limitations.  I find the trick is to change as much as the team wants to and understands (see How Understanding Helps Transitions).  One needs to look at how much the organization wants to change, how much they need to change and how much change they can withstand.  Then decide how to manage the transition.

I’ve written a blog about big Vs small change called Big Change Up Front (BCUF).

7) Shielding the team from management makes sense

Because of my myopic focus on the team and a lack of focus on business value, I really didn’t know how to relate to management.  Oddly enough, sometimes they were more in favor of Agile adoption than the team, but when they weren’t, they seemed to be the problem we needed to overcome.  Keeping them away from the team seemed like a good idea because “they just didn’t get Agile.”  Dilberts became more prevalent and seemed to explain their lack of getting it.  It wasn’t our fault as developers that they were being pointy haired – best to just keep hem away.

Of course, what was really going on is that I didn’t, at the time, understand why Scrum worked.   I wasn’t adept at Lean product development flow, I hadn’t read Reinertsen’s amazing book Managing the Design Factory or his more recent, even more amazing, The Principles of Product Development Flow: Second Generation Lean Product Development.   Because of my lack of understanding Lean flow, I couldn’t relate to management in the way David Anderson relates he did in Kanban: Successful Evolutionary Change for Your Technology Business.  Management doesn’t understand when you explain management in terms of development – but that’s not their fault.

8) Even if you have specialized skill sets you can figure out how to have Scrum work if you try hard enough

I can still remember after doing a very successful Agile transition based on Scrum and TDD with a group of about 300 developers that another group in the same company contacted us to help.  Unfortunately, this group, while smaller, had a few specialized folks.  In this case, it was someone who knew parts of the code better than anyone else who also was their best subject matter expert and architect.  Getting others to fill his shoes was not going to happen quickly (if at all).  A method where he could be shared across teams was needed. At the time I did not know Kanban (actually, it hadn’t been invented yet).  Given I knew Agile was good and they weren’t doing it I just put it on them as not understanding the benefits enough. 

9) Finding your impediments is the right way to learn

Agile will definitely bring up anything wrong with your process.  If you don’t understand the theory of lean-flow which underlies software development (or as is still the case for certain Scrum thought leaders, deny it is  good to know) then uncovering impediments and solving them is one way to go.  Unfortunately, I found that doing this often 1) didn’t really uncover the impediments because they were just considered the way it was, 2) even when uncovered, no one knew how to solve them. 

While it sounds like a great argument to put yourself into a structure that we know works and then eliminate anything that’s in your way, it is not always the best way to achieve your goals.  True, doing “Scrum-but” to avoid solving the impediments is not good, but understanding the cause of your impediments and making changes to your process based on eliminating the problems being manifested by the impediments is often a more effective manner.

A few years ago I postulated that “all problems with one’s process will show up as delay in the workflow.”  I have seen nothing to date that contradicts this.   Understanding this provides other ways to solve problems than require a particular structure and then work through it.   Often is is faster and more economical, not to mention easier on the people involed.

10) If people are motivated enough they will continue to have good retrospectives

I’m not sure I ever said this, but I remember somewhat thinking it.  Anytime someone wouldn’t do Scrum or would do it poorly, I always had the sense that they just weren’t committed enough to get a better understanding in order to do it right.  Now I realize that this was not only incredibly arrogant of me, it was a classic case of me ascribing motivations to others’ behavior (the fundamental attribution error).

When looking back over this it is a bit alarming my level of arrogance and limited understanding that there were other approaches.  Perhaps I am much worse than others.  However, I think this is just part of our human condition – we believe, at the moment of making decisions, that what we perceive is reality.

Where I Am Today

I will say that I have had two fundamental shifts since those early Scrum days. The first is my attitude now that whenever someone doesn’t understand something, that it is more due to me than them.  See What to Say When Someone Just Doesn’t Get It. Just as important, I have found that the Lean mindset, in conjunction with Kanban, and now a considerable amount of experience with large and small organizations, that we have enough knowledge to pretty much help anyone – albeit in ways particular to their own circumstances.  This combination of more complete knowledge, a set of approaches and an attitude that people are motivated if you give them a good solution, has taken me away from the myopia I can see I had exhibited.

I am excited to say that I think the industry pretty much understands what is needed for effective, efficient software development.  While technical practices have been ahead of the process issues, and while there is still a lot more to learn, an understanding of what it takes to do software development is now pretty well known.  Of course, the real problem is getting folks to do it - see People Are Complex, Software Development Isn’t.

Here are additional resources related to this topic:

Our Next Webinar: Essential Skills fo rthe Agile Developer

Hopefully this will have been some help to you. 

As always, if you’d like to chat, give me a call, always happy to help.

Al Shalloway
CEO, Net Objectives
Certified SAFe Program Consultant and LKU Kanban Instructor

 

Author: 

Share this:

About the author | Al Shalloway

Al Shalloway is the founder and CEO of Net Objectives. With over 40 years of experience, Alan is an industry thought leader in Lean, Kanban, product portfolio management, SAFe, Scrum and agile design.


Comments

Overall I thought this was a pretty good post.

Scrum, as it is actually practiced in the wild, has changed significantly over the years (especially the last 5-8 years). What used to be called Scrum-but in the early days is now almost the norm and known as adapting-the-process-to-your-context. Although I really like your new definition of Scrum-but (at least its new to me) being changes to the process for the purpose of avoiding resolving an impediment.

I'd like to give a shout-out to a recently published, excellent book on Modern Scrum (as I like to call it) -- Scrum as it is really practiced in the field: Essential Scrum, by Kenneth Rubin -- it is an excellent book.

In fact, my short list of "the first thing to read" if you want to get started with Agile is now just two books: Kenny's book mentioned above, and your book, "Essential Skills for the Agile Developer".

On item #6 in your post...

I still feel like this topic area is incomplete and needs more work. You've been taking the position that non-disruptive, incremental change will get you where you want to go. Whereas I've taken the position that your approach only fully works when you have experienced experts providing guidance, and that its often better to take the radical break all-at-once approach to force the "ah ha" moment that comes from finally "getting it" through the experience of actually "doing it".

Anyway, I suspect we're both right and we're both wrong and the the "ultimate truth" is not necessarily in the middle, but off to the side somewhere we haven't yet looked.

I think I'm starting to sound much too "new age" to sit right with my practical nature, so suffice it to say that I just thing this topic needs a lot more thought.

Curt:I am 100% in agreement with you.  There are times you want to jump, times you want to crawl, times you want to make a small jump.   I was focusing in this blog on my misconceptions - that you always want to jump. I added a paragraph to the end of #6 to help clarify this. Please let me know if you feel it sufficient.

Free Email Updates!

Sign up for free email updates from
Net Objectives Thoughts Blog

Blog Authors

Al Shalloway
Business, Operations, Process, Sales, Agile Design and Patterns, Personal Development, Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Cory Foy
Change Management, Innovation Games, Team Agility, Transitioning to Agile
Jim Trott
Business and Strategy Development, Analysis and Design Methods, Change Management, Knowledge Management, Lean Implementation, Team Agility, Transitioning to Agile, Workflow, Technical Writing, Certifications, Coaching, Mentoring, Online Training, Professional Development, Agile, Lean-Agile, SAFe, Kanban
Ken Pugh
Software Design, Design Patterns, Technical Writing, TDD, ATDD, Coaching, Mentoring, Professional Development, Agile, Lean-Agile, SAFe, Scrum
Scott Bain
Analysis and Design Methods, Agile Design and Patterns, Software Design, Design Patterns, Technical Writing, TDD, Coaching, Mentoring, Online Training, Professional Development, Agile