My List of Limitations of Scrum

April 7, 2010 — Posted by Al Shalloway

I recently tweeted that I was sometimes irritated that when I've stated something about Scrum that I consider a shortcoming, I usually get called a "Scrum Basher". I would much prefer people engage me on what I have designated as a short-coming of Scrum. If, in fact, my assessment is right, I would be helping those who would be running into a problem. If, in fact, my assessment is wrong, it would be better to engage in a conversation with me to let both myself and others, that my opinion was wrong.

Unfortunately, instead of engaging in a useful conversation, I have typically just been personally characterized in a negative fashion – having been charged as having poor ethics, being market not value driven or worse (some have called for my leaving the industry). I believe my record stands on its own as own although I suspect few people know or remember that, at one time, Net Objectives did as much or more Scrum training as any company out there. I voluntarily stopped offering Scrum training through the Scrum Alliance and even turned down an offer to have one of my staff become a CST a few years ago. This personally cost me a few hundred thousand dollars –but I have never regretted the decision made because I felt a free hand in providing the best Scrum training possible was necessary. We still offer Scrum training, but now it is guided by Lean thinking. We also provide Kanban and Lean-Agile training when that is more appropriate, as our experience has shown is usually the case. I take pride in that many things I have been saying for years now seem to be being said by other thought leaders. I at least take a little consolation in having opened the door a little.

It is important to also realize that my criticisms aren't mostly for Scrum, but for the way it is being over-marketed and suggested to be used in places where I don't think it works well. From 2001 to 2004 I pretty much just went along with Scrum as it was being espoused. Most of our clients were relatively small and Scrum worked reasonably well. Starting in 2005, I had a series of conversations with Ken Schwaber and Mike Cohn about how Scrum and the Scrum Alliance model needed to evolve – both in terms of what Scrum was and in the certification program itself. Needless to say, I was pretty much ignored. For a while I just kept quiet. But as more and more companies adopted Scrum and had difficulties with it, and as more and more companies merely assumed that Scrum was always the place to start, I found it hard to keep quiet – both for the sake of those already having problems and those who were about to. Net Objectives' vision is "effective software development without suffering" so keeping quiet in the face of this suffering, when I know of alternatives, would be a major out of integrity for me.

In response to my tweet, I was asked me what points these were that I was referring to. A good question, so I thought I'd list them. Note, these criticisms are not merely whines. Along with each one I've listed a proposed solution or help in resolving it. Note, for some of this information you'll need to register at our site, but most of it is labeled as guest access. I've organized these by topic. Yes, there are quite a few (last count, 16). That doesn't mean I don't like Scrum. It means I think Scrum applies in certain places (not everywhere) and that Scrum needs to be extended in many other places.

I'm more than happy to discuss this. Please do so at our Net Objectives Support Group (LinkedIn) or Business Driven Software Development group (enrollment restrictions apply). By the way, many of these solutions are in our Lean-Agile Pocket Guide for Scrum Teams (including an online version) and our Scrum-Clinic. Our new book Lean-Agile Software Development: Achieving Enterprise Agility also discusses many of the knowledge/skills we feel are needed for successful lean-agile at the enterprise level.

List of issues: (details follow):

Scrum beyond the team

  • Scrum-of-Scrums is insufficient except for highly functional organizations.
  • To expand agile throughout the enterprise, middle management is essential and that calling them chickens does not help this.

Starting an Agile Transition

  • Starting a pilot project is often not the right way to start an agile transitions.
  • Scrum is not always the best team process.
  • If you start your transition with scrum and the chief problem is not at the team level, then doing so may actually hurt the organization.

What You must Include in the Scrum Framework

  • In complex cases, you must look at the theory of flow in software development to make decisions on how teams should work together.
  • Understanding the theory of flow will greatly help a development team manage their work in progress.
  • Using Acceptance Test-Driven Development (ATDD) is essential for virtually all teams.
  • Explicitly defined workflows help the team learn.

Why Scrum Works or Fails

  • Scrum does not work primarily because teams are self-organizing.
  • Merely exposing impediments does not mean the organization will know how to solve them.
  • Sometimes teams are impeded by something but they aren't aware of the impediment.
  • Scrum is based on Lean-Thinking
  • The low success rate of Scrum adoption is not due to lack of motivation or discipline but the lacking of key knowledge.

Certification and CSM Training

  • For certification to have any value, you must state what competencies you are certifying.
  • There is a great variety of CSM training. Ranging from Team training to Scrum Master Training

 

Scrum Beyond the Team

Scrum-of-Scrums is insufficient except for highly functional organizations.

This was the issue that got me thrown off the Scrum Development group the second, and last, (since I don't have any intention of rejoining) time. I suggested that a higher perspective guided by Lean-Thinking worked better. We devote a chapter to this in our Lean-Agile Software Development: Achieving Enterprise Agility book.

To expand agile throughout the enterprise, middle management is essential and that calling them chickens does not help this.

Managers are not here just to get the teams coffee or to stay out of their way. It's been nice to see that Scrum is starting to acknowledge management as needing to fill the role of removing impediments for the teams and not merely staying out of their way. But there is more to management than this. The software development value stream includes more than development teams. This needs to be both managed by development and management may impose certain requirements on teams to ensure they are working on the most important things possible. See Guy Beaver's blog Mid-Management in the Lean-Agile Enterprise: Competencies Required and my blog Is 'Chickens and Pigs' Counter-Productive? (and more in our lean-agile book, of course).

Starting an Agile Transition

Starting a pilot project is often not the right way to start an agile transitions.

Here's an update from an article to be published by the Agile Journal (I'll provide a link when it comes out):

The common mantra in the Agile world today is to start a pilot project – focused on creating team agility. The intention is to scale this process throughout the entire organization after one or two teams have learned it. In many organizations, however, the real problem isn't so much the teams' process as much as it is too much work being handed to the teams which keeps them working in a haphazard and inefficient fashion. Oddly enough, this team-pilot approach often has great initial success but is not able to scale to the organization. This occurs because the team selected for the pilot projects is told to work on one project. They achieve much greater productivity mostly because they have created the team to be focused on one-project, are often co-located, and have all of the skills they need to do this on one team. We've seen these factors have just as much impact on a team's efficiency than agile methods.

Of course, they may not truly understand this and likely will attribute it to their new process. As they add more agile teams, the rest of the organization actually gets worse and worse. Eventually, the local improvements of the agile teams is more than balanced out by the increasing burden placed on the other teams. Some point to this as evidence that agility works since the agile teams are doing well – even if the organization is not improving. Ironically, the more the teams as a whole are being overloaded (that is, the more the need for proper portfolio management) the better the pilots will appear compared to the other teams. This is one of the key reasons agile pilots work but then agility at the enterprise level fails.

See Where to Being Your Transition to Lean-Agile – an earlier article of mine.

Scrum is not always the best team process.

Should this be a surprise to anyone? We hear there is no one size fits all. So why would we expect one method to be best everywhere? Today, there are several team agile methods you can choose from. While Scrum is a popular choice, it is often not the correct one. Scrum is powerful at the team level, but many organizations don't have well-defined teams in that some people, with essential knowledge, are required to be split across several teams. Other challenges may make creating the proper framework within which to do Scrum difficult. Scrum forces an organizational structure that creates tension to remove impediments but it doesn't tell you how to do this. Kanban is often a better choice because it can work within your existing organizational structure. It creates tension across business-management-team by managing work in progress levels and providing Lean principles on which to guide improvement. Kanban keeps you always in a working environment and creates tension for improvement by creating visibility on if you are violating key development principles. A transition to Scrum is therefore often abrupt and chaotic in nature while Kanban starts where you are and is evolutionary in nature. Organizations should properly investigate which approach is more appropriate for them.

Get an overview of the options with my webinar on The Different Agile Approaches I did for the PMI.

If you start your transition with scrum and the chief problem is not at the team level, then doing so may actually hurt the organization.

Actually, I think I've made my point here in the prior two points.

What You must Include in the Scrum Framework

Having worked with dozens of Scrum teams personally, and having overseen the scores more teams my company has worked with, I have seen several anti-patterns in Scrum adoption. This is not a failure of Scrum, per se, but rather what Scrum proponents are including in their training. Many of this required (in my mind) information is described in our Scrum clinic.

In complex cases, you must look at the theory of flow in software development to make decisions on how teams should work together.

Interactions between teams is fairly complex. Expecting people to solve problems without a tangible thought framework to work with together is naïve and fraught with danger. The best place to learn about flow is Don Reinertsen's Managing the Design Factory. If you buy it, read it and don't like it, I'll send you a free copy of my lean-agile book to make up for this. Offer good for 3 months. J

Understanding the theory of flow will greatly help a development team manage their work in progress.

This is the basis for Kanban. If you want to continue to do iterations, then you will likely find Scrumban a better fit. I can think of few situations where managing your work in progress is not a good idea.

Using Acceptance Test-Driven Development (ATDD) is essential for virtually all teams.

Until recently, Scrum has pretty much ignored technical practices. ATDD, and the similar Behavioral Driven Development, are more process oriented methods than technical practices. Using one or the other is essential for teams. They both provide immediate improvements and enable sustainability. When transitioning to agile methods it is critical to find some immediate return that also improves quality and lowers (or at least stops increasing) technical debt. Our book, Lean-Agile Software Development: Achieving Enterprise Agility, has a chapter on this. If you prefer, here's a webinar on the subject. If you want more technically oriented advice see our Roadmap to Lean-Agile Programming Competencies or our upcoming book Essential Skills for the Agile Developer: A Guide to Better Programming and Design.

Explicitly defined workflows help the team learn.

I did not discover this – it was demonstrated over and over again at the Lean/Kanban conference in Miami last year. Take a look.

Why Scrum Works or Fails

Scrum does not work primarily because teams are self-organizing.

I've done a webinar on this – Understanding Why Scrum Works. I've also written a blog on this Challenging why (not if) Scrum works.

Merely exposing impediments does not mean the organization will know how to solve them.

See my blog Challenging Why (not it) Scrum Fails or Going Beyond Scrum, Part 1 and Going Beyond Scrum, Part 2 from Lean-Agile Software Development: Achieving Enterprise Agility

Sometimes teams are impeded by something but they aren't aware of the impediment.

See my blog Challenging Why (not it) Scrum Fails or Going Beyond Scrum, Part 1 and Going Beyond Scrum, Part 2 from Lean-Agile Software Development: Achieving Enterprise Agility

Scrum is based on Lean-Thinking

All practices (including Scrum) are not always applicable in all situations. If you state a practice you must include the principles on which it is based. Understanding Why Scrum Works discusses how Lean can be viewed as a basis for Scrum and therefore help seeing how to apply Scrum practices in your situation. In The 5 whys of Lean as an Answer to the But of Scrum I discuss how lean-thinking can help Scrum implementations.

The low success rate of Scrum adoption is not due to lack of motivation or discipline but the lacking of key knowledge.

Here's another excerpt from the Agile Journal article I mentioned earlier.

Ken Schwaber, co-creator of Scrum, has said "I estimate that 75% of those organizations using Scrum will not succeed in getting the benefits that they hope for from it." While this number has been generally agreed to among software consultants, the reason why is not. I was initially going to say "hotly debated" but in fact, most agilists haven't investigated the cause for this other than the immediate assumption that the failure occurs due to lack of motivation or discipline within the organization.

Our experience is that team-focused agile methods don't provide the insights required to remove the organizational impediments that they face. A more holistic, value-stream oriented view geared toward shortening cycle-time provides these insights. Scaling often fails because the root cause of the impediments in the value stream are not observed, let alone dealt with. Agility can be achieved across an organization when the entire value stream is considered – not merely for a lucky team or two selected for the pilot project.

Certification and CSM Training

I am absolutely not against certification. In fact, I am working on a meaningful certification program with the Lean SSC. What I don't like is certification in a course that is called certification of a person.

For certification to have any value, you must state what competencies you are certifying.

I have always felt that underneath a certification program should be the competencies that are being certified. In the case of Scrum this would be what we call "team agility" or those steps related to team process. Because I have never seen this identified, we wrote a book called The Lean-Agile Pocket Guide for Scrum Teams (including an online copy).We've put this online as well). We've also created the Scrum Cinic to address issues teams have faced with Scrum.

But building software, as your blog addresses, must also include technical skills (we refer to this as (technical agility). We've done some work here as well – creating a Road-map to Lean-Agile Programming Competencies. This not only lays out what you should know but provides webinars, articles and more to get you there.

There is a great variety of CSM training. Ranging from Team training to Scrum Master Training

There is a huge variety in CSM courses being offered now. Some are about learning what Scrum is. Some are about learning what a CSM should do. These are different courses, unfortunately being offered under the same moniker. I've created a Questions to Ask in Selecting a Trainer for Your CSM Training.

Conclusion    

Obviously I have a lot of passion here. And just as obviously, I've been talking about this for years. Unfortunately, my character has been discussed more than these issues have. I see too many companies starting their agile transitions assuming a pilot project with Scrum is the way to go – without being aware of alternatives or even that it could be detrimental for them to do so. One shouldn't expect those making money on selling Scrum to offer alternatives, especially if that is all they sell. This is why I do it. I really don't mind when a company goes with Scrum training when Scrum is their best alternative. But when they don't know what their alternatives are, it's like buying the most popular selling car (or anything) without looking to see if it's the right one for you.

Subscribe to our blog Net Objectives Thoughts Blog

Share this:

About the author | Al Shalloway

Al Shalloway is the founder and CEO of Net Objectives. With 45 years of experience, Al is an industry thought leader in Lean, Kanban, product portfolio management, Scrum and agile design. He helps companies transition to Lean and Agile methods enterprise-wide as well teaches courses in these areas.



        

Blog Authors

Al Shalloway
Business, Operations, Process, Sales, Agile Design and Patterns, Personal Development, Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Cory Foy
Change Management, Innovation Games, Team Agility, Transitioning to Agile
Guy Beaver
Business and Strategy Development, Executive Management, Management, Operations, DevOps, Planning/Estimation, Change Management, Lean Implementation, Transitioning to Agile, Lean-Agile, Lean, SAFe, Kanban, Scrum
Israel Gat
Business and Strategy Development, DevOps, Lean Implementation, Agile, Lean, Kanban, Scrum
Jim Trott
Business and Strategy Development, Analysis and Design Methods, Change Management, Knowledge Management, Lean Implementation, Team Agility, Transitioning to Agile, Workflow, Technical Writing, Certifications, Coaching, Mentoring, Online Training, Professional Development, Agile, Lean-Agile, SAFe, Kanban
Ken Pugh
Agile Design and Patterns, Software Design, Design Patterns, C++, C#, Java, Technical Writing, TDD, ATDD, Certifications, Coaching, Mentoring, Professional Development, Agile, Lean-Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Marc Danziger
Business and Strategy Development, Change Management, Team Agility, Online Communities, Promotional Initiatives, Sales and Marketing Collateral
Max Guernsey
Analysis and Design Methods, Planning/Estimation, Database Agility, Design Patterns, TDD, TDD Databases, ATDD, Lean-Agile, Scrum
Scott Bain
Analysis and Design Methods, Agile Design and Patterns, Software Design, Design Patterns, Technical Writing, TDD, Coaching, Mentoring, Online Training, Professional Development, Agile
Steve Thomas
Business and Strategy Development, Change Management, Lean Implementation, Team Agility, Transitioning to Agile
Tom Grant
Business and Strategy Development, Executive Management, Management, DevOps, Analyst, Analysis and Design Methods, Planning/Estimation, Innovation Games, Lean Implementation, Agile, Lean-Agile, Lean, Kanban