Why Scrum Is Going the Way of XP and Why that’s a Good Thing

February 8, 2009 — Posted by Al Shalloway

Being in the software industry for 38+ years doesn’t make you any smarter but it does mean you’ve seen a lot.  In the early 2000s, eXtreme Programming was the cool new Agile methodology. If you asked someone what they thought about Agile, they might comment about not liking paired-programming or they might think up-front testing was not worth the effort, both of which are good practices that are still not appreciated enough. Equating Agile with XP, their impressions of XP practices flavored their impression of Agile. That is unfortunate.

While XP is a kind of Agile, it is not the same as Agile. There are other Agile methods.

Nowadays, people are more apt to equate Agile with Scrum. Organizations that are contemplating Agile often send someone to a CSM (Certified Scrum Master) course to learn about it. Now, that is soft of like sending someone to a course to learn how to be a football coach in order to learn football; it is not necessarily the best approach. Even worse, it limits what you can learn about Agility.

Just like the case with XP, Scrum is a kind of Agile, but it is not the same as Agile.

Rather, "Scrum is a management, enhancement and maintenance methodology for an existing system."[1]

The Problem of Naive Definitions

For a few years after XP had emerged, there was a great expansion of interest in and adoption of it. It seemed like a new book was coming out about XP every other day (not really, but maybe every other week). Practitioners would tell you that you had to be doing all of the 12 practices of XP or you weren’t doing XP. After a while, XP seemed to morph from following some principles and practices to “XP is what you do after you’ve been doing the XP practices for a while."[2] Maybe that is a pretty good, Zen-like answer, but not really helpful (except, maybe, to consultants).

Now, I make my living as CEO of a company that helps other people develop software faster, with higher quality and lower cost.  Most of the organizations we work with either don’t understand Agile methods well or they are having troubles picking them up. It does not help them to define an Agile method in terms that only a well-informed practitioner could understand. It implies that you must hire a trained coach or consultant to be able to do anything. And that is not something that they can always afford.

Well, many people will go ahead and adopt the practices, even if they don't truly understand. And they will have some good success: those XP practices are so good that if people truly follow them, they will do pretty well. And they will also get bit; for example, the mandate of writing simple code fails when one misunderstands what is meant by "simple." 

Understand Both Principles and Practices

At Net Objectives, we have found that people need to understand both the practices they need to start with as well as the principles upon which those practices are based. That way, they have a better understanding of what the practices mean and, more importantly, they can more easily modify the practices for their situation. 

I believe that the success of XP was due more to quality of the people who were creating and promoting it than to practitioners clearly understanding XP principles. At the beginning, there was little understanding about why XP worked. Consider the conversations about XP at the time about how design is not needed and how, if you didn’t follow all twelve practices, you wouldn’t get much of the value, how if you only did 10 out of the 12 practices, you would get much, much less than 84% of the value of XP. Later on, XP leaders recanted both of these claims as they and others used XP in more situations and gained a deeper knowledge about why it worked.

Today, most people consider XP to be a set of excellent engineering practices which has been adopted by many Scrum practitioners. This doesn’t do it proper justice. I know many solid XPers that use XP as a full methodology for developing software –not merely a set of engineering practices. The only problem is, it is still hard to explain what the Zen-like definition of XP actually means.

Going Down the Same Naive Path

Scrum, as it was originally presented, is exceptionally simple. Its promotion through certification courses and the Scrum Alliance has made it a very popular methodology. (Now, some see "Scrum Certification" as meaning "I paid for and attended a two-day course"... which feels more like great marketing than solid certification of knowledge and skill). As Scrum’s popularity has grown, its followers have wanted to expand its scope, yet without good definitions.

For example, I believe Jeff Sutherland developed a good approach with his "Type B" and "Type C" Scrum. He had good, clear definitions, using different methods for different reasons: "Type A" is the original Scrum; "Type C" is a way to use Scrum for product portfolio management. Unfortunately, Ken Schwaber mashed this up when he contended, “It is all Scrum.” He means Scrum is what you do no matter where you are in the software process. This sounds a bit like that Zen-like definition of XP.

I have asked many people to define Scrum and have received many different answers.

  • It’s what the creator of Scrum says (i.e., Jeff Sutherland)
  • It’s what the acknowledged leader of the Scrum community says it is (i.e., Ken Schwaber)
  • It’s what it was originally defined to be (i.e., back in 1995)
  • It’s what the Scrum Alliance says it is or it’s what the best practices of Scrum says it should be

More recently, I heard a Certified Scrum Trainer say he couldn’t say what Scrum was, but could only say what he teaches (so at least I am not alone in the confusion).

In my mind, this confusion results in Scrum being little more than:

  • Defining proper software development as an empirical process
  • Using time-boxing as a way to develop software incrementally, and
  • Having self-organizing teams inspect and adapt to make progress.

Now, I am positive that these are all good things! However, it rather limits Scrum from taking advantage of the many things that have been discovered by others in the industry (or outside the industry). For example, for several years, Scrum practitioners have avoided the entire conversation about engineering practices. This led to numerous projects that, although they had initial success, later failed due to poor code quality. Since then, most Scrum practitioners promote XP engineering practices.  While currently rejecting Lean thinking as a necessary part of Scrum thinking, I predict that in a few years, Scrum leaders will be saying they do Scrum with XP Engineering practices for code quality and Lean thinking to help them guide their practices – but that’s another story – and will take time to see what actually happens.

I see Scrum following the same path XP did, expanding its scope without understanding its natural limits. That leads to failures. Scrum’s leaders acknowledge that Scrum only succeeds one third of the time. They typically blamed this on early Scrum practitioners who don’t properly follow Scrum.  But they ignored the root causes, including:

  • Naively certifying people and equipping them poorly to do the job
  • A lack of focus on the principles underlying Scrum 

Agile Is Not (Only) Scrum and XP

The good news is that Scrum has brought awareness to the software industry that Agile methods are good, that we must no longer pretend that defined processes will work in such complicated situations. It has done a great job in its natural setting, assisting teams in building/enhancing software development in a short period of time that best meets the needs of the customer.

I believe that we are entering a new phase in Agile software development which some people call Lean-Agile. That is, Agile software practices that are done within the context of Lean. It supplements the work of teams (e.g. using Scrum). And it expands naturally beyond the team to help the enterprise.

Lean provides a framework for thinking, while being able to absorb other methodologies. It does this in a way that still enables it to keep a clear sense of what its principles are. This is because Lean is based on a solid foundation, including:

  • Respect people
  • Most errors are of a systemic basis (meaning you must fix the system to stop the errors from recurring)
  • Use “Plan–Do–Check–Act” (similar to Scrum’s inspect and adapt)
  • There is a strong relationship between delays in the work flow and waste
  • Focus on adding value to your customers
  • Focus on reducing cycle time (the time from start to finish) instead of maximizing productivity of your resources

Lean has incorporates many disciplines that are consistent with these concepts. As quick examples,

  • Utilization theory, invented almost a hundred years ago by AT&T, provides a great understanding about why limiting work in process (WIP) is crucial.
  • The psychology of productivity, which acknowledges that people doing too many tasks will thrash and have their productivity severely lowered.

As Lean incorporates principles from these other disciplines, it becomes stronger and better defined. It leads to powerful approaches and understandings and this is one reason I have so much passion on Lean.

I believe in the next couple of years, we will see Lean-Agile taking the lead in creating a context for Agile software development.  Lean thinking can be used to guide the transition to more effective software development. Scrum practices and Kanban are two good approaches for helping the team. XP engineering practices continue to evolve and provide a great basis for writing quality code.

This layering of Lean, Scrum and/or Kanban, and XP provides a way for clear thinking about different levels and areas of software development. It will allow for a better clarification of what is needed as well as methods to improve each area.  This is a good thing.  It should also lead to a better understanding of what good practices and principles for software development are.  This should eventually lead to raising the bar in both what it means to write quality code as well as what it means to properly do software development.  Both of these are needed for the software industry to become a true profession.

While Scrum may no longer be the leading methodology in the future Agile world, in its proper context, it will be able to add even more value. For more about this, I'd invite you to read An Agile Developer’s Guide to Lean-Thinking and Core Developer Skills. Note, this content is part of our portal premium content.

[1] SCRUM Software Development Process, K. Schwaber,  OOPSLA Business Object Design and Implementation Workshop.


[2] I’ve heard this from numerous XP aficionados, but first from Ron Jeffries.

Subscribe to our blog Net Objectives Thoughts Blog

Share this:

About the author | Al Shalloway

Al Shalloway is the founder and CEO of Net Objectives. With 45 years of experience, Al is an industry thought leader in Lean, Kanban, product portfolio management, Scrum and agile design. He helps companies transition to Lean and Agile methods enterprise-wide as well teaches courses in these areas.


Blog Authors

Al Shalloway
Business, Operations, Process, Sales, Agile Design and Patterns, Personal Development, Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Cory Foy
Change Management, Innovation Games, Team Agility, Transitioning to Agile
Guy Beaver
Business and Strategy Development, Executive Management, Management, Operations, DevOps, Planning/Estimation, Change Management, Lean Implementation, Transitioning to Agile, Lean-Agile, Lean, SAFe, Kanban, Scrum
Israel Gat
Business and Strategy Development, DevOps, Lean Implementation, Agile, Lean, Kanban, Scrum
Jim Trott
Business and Strategy Development, Analysis and Design Methods, Change Management, Knowledge Management, Lean Implementation, Team Agility, Transitioning to Agile, Workflow, Technical Writing, Certifications, Coaching, Mentoring, Online Training, Professional Development, Agile, Lean-Agile, SAFe, Kanban
Ken Pugh
Agile Design and Patterns, Software Design, Design Patterns, C++, C#, Java, Technical Writing, TDD, ATDD, Certifications, Coaching, Mentoring, Professional Development, Agile, Lean-Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Marc Danziger
Business and Strategy Development, Change Management, Team Agility, Online Communities, Promotional Initiatives, Sales and Marketing Collateral
Max Guernsey
Analysis and Design Methods, Planning/Estimation, Database Agility, Design Patterns, TDD, TDD Databases, ATDD, Lean-Agile, Scrum
Scott Bain
Analysis and Design Methods, Agile Design and Patterns, Software Design, Design Patterns, Technical Writing, TDD, Coaching, Mentoring, Online Training, Professional Development, Agile
Steve Thomas
Business and Strategy Development, Change Management, Lean Implementation, Team Agility, Transitioning to Agile
Tom Grant
Business and Strategy Development, Executive Management, Management, DevOps, Analyst, Analysis and Design Methods, Planning/Estimation, Innovation Games, Lean Implementation, Agile, Lean-Agile, Lean, Kanban