Note: In reviewing this blog I do notice it sounds a little like an advertisement because I've described things from the perspective of what Net Objectives has done. However, I believe it is important to describe actual events and not theory. Most of what I write about comes from my own experience and the experience of other Net Objectives consultants. I describe what we have done to create the context for what we've learned. I am proud of our success and am merely trying to explain the tone.
For a long time, we at Net Objectives have felt that Scrum is insufficient for the enterprise. Today, we no longer consider it to be the premier method for team-based software development. This blog post explains why we feel this way.
Most everyone agrees that there is not one set of practices that works well in every context. Practices always need to be tailored for the specific context. Doing so, there are certain principles that must be attended to. If you are designing an aircraft, you may do things differently for a jet, a glider, and a propeller-driven airplane; however, every aircraft designer has to pay attention to the reality (principles) in which the craft flies: gravity, weather, and so forth. Failing to attend to these principles leads to poor results.
At Net Objectives, we are committed to discovering the principles that software developers must attend to. We combine our experience and a scientific approach to continuously extend the capability of what is possible in creating effective software development organizations. We provide these learned methods to our clients and the community in general so as to assist people in achieving their goals and in assisting them in making their organizations more successful.
Our approach is based on Lean thinking. Why? Because Lean thinking has a proven track record in many industries and our clients have created a solid track of success with it as well. Lean provides a set of principles which can be used to improve on product development (we consider all software development, even IT work, to be a type of product development). It also provides a more useful paradigm of management, helping management engage with the team as leaders and coaches and not micro-managers. Lean thinking leads to continuous process improvement.
With Lean thinking, our practices apply across the enterprise and fit the local context of teams. We support both iterative methods, using Lean-Agile software development with iterations, and pure flow-based methods, using Lean-Kanban software development with cadence.
There are some Scrum trainers who have noticed the power of Lean; however, most seem to get it backwards. They start with a Scrum mindset and then try to do Lean. This is not at all the same as starting with a Lean mindset and doing iterations (see blog Differences in Beliefs Results in Differences in Approach ). Your starting point matters.
So many struggle and need an alternative
Our commitment to helping organizations become successful means we need to start where the organization is, and that usually means starting with some form of Agile. Very many of them have started down the Scrum path – getting several people trained to be certified Scrum Masters – and they are struggling with Scrum. In fact, the number of companies not getting value from Scrum is high (75% by Ken Schwaber's own estimate ). They need an alternative.
Often, these companies get blamed for their struggles. They are told they lack resolve. They are told they need more coaching. There is even a name for what they do – "Scrum but." The truth is that Scrum has limitations (see The Five Whys as the Answer to the "But" of Scrum) but the Scrum community has had a hard time looking honestly at what these limits are and why Scrum fails to achieve success most of the time. I have inquired into the workings of Scrum with blogs Challenging Why (not if) Scrum Works and Challenging Why (not if ) Scrum Fails.
We developed these ideas more deeply in chapter five, "Going beyond Scrum", of our book Lean-Agile Software Development: Achieving Enterprise Agility. In a nutshell, Scrum overly focuses on the team and provides little help in solving the impediments the team faces other than telling them to figure it out via inspect and adapt. It also is management neutral at best and management disparaging at worst. This combination makes it difficult for Scrum to provide help to most any organization over 50 people when the problems aren't caused by the team.
At Net Objectives, we have seen many patterns as organizations try to adopt Scrum. These patterns help us work with organizations, repeatedly helping teams extend Scrum with Lean thinking. We have done this so often that we started a Scrum Clinic to be a place for people having problems with Scrum to ask questions and get answers. It led Jim Trott and I to write a Lean-Agile Pocket Guide For Scrum Teams and make it available online. In a nutshell, our consultants at Net Objectives as well as many others have found the concepts of flow, based on Lean thinking, to be essential to do effective software development.
Teams don't get enough guidance
Even in contexts where Scrum does work, I have seen teams struggle with issues like estimation, opening up too many stories, and not ensuring stories get finished within the sprint. Too often, they are left to their own intuition and skill to figure out how to solve their problems. It would be much better to give them guidance based on what others have already learned – that should be the cornerstone of any good method – but that is not the approach of classic Scrum. We find that teams get great benefit solving problems by learning Lean thinking and some better practices, such as Team Estimation.
Certainly, Scrum is evolving but very slowly. It is only recently that Scrum began to recognize the need to address technical aspects of software development and started to incorporate solid engineering practices years. It is sad because this only came after teams had experienced so much pain, when the problems of not doing test-driven development became obvious. Perhaps Scrum will eventually absorb flow, visibility and cadence into its practice. But why wait? We know these are helpful now.
Essentially, software development methods need a set of principles upon which to guide them. I have found that Lean has provided three sets of knowledge – all of which work well together to create a synergistic mindset. These are Lean thinking (flow and visibility across the value stream), lean-management (providing a vision for effective management), and knowledge stewardship (getting to continuous process improvement). Ron Jeffries recently posted a blog that seems to be contrasting Scrum and Lean; this post seems to say Lean has fixated on looking at something that may or may not apply for software development. This reflects a misunderstanding of Lean: In Lean, we do not fixate on anything; we never hold anything sacred but instead take a scientific approach to see what works.
For example, Lean thinking says we are to eliminate those delays that cause waste for the team. Moving testing up front is a natural and integral result. It builds in quality and it improves flow by removing delays. It is ironic that Ron picked up-front acceptance testing as a counter point since this has been a lean practice for some time. In fact, this is how I have enrolled countless teams in undertaking acceptance and unit test-driven development.
I agree with Scrum practitioners that we need skilled people. However, I do not agree that having skilled people is sufficient. There are many smart, committed people out there who are still struggling. What we also need is better guidance. Sadly, at this point Scrum does not provide significant guidance beyond "inspect and adapt." Some even go further to suggest that we should not be giving guidance to teams. They should figure it out for themselves.
In my mind, Scrum served its purpose to spread Agile throughout the software development community. It is time for it to let the next best thing come to center stage.