When, Why and How To Estimate

July 23, 2015 — Posted by Al Shalloway

There are three disparate views on estimation.  One is that estimation is harmful because management misuses the estimates, another is that estimation is a form for waste because it doesn’t provide value directly, and a third is that estimation is critical in order to do planning.  Let’s discuss each of these views and see what we really need to do regarding estimation.   The origins of most of the conversations of these three camps are #noestimates twitter community (believing estimates are misused by management), Kanban (believing it’s waste), and Scrum (believing in estimation).

The #noestimates Twitter Community

This movement was started by Woody Zuill, a consistently out of the box thinker, to engage in the conversation of where estimation was valuable.  It was not his intention not to do them ever, but rather learn where they are valuable and where they are not.  It has morphed into its current state where people talk about the lack of accuracy of estimates in general and their lack of utility because of it.

We have found that poor estimation is usually due to being unable to anticipate the added work created by interruptions and technical debt and not so much due to developers not understanding what it takes to get the work done.  This information can be useful to identify challenges that must be solved.   In any event, there is no need for each estimate to be accurate.  The intention is for estimation to be generally accurate across the sum of the estimates.  This is not impossible to achieve and provides enough basis for computing a consistent velocity on a team-by-team basis.

The “estimation is waste” Community.

Estimates are sometimes called waste because they provide no direct value.  However, they are often useful for coordinating teams.  They can also be useful to make sure people align on what the item being estimated is.  Finally, it is often a good practice to break larger stories down into smaller stories that can be completed in a fraction of an iteration.   It is not wasteful if it enables delivery of value quicker.  The key is not to dogmatically call it waste without a discussion, but to determine if true value add is being provided.

The Scrum Community Which Requires It

Scrum, and approaches based on it, such as SAFe, require estimation.  If you are doing Scrum, you should be estimating.  An alternative would be to do Kanban which doesn’t require it but still can have many of the Scrum ceremonies.  While many in the Scrum community complain about the time it takes to do estimation, this is mostly due to using Planning Poker

Planning poker is definitely useful in some situations and is easy to learn.  However, for the very reason it is effective in some situations, it is very inefficient in most. Planning poker requires more conversations than the other methods introduced here and this slows things down. The relative estimation method used is also less efficient than what is used in the alternative methods.  However, if you need to get folks to talk, Planning Poker may be your best choice.

Using Story Points

There are many ways to size stories – t-shirt sizes, hours, ideal engineering days, story points, …  Story points are pretty much the universally agreed upon sizing method for those industry though leaders who agree with estimation.  See Agile Estimation: 9 Reasons Why You Should Use story Points for the reason why.   When using story points, it is also suggested you use the modified Fibonacci sequence to size your stories.  This limits your choice sizes to: ?, 0, ½, 1, 2, 3, 5, 8, 13, 20, 40, 100, 200, 300, infinity.

Alternatives to Planning Poker

There are two popular alternatives to Planning Poker that are 4-10 times more efficient (meaning takes 10-25% of the time Planning Poker takes) while achieving similar estimation quality.  These methods are Steve Bockman’s Team Estimation and James Grenning’s Planning Poker Party. Both are much more efficient because they use true relative estimation (i.e., you first compare the stories/features with each other, making similarly sized groups) and then you assign them values.  

When Estimation Isn’t Needed

If you are in a maintenance organization, planning may be unnecessary.  But if you are needing to plan out multiple releases and/or need to take advantage of team-members whose skill set/abilities/experience is in high demand because few have it, estimation is often essential.  The mantra of "no estimation" seems to me to be a reaction of inefficient methods in the Scrum camp and an over-enthusiasm for the Lean mantra of eliminating waste from the Kanban camp. Actually, I don't like "eliminate waste" as a Lean Software Development mantra.  It is too easily confused and taken up by folks who aren't really clear what waste is.  Not all planning is waste.  Not all estimation is waste. It depends how you do it. 

Sometimes people believe estimation is required by stakeholders. Very often stakeholders are demanding estimates because they believe it is one way to raise productivity.  It often isn’t.  In many situations it doesn't need to be done for them.  In some situations it is actually useful for the team.  So in making the decision, the entire workflow from initial request to deployment should be considered.

In Summary

If teams are wanting to not do estimates because of what management does with them, then we have a problem between management and teams.  Just avoiding the estimates doesn't solve the problem.  Always go to what you are trying to achieve and why when looking at practices you are undertaking.  If estimation is taking more time than it's worth, see if there are more efficient methods than what you are using.  Don't throw out the baby with the bath water.

 

Subscribe to our blog Net Objectives Thoughts Blog

Share this:

About the author | Al Shalloway

Al Shalloway is the founder and CEO of Net Objectives. With over 40 years of experience, Alan is an industry thought leader in Lean, Kanban, product portfolio management, SAFe, Scrum and agile design.


Comments

Hi Al

The "Agile Estimation: 9 Reasons Why You Should Use story Points" link is invalid.

Thanks
Phil

Great summary of the camps, Al.

But I would add more about why estimation is so valuable.

I use a variation of Steve Bockman's Team Estimation technique, myself. But common to all our techniques: The entire team. Product Owner in the room. Entire backlog: stories, epics, features, or some combination thereof.

My teams are typically sizing from 50 to 150 cards. The longest I've seen it take - and I've used it with lots of teams across lots of domains - is 3 1/2 hours (though I expect I'll see four hours one of these days).

With the entire team spending 3-4 hours, the question of waste has to be addressed. Teams coming into agile have, previously, often sent just their leads or their managers to do estimating. We're bringing in the whole team. The whole team times a typical 3-4 hour timespan is an expensive meeting.

Why it's not waste is just how rich teams find its value.

There are two outcomes that are fairly obvious: Product owners come away able to order the backlog by bang-for-the-buck because we just gave them the "buck" on every feature. (In my experience, product owners are, on their own, frequently wrong in their guesses; and extremely grateful for team feedback that gives them real sizes to work with.)

Second, we get story points to use to develop velocity so that, two or three sprints down the line, we can begin to predict how deep in the backlog we can get by a certain date - which lets us draw a line there and ask, "Do we really have the right stuff above the line?" Dates and market windows are real. Estimating an entire backlog of features in three hours is a pretty cheap way to get insight (especially if you compare that to the days of painful estimating in dark rooms that we spent building GANTT charts.)

Those two outcomes alone might be enough. But there's more.

Engaging the entire team delivers value to the process itself. Wisdom of the crowd works. Partly because it engages the expertise and experience (and wisdom) of every individual. Each knows stuff the others don't. Sometimes it's expertise: someone has done this kind of story before: turns out Sue could code it in her sleep. Sometimes it's experience with the existing code: turns out Bill has worked in the code that Sue would add to, and it's laced with technical debt issues that will likely cause bugs not just locally but in unpredictable ways in other code routines and modules as well. Without Sue and Bill in the room, you don't get either insight.

Then there are teambuilding benefits. I've had product owners come away saying, "I didn't know estimating could be fun." And I've had developers and testers and analysts and designers across teams come away saying, "We've never had these cross-team discussions."

Then there is the benefit of giving the entire team an overall project understanding. I've had engineering managers wow'd by seeing their entire team, in just 3-4 hours, getting a holistic picture of the project.

The Team Estimation meetings I've run have been 3-4 hour meetings that, in fact, are typically tiring in addition to being interactive and fun - but have been so rich in benefits and side benefits that the term "waste", regardless of definition, simply could not be pejoratively applied.

Blog Authors

Al Shalloway
Business, Operations, Process, Sales, Agile Design and Patterns, Personal Development, Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Cory Foy
Change Management, Innovation Games, Team Agility, Transitioning to Agile
Guy Beaver
Business and Strategy Development, Executive Management, Management, Operations, DevOps, Planning/Estimation, Change Management, Lean Implementation, Transitioning to Agile, Lean-Agile, Lean, SAFe, Kanban, Scrum
Israel Gat
Business and Strategy Development, DevOps, Lean Implementation, Agile, Lean, Kanban, Scrum
Jim Trott
Business and Strategy Development, Analysis and Design Methods, Change Management, Knowledge Management, Lean Implementation, Team Agility, Transitioning to Agile, Workflow, Technical Writing, Certifications, Coaching, Mentoring, Online Training, Professional Development, Agile, Lean-Agile, SAFe, Kanban
Ken Pugh
Agile Design and Patterns, Software Design, Design Patterns, C++, C#, Java, Technical Writing, TDD, ATDD, Certifications, Coaching, Mentoring, Professional Development, Agile, Lean-Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Marc Danziger
Business and Strategy Development, Change Management, Team Agility, Online Communities, Promotional Initiatives, Sales and Marketing Collateral
Max Guernsey
Analysis and Design Methods, Planning/Estimation, Database Agility, Design Patterns, TDD, TDD Databases, ATDD, Lean-Agile, Scrum
Scott Bain
Analysis and Design Methods, Agile Design and Patterns, Software Design, Design Patterns, Technical Writing, TDD, Coaching, Mentoring, Online Training, Professional Development, Agile
Steve Thomas
Business and Strategy Development, Change Management, Lean Implementation, Team Agility, Transitioning to Agile
Tom Grant
Business and Strategy Development, Executive Management, Management, DevOps, Analyst, Analysis and Design Methods, Planning/Estimation, Innovation Games, Lean Implementation, Agile, Lean-Agile, Lean, Kanban