Why Test-Driven Development Really Isn’t Test First

August 9, 2008 — Posted by Al Shalloway

OK, that was an intentionally provoking title. I believe in TDD, but I don't believe it's really Test-Driven Development. It's something else. I remember when people first started calling it TDD I used to say TDD stood for Test-Driven Design. I'd say this because your tests helped you define how you designed your code. But this assumes we even know what I mean by Test-Driven Development. What kind of testing first am I even talking about? When TDD first became popular, we typically meant TDD meant writing unit tests first. Many of us realized that if we combined a series of unit tests together we could get the equivalent of automated acceptance testing.

However, there was a better way. With the invention of Fit (Framework for Integrated Testing) defining acceptance tests became a separate process than combining unit tests together. We now had an easy method to have non-programmers define and (virtually) implement acceptance tests. These two testing practices were still often practiced separately.

Eventually I came across Rick Mugridge's Fit For Developing Software (the first 50 pages of which are a must read for all developers in my mind regardless of your testing practices). He eloquently states how defining acceptance tests up front improve the quality of the conversation between QAs, BAs, and devs. Given that these people will (or at least should) reflect on acceptance test definitions at one time, this implies we should pick the most beneficial time to do this – and that'd be up-front (I write more about this in The Role of Quality Assurance in Lean-Agile Software Development).

This creates value (better conversation, better understanding, clearer scope) without creating extra work. This leads to the insight that we should not drive our acceptance tests from our unit tests, but rather do it the other way around since we need to be creating our acceptance tests first. In other words, our unit tests need to be created within the context of manifesting the behavior our acceptance tests dictate.

As a disciple of Christopher Alexander, this makes sense in another way. Alexander's work proposes that designing from context (the big picture) creates better designs. However, I believe this is a broader principle than just a design principle. In general, I suggest keeping in mind why you are doing anything (the context) improves whatever it is you are doing (whether it is design or something else).

A big part of agile software development is discovering what the customer wants or needs. In doing this, one writes stories. Following Alexander's ideas here would mean that we should start with the big picture of the behavior we want, then go into the details. Again, this means driving unit tests within the context of the desired behavior. Hence, TDD is not even design first as much as it is analysis and then design. So perhaps we have Test Driven Analysis and Design and then Development (TDADD?).

But are we really testing first? I don't think so. Let's look at another activity – making and using plans. We can talk about planning in three ways:

  • Planning (the action of making the plan)
  • Plan (a description of steps we intend to perform)
  • Following the plan (doing these steps)

Now in testing, we have words for steps 2 and 3 above, but not for step one. For example, the equivalent of a plan is a test specification. The equivalent of following the plan is running the tests. But what is the equivalent of planning? I would say it is creating the test specification. In other words, we have:

Plans

Tests

Planning

Creating the test specification

Plan

Test specification

Following the plan

Running the test

So what we actually do is up-front Test-Specification (OK, so I'm being picky – but a test specification is different from running the test). Test-specifications, of course, are another way of stating what the system needs to do – that is, it is analysis. I guess you could say we do TSDADD (Test-Specification Driven Analysis and Design and Development). No wonder they shorted it to TDD! Funny, however, that it wasn't given an intention revealing name! ;)

The challenge now with the term TDD is that it has a lot of latitude on what it means. Am I starting from the functional level (unit tests - TSDDD)? Or the behavior of the system level (acceptance tests - TSDADD)? To avoid confusion in the rest of this article, I am going to refer to what I've been (tongue-in-cheek) calling TSDADD, Acceptance Test-Driven Development (or ATDD). I will refer to the class writing unit tests first UTDD. ATDD is very similar to Dan North's BDD but has some differences that I will not review here.

As stated earlier, starting from the behavioral level (the big picture) creates better conversations and context. Let's look at the advantages of doing so:

  • Better design
  • Improving the clarity of scope – avoiding excess work
  • Reducing complexity

Better Design

Let's get back to Alexander's hypothesis. Alexander states that designing from context gives insights into what the functionality you are creating needs to do. Designing from the whole enables you to see how the smaller pieces should fit together. This is a fundamental design principle which works in other industries besides his (building construction). Although software development is not at all like building buildings, design patterns are somewhat based on this concept (although I believe there is even more confusion about what design patterns are – see Can Patterns Be Harmful?). One aspect of this is The Dependency Inversion Principle. We should expect then, that doing ATDD should result in better designs than the classic UTDD where we specified the pieces first and put them together.

Improving Clarity of Scope – Avoiding Excess Work

By providing our acceptance tests up front, we help the developer understand what is in the scope of the requirement. This avoids developers from overbuilding the system. This, of course, helps avoid excess work.

You might be concerned that this minimalist design will not prepare us for future changes. But just the opposite is true. By having an automated test suite, high quality code and an understanding of quality design (design patterns) you set yourself up to be able to add design changes later, when you both know they are needed and you know how to implement them. This is much more efficient than designing ahead of time.

Another way acceptance tests can help us avoid work is they can give us an indication of how far down we have to go in specifying unit tests. UTDD somewhat requires every function to be tested. Because the unit tests are your safety net. But in ATDD, the acceptance tests are your safety net – the unit tests are there to help you design your functions and to find errors faster (both good things – but not always necessary).

Reducing Complexity

Overbuilding systems is one of the greatest causes of waste in software development. Function that isn't used doesn't just waste the time it took to write it, it makes the system more complex. This makes adding other functionality or fixing existing errors much more time-consuming. Not to mention that the system is likely to have more errors in it. By clarifying the scope of work in a clearer way, we can help avoid this.

Other Advantages

If you are building software that must meet government testing specifications (e.g., health care instrumentation) it is easier to prove the software is doing what it needs to if you have a full set of automated acceptance tests. If you just have a set of unit tests (even a complete set) you will still have to demonstrate that your unit tests demonstrate acceptance test criteria. If you are going to have to specify these acceptance tests anyway, might as well do them first for all the reasons we've been mentioning.

A Comment on Paired Programming

I have always liked paired programming. I had done things like it, but not in a disciplined manner, years before I heard of eXtreme Programming. Most people who haven't done it, however, have difficulties seeing why (or how) it would be useful. As an educator (trainer/coach) my work requires not just understanding why something works, but knowing how to enroll others in why things work. Actually, this is true of all educators, but is something we particularly ascribe to at Net Objectives. People readily see the advantage of having several people involved in a conversation about acceptance tests (again, see The Role of Quality Assurance in Lean-Agile Software Development). Paired programming provides somewhat the same advantage at the coding (unit-testing) level.

No Excuses

By the way, there may be certain arguments for not writing and maintaining a set of automated tests. I don't think I would agree with them, or at least, in virtually all cases I am fairly sure I wouldn't. However, there is little argument in not at least specifying the tests first. This is because you are going to have to specify them at some time. You might as well do it at the time it provides the greatest value.

Summary

There are two kinds of Test-Driven Development. The classic style of writing unit tests first (often called TDD but we are now calling UTDD) and the more productive method of writing acceptance tests first and then your unit tests (which we are calling ATDD). Both are really about specifying your tests first. Then writing the tests, then writing your code, then running the tests. This creates many advantages. By creating unit tests from the context of your acceptance tests, you will get a better definition of your scope, avoid doing extra work, reduce complexity and achieve better designs.

Subscribe to our blog Net Objectives Thoughts Blog

Share this:

About the author | Al Shalloway

Al Shalloway is the founder and CEO of Net Objectives. With 45 years of experience, Al is an industry thought leader in Lean, Kanban, product portfolio management, Scrum and agile design. He helps companies transition to Lean and Agile methods enterprise-wide as well teaches courses in these areas.


Comments

Oddly enough, after posting this blog, I read the prior blog.  Wanted to make sure people were clear this is not a response to Rob's blog (which you should read if you haven't).

Alan Shalloway, CEO Net Objectives

Blog Authors

Al Shalloway
Business, Operations, Process, Sales, Agile Design and Patterns, Personal Development, Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Cory Foy
Change Management, Innovation Games, Team Agility, Transitioning to Agile
Guy Beaver
Business and Strategy Development, Executive Management, Management, Operations, DevOps, Planning/Estimation, Change Management, Lean Implementation, Transitioning to Agile, Lean-Agile, Lean, SAFe, Kanban, Scrum
Israel Gat
Business and Strategy Development, DevOps, Lean Implementation, Agile, Lean, Kanban, Scrum
Jim Trott
Business and Strategy Development, Analysis and Design Methods, Change Management, Knowledge Management, Lean Implementation, Team Agility, Transitioning to Agile, Workflow, Technical Writing, Certifications, Coaching, Mentoring, Online Training, Professional Development, Agile, Lean-Agile, SAFe, Kanban
Ken Pugh
Agile Design and Patterns, Software Design, Design Patterns, C++, C#, Java, Technical Writing, TDD, ATDD, Certifications, Coaching, Mentoring, Professional Development, Agile, Lean-Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Marc Danziger
Business and Strategy Development, Change Management, Team Agility, Online Communities, Promotional Initiatives, Sales and Marketing Collateral
Max Guernsey
Analysis and Design Methods, Planning/Estimation, Database Agility, Design Patterns, TDD, TDD Databases, ATDD, Lean-Agile, Scrum
Scott Bain
Analysis and Design Methods, Agile Design and Patterns, Software Design, Design Patterns, Technical Writing, TDD, Coaching, Mentoring, Online Training, Professional Development, Agile
Steve Thomas
Business and Strategy Development, Change Management, Lean Implementation, Team Agility, Transitioning to Agile
Tom Grant
Business and Strategy Development, Executive Management, Management, DevOps, Analyst, Analysis and Design Methods, Planning/Estimation, Innovation Games, Lean Implementation, Agile, Lean-Agile, Lean, Kanban