Blog Series: TDD and Process, Part 2

March 31, 2017 — Posted by Scott Bain

Part 2: Basic (A)TDD Automation

In part 1 we expressed a view that many people hold when it comes to the various kinds of tests that we write in a process that centers on Test-Driven Development:

“Your tests can either provide you a great deal of value and precise information when they fail, or they can be more broadly useful to the organization when they are passing (when they accurately represent the behavior the system has).  You can’t have both.”

In other words, you can have fast, fine-grained tests that nobody but the technical people understand (the typical unit tests), or you can have slow, coarse-gained tests that everyone can read, write, and modify (the typical acceptance test).  Or, that you can create both but this will create tons of redundancy and thus will make test maintenance a nightmare.

This is what we seek to debunk; to show that we can have the best of both worlds without a lot of excessive work and without suffering under the burden of redundancy.

All TDD is TDD

Step one here is to demystify the difference between TDD, ATDD, BDD, etc....  It’s all TDD.  When people talk about automating TDD using xUnit or Fitnesse, or Cucumber/Specflow, or Robot, or whatever, the real difference is not the nature of the activity we are engaging in, but rather the “test automation framework” (TAFW from this point forward) we have chosen.  The one thing does not imply the other: you can write acceptance tests in xUnit, you can write unit tests in Specflow.  We choose the test automation framework based on two issues:

  1. Who are the stakeholders for the behavior we are seeking to create?  Who will be able to say “yes, that is the right behavior” or “no, that’s not valuable”, or “you’re missing something important here”, that sort of thing.  If we are creating a customer-facing feature, it might be a marketing person or a business analyst.  If we are creating an internal DevOps tool, it might be the developers and testers who will be using it.   This leads us to the clearest form of expressing the behavior, namely that which is best understood by the stakeholders we have identified.
  2. What is the best TAFW for that expression?  Different frameworks are more facile at expressing behavior in one way or another.  FIT is better if you’re using pictures.  Specflow is better if you’re using human language (literally Given/When/Then as shown in part 1 of this series).  xUnit is better if you’re expressing the behavior in technical language (code).

In any case, the general architecture of automation looks like this:

TAFW Architecture

Some document exists that makes clear what the behavior that is needed looks like, aka the specification.  Something reads that text (a framework for this purpose, but frankly it could be a human being) and then looks for code that has been written to execute each step against the production code.  Developers provide this code and thus make it an “executable specification”.

If we do this in the test-first process, then this test execution should run successfully, but the test should fail.  This “drives” the developers to create the production code needed to make it pass.  Thus, test “driven” development.

Automation

The beginning of creating the efficiency, speed, and granularity we want is through automation.  Even though I wrote “frankly, it could be a human being” the fact is that even though you may start this way (if your organization is new to TDD) you probably don’t want to stick with that.  You’ll eventually want to automate the parsing.  There are lots of ways to do this, but for this blog series we’re going to focus on a language called “Gherkin” for the specification and Specflow (the .Net version of Java’s Cucumber) as the TAFW, and therefore C# for all the developer work.  None of the points I’m going to make would be different if you changed any of these specifics.

Remember the simple specification we looked at in part 1?:

Given: A base sale amount of $200.00

And: A commission rate of 8%

When: The total sale is calculated

Then: The total sale is $216.00

Gherkin uses this very same sort of language, with just a few more syntactical specifics:

Feature: Sales Calculator

Scenario: Appling Sales Commission

    Given A base sale amount of $200.00

    And A commission rate of 8%

    When The total sale is calculated

    Then The total sale is $216.00

Anyone with a little bit of instruction can learn to read this[1], or even write this in the first place.  One does not need to understand computer languages, idioms, or really anything other than (in this case) English.

What Specflow will do, when a test execution is attempted, is to parse this file (called a “feature file”) and pull it apart, then look for methods in code that match each Given, When, and Then statement.  In this case, regular expressions are used to match things up.  Since we have not written any code yet, such methods will not be found and the tool will “complain”, showing what it was looking for and could not find. 

For example, if all we had created was the feature file above, and we attempted to run it as a test though Specflow, here is what the test results would say:

No matching step definition found for one or more steps.
using System;
using TechTalk.SpecFlow;
namespace MyNamespace {
    [Binding]
    public class StepDefinitions {
        [Given(@"A base sale amount of \$(.*)")]
        public void GivenABaseSaleAmountOf(Decimal p0){
            ScenarioContext.Current.Pending();
        }

        [Given(@"A commission rate of (.*)%")]
        public void GivenACommissionRateOf(int p0){
            ScenarioContext.Current.Pending();
        }
  
        [When(@"The total sale is calculated")]
        public void WhenTheTotalSaleIsCalculated(){
            ScenarioContext.Current.Pending();
        }

        [Then(@"The total sale is \$(.*)")]
        public void ThenTheTotalSaleIs(Decimal p0){
            ScenarioContext.Current.Pending();
        }
    }
}

Convenient!  All we do is copy the code into a .cs class and replace each

“ScenarioContext.Current.Pending();”

…with the correct code to exercise the component(s) we’re developing when they are called.  Most developers would also want to clean this up a bit to use more meaningful parameter names, maybe change the method names that conform to their organization’s coding standards, use a real namespace, etc.  So long as the regular expressions contained in the attributes continue to match the text in the feature file, Specflow does not care.

How does this solve the problem of “you can’t have it all?”  That has everything to do with the way we design the binding layer, which is the topic for part 3 of this series.

 




[1] For a full definition of Gherkin, visit this link: https://github.com/cucumber/cucumber/wiki/Gherkin

 

Subscribe to our blog Net Objectives Thoughts Blog

Share this:

About the author | Scott Bain

Scott Bain is an consultant, trainer, and author who specializes in Test-Driven Development, Design Patterns, and Emergent Design.



        

Blog Authors

Al Shalloway
Business, Operations, Process, Sales, Agile Design and Patterns, Personal Development, Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Cory Foy
Change Management, Innovation Games, Team Agility, Transitioning to Agile
Guy Beaver
Business and Strategy Development, Executive Management, Management, Operations, DevOps, Planning/Estimation, Change Management, Lean Implementation, Transitioning to Agile, Lean-Agile, Lean, SAFe, Kanban, Scrum
Israel Gat
Business and Strategy Development, DevOps, Lean Implementation, Agile, Lean, Kanban, Scrum
Jim Trott
Business and Strategy Development, Analysis and Design Methods, Change Management, Knowledge Management, Lean Implementation, Team Agility, Transitioning to Agile, Workflow, Technical Writing, Certifications, Coaching, Mentoring, Online Training, Professional Development, Agile, Lean-Agile, SAFe, Kanban
Ken Pugh
Agile Design and Patterns, Software Design, Design Patterns, C++, C#, Java, Technical Writing, TDD, ATDD, Certifications, Coaching, Mentoring, Professional Development, Agile, Lean-Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Marc Danziger
Business and Strategy Development, Change Management, Team Agility, Online Communities, Promotional Initiatives, Sales and Marketing Collateral
Max Guernsey
Analysis and Design Methods, Planning/Estimation, Database Agility, Design Patterns, TDD, TDD Databases, ATDD, Lean-Agile, Scrum
Scott Bain
Analysis and Design Methods, Agile Design and Patterns, Software Design, Design Patterns, Technical Writing, TDD, Coaching, Mentoring, Online Training, Professional Development, Agile
Steve Thomas
Business and Strategy Development, Change Management, Lean Implementation, Team Agility, Transitioning to Agile
Tom Grant
Business and Strategy Development, Executive Management, Management, DevOps, Analyst, Analysis and Design Methods, Planning/Estimation, Innovation Games, Lean Implementation, Agile, Lean-Agile, Lean, Kanban