Manual Test Automation - Part 1

January 18, 2017 — Posted by Maurizio Mancini

This 3 part blog post and the title for it is inspired by a colleague who is currently living the manual test automation nightmare on an engagement. He reminded me how common a pattern it is and one that I have seen way too often over various engagements. Based on recent engagements and stories I hear from other colleagues in the industry, it is clear that it will be a trend that will continue for some time to come.

When people talk about test automation it means different things to different people so let’s start this blog post by looking at the definition of the word automation.

According to Merriam Webster, automation is defined as follows:

  1. the technique of making an apparatus, a process, or a system operate automatically
  2. the state of being operated automatically
  3. automatically controlled operation of an apparatus, process, or system by mechanical or electronic devices that take the place of human labor

The keywords in this definition are “automatically” and “take the place of human labor”. In my opinion, this is a clear and precise definition of what automation is.

When it comes to Test Automation in software, for some reason the definition of automation quickly gets skewed and/or adapted depending on the organization and the people that are implementing and executing software test automation.

Whenever I start an evaluation of a customer's Test Automation efforts, I start by asking one basic question:

“Does your Test Automation execute on a regular basis without someone babysitting the automated scripts?”

With many customers, the answer is usually; “Well, it is automated and sort of runs on its own but we can explain…”

This generally is the first sign that a customer is caught in the “Manual Test Automation” rat hole. Usually what follows is a whole bunch of “reasons” as to why their test automation needs manual intervention.

Here are but a few of the common patterns that could be an indication that you are running manual test automation:

  • The test environment is not reliable.
  • The test data keeps changing or you need to find data.
  • The scripts are End to End test automation scripts.
  • The End to End environment consists of many systems (Internal and External) that affect the system being automated.
  • The test automation tool is a “scriptless” one.  
  • The solution is very complex.

If this is your reality, then you may be stuck in the Manual Test Automation rat hole.

Why are you doing Test Automation?

Getting out of the Manual Test Automation rat hole starts by asking yourself, “Why are we doing Test Automation?” Do you have a clear and defined goal of what you expect out of Test Automation? If you cannot answer these questions quickly and with a clear answer, then chances are Test Automation has been implemented for the wrong reasons. Generally, when the goal of software test automation is not clear, the implementation is not optimal.

Here is an example of the 3 most common patterns I have seen across engagements. Do you see any that describe your organization?

Reason 1 - Test Automation was implemented to “fix” the problem of bugs/defects slipping into production.

I have already covered the issue of defects/bugs slipping into production in another blog post. Trying to use test automation to “fix” the problem of bugs slipping into production is just another example of that same problem. In summary, the organization should be focusing on trying to avoid having defects/bugs getting created in the first place rather than trying to implement test automation at the end of the development cycle trying to “catch all the bugs” before they make it to production.

Reason 2 - Test Automation is being implemented as the “magic bullet” to fix all of the software delivery issues.

In my experience Test Automation that is “bolted on” after the software is completed, is usually expensive, doesn't provide good test coverage and is being seen as the savior to fix deeper organizational issues.

If test automation was implemented in this context, then it is usually indicative of an organization that:

  • May have a management team that doesn’t fully comprehend software development.
  • Is weak at producing clear requirements for the teams to work with and that show clear Business Value.
  • May have an organizational structure where communication between various teams is not optimal.
  • Has developed/implemented a software system which is not optimal and requires constant “patching”. The technical debt that has been incurred over the years of development is just too great and is not being addressed.

It usually points to an organization whose managers look to test automation to try and save the day rather than deal with these deeper issues.

Reason 3 - Test Automation is being implemented with the intent to replace manual testing which is seen as being a problem in the organization. The problems that usually exist in this case are:

  • Manual testing keeps taking longer and longer
  • Manual testing is seen as being ineffective and non-comprehensive
  • Manual testing is conducted from a End user perspective only

In this case the organization has yet to realize that building quality software is a team effort and management is still thinking that the responsibility for quality lies solely with one team - the QA/QC team.

In part 2 of this blog post, we will review 5 suggestions you can start with to improve your test automation strategy.

If you would like to discuss how Exempio can help you develop a Test Automation strategy that can work with your software delivery goals, whether you work in Agile or not, please​ ​contact Exempio or Net Objectives​ for a consultation.

Subscribe to our blog Net Objectives Thoughts Blog

Share this:

About the author | Maurizio Mancini

Maurizio Mancini is a leader in the quality and process industries with a sixth sense for Agile, quality, and business process. He is best known for cutting through the noise and getting to the heart of any organizational problem whether that problem consists of choosing the right software development process, implementing modern quality approaches, or just finding the right balance between people, process and tools. Maurizio’s approach consists of empowering people and teams so that the team's talent and creativity comes through naturally. This mindset favors the adoption of Agile values in any environment.



        

Blog Authors

Al Shalloway
Business, Operations, Process, Sales, Agile Design and Patterns, Personal Development, Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Cory Foy
Change Management, Innovation Games, Team Agility, Transitioning to Agile
Guy Beaver
Business and Strategy Development, Executive Management, Management, Operations, DevOps, Planning/Estimation, Change Management, Lean Implementation, Transitioning to Agile, Lean-Agile, Lean, SAFe, Kanban, Scrum
Israel Gat
Business and Strategy Development, DevOps, Lean Implementation, Agile, Lean, Kanban, Scrum
Jim Trott
Business and Strategy Development, Analysis and Design Methods, Change Management, Knowledge Management, Lean Implementation, Team Agility, Transitioning to Agile, Workflow, Technical Writing, Certifications, Coaching, Mentoring, Online Training, Professional Development, Agile, Lean-Agile, SAFe, Kanban
Ken Pugh
Agile Design and Patterns, Software Design, Design Patterns, C++, C#, Java, Technical Writing, TDD, ATDD, Certifications, Coaching, Mentoring, Professional Development, Agile, Lean-Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Marc Danziger
Business and Strategy Development, Change Management, Team Agility, Online Communities, Promotional Initiatives, Sales and Marketing Collateral
Max Guernsey
Analysis and Design Methods, Planning/Estimation, Database Agility, Design Patterns, TDD, TDD Databases, ATDD, Lean-Agile, Scrum
Scott Bain
Analysis and Design Methods, Agile Design and Patterns, Software Design, Design Patterns, Technical Writing, TDD, Coaching, Mentoring, Online Training, Professional Development, Agile
Steve Thomas
Business and Strategy Development, Change Management, Lean Implementation, Team Agility, Transitioning to Agile
Tom Grant
Business and Strategy Development, Executive Management, Management, DevOps, Analyst, Analysis and Design Methods, Planning/Estimation, Innovation Games, Lean Implementation, Agile, Lean-Agile, Lean, Kanban