To Automate or Not to Automate? Part 2

It’s time for part 2! In our previous entry about testing automation, “To Automate or Not to Automate: Part I” we discussed two key considerations which can help you decide if automated testing is a fit for your organization: first, what’s at stake, and second, the present state of affairs. Today we will be discussing another power couple which can turn your testing decision upside down: budget and organizational readiness.


One of the common myths on automated testing is the notion is that it is too costly and will bloat the project budget. It might be a stretch to say that without a plan this is guaranteed to be true, but achieving ROI on automated testing is much more likely with a testing roadmap.  A testing roadmap will bring discipline to the decision about test automation by evaluating:

  • Is the current testing hot button a one-time issue or is it likely to re-occur?
  • Are there additional projects forthcoming which will employ the same technology toolsets?  Whatever automated testing tools you invest in will be around for some time to come, so they need to play well with other applications that are coming down the road.
  • Can we create efficiencies by combining the production of automated test scripts with other activities such as generating change management and training materials? This strategy can pay tremendous dividends, but is only possible by planning in advance.
  • If you already have an automated testing tool that is currently sitting on the shelf, first of all, you are in good company. Many organizations purchase testing tool sets but do not implement them, for a variety of reasons that we will explore in later blog entries.  However, owning the tool, while eliminating one cost from the ROI equation, does not translate to an automatic decision to deploy test automation. After all, not only is there a cost and effort associated with initial development of the automated test scripts – a skill set which likely does not exist in house if the tools aren’t in use – but after the initial deployment, someone will need to maintain the test scripts as the systems evolve over time.

What is worse than not having budget? A poorly planned project which results in achieving an accurate appreciation of project costs mid-way through the initiative (translation – you blew the budget). A roadmap approach will allow you to formulate a more realistic evaluation of the costs AND benefits of test automation over time, taking into account your organization’s IT portfolio, internal skill base, vendor relationships and project priorities. One way to catalyze this planning process is through a testing strategy engagement.

Organizational Readiness

The last factor that we will consider today in relation to test automation is organizational readiness. Test automation will require time, planning and resources for a successful outcome. Even if you are leveraging automation tools for a packaged application and plan to purchase vanilla test scripts to jump start your test script development, the effort to map and gap business processes and update the test script library should be systematic, well documented and planned with an eye towards dependencies within ongoing IT projects. Reallocating people and tasks to the testing initiative will require full support from IT managers who may already find their staff stretched thin. If experience with testing best practices and specific testing tool sets does not exist in house, it may be necessary to engage external vendors to make sure that the program gets underway with solid footing. On the IT operations side, launching an automated testing initiative may require setup of an additional testing environment, which has the potential to drive additional costs beyond the testing tools in the form of DBA labor, SysAdmin labor and storage consumption.

The maturity of IT operations and overall strength of IT governance are very relevant to whether the organization will be able to derive full value from automated testing. Will the organization have the focus and follow through to improve testing or will there always be a “crisis of the week” to address? Will resource continuity be maintained or will the initiative move at a snail’s pace as resources come on the project and are brought up to speed only to be reassigned to more pressing issues?

One flash test for IT maturity is to look at the company’s IT support operations.  You can gain some insight into the IT maturity by investigating how the company addresses production issues.

  • Are issues consistently ticketed?
  • Is there a root cause analysis even for small issues?
  • Is there an attempt to analyze frequently recurring issues and invest in solutions?

Of course, the consistency of business processes and the documentation of system setups and processes are highly desirable as a foundation for test automation.  This is relevant both for the expeditious development of test scripts and as an indicator of the organization’s ability to  continuously maintain a library of test scripts.   IT Convergence provides documentation services for organizations that need to catch up and of course integration between documentation and testing toolsets (such as Oracle UPK and OATS) represents a tremendous source of value.

To conclude, it’s safe to say that when employed properly, automated testing tools can drive cost reductions, prevent costly failures and allow IT staff to focus on higher value-add activities. However, realizing the value of these tools requires up-front planning and a realistic appraisal of the potential value as well as the organization’s ability to successfully execute the project. Creating a testing roadmap is an excellent way to go into these projects with eyes wide open. Contact us to learn how to get started.