Search for “automated testing” on Google and it will return about 6,67,000 results in 0.28 seconds. Undoubtedly, there is the lot of interest, curiosity, notions, and misconceptions about it. Take any software project and you will hear discussion about test automation at least once during its lifetime. There have been long-standing debates about Manual Testing Vs Automated Testing. There are ardent followers of automated testing who believe that it is a magic wand which can completely replace the manual testing (not true!) and there are groups who swear by manual testing and believe that the test automation is just a fad (again, not true!).
Although Test Automation has come a long way from age-old Record and Playback framework-based test automation, we see that there are some fundamentally wrong things, as described below, which are behind unsuccessful test automation projects.
# Work in isolation
Automation teams are the technical people who ‘understand’ technology and that’s all they need to know. You give them the requirement and they will write the code for that. Unfortunately, it is not that simple. Writing good automation scripts require knowledge of the application and knowledge of the domain. Companies generally have separate teams for manual testing and automated testing. The manual testing teams are typically the application experts who have the deep understanding of the application. Automation teams work separately with the help of “test cases” and “specs” which are documented. In our opinion, this is a recipe for failure because, in the absence of complete application knowledge, an automation engineer cannot create a robust and scalable test automation framework which is extensible – even if she is a good a test engineer or has deep knowledge of testing programming language.
# Incorrect Expectations
One of the most common misconceptions about automation is that it is required only when you want to save time and money on testing. While saving time and cost are some of the major benefits of test automation, the real benefits go beyond that – automated testing is useful to build effectiveness, efficiency, increase test coverage, have consistency in testing, guarantee accuracy and help in regression testing. It also helps in improving the team morale because, with automation, you create time for the team to work on more challenging tasks. Having the right expectations from automation makes it easier for organizations in accurately measuring the true Quality of application.
# Automation as an afterthought
“Let the application get built first, and we will do automation for version 2.0” – we hear it quite often. There is a common belief that test automation cannot be started unless the application is fully built and ready. In reality, the true ROI from test automation can be achieved if it starts along with the development of the application – as soon as the wireframes are finalized. Having a test automation design and strategy ready early on in the project lifecycle can help in faster time to market, more test coverage and better application stability.
# Aim for 100% Automation
There are thousands of combinations because of which it is practically impossible to achieve 100% test automation. Test automation can enable more tests with more data and configurations – but that does not necessarily mean better quality. Instead of aiming for 100% test automation, the goal should be on testing the critical functionality for the application and building a solid automation design and strategy which is sustainable and extensible to accommodate the future versions of the application.
# ROI Measurement
To derive true Return on Investment, first the organizations need to be clear on the objectives of automation and the ROI has to map with the objectives. Reduced costs, faster testing, and reliability are some of the parameters for realizing the Test Automation ROI. One needs to be aware that automation is NOT for improving test processes or it is not a replacement for manual testing.
Remember that test automation is a strategic decision and the real ROI is realized in the long run when the same tests are executed multiple times at regular intervals. In my upcoming posts, we will cover ROI measurement in more detail. Stay tuned!