With the software industry getting more focused and attuned towards directly impacting customer satisfaction than ever before, it has become all the more important for software to become more responsive. Software changes are more frequent and demand stringent Quality parameters which enforce a highly efficient and automated development and quality process. Test Automation for this reason has seen a sea change in its adoption levels in recent years.
Though there are multiple factors responsible for delivering successful test automation, the key is selecting the right approach. Now, depending on your requirement for automation, approaches can vary. Here are the basic approaches popularly adopted.
Record and Playback
Record and playback just does not work. Just in case you are tempted by its ease, any automation expert can tell you that this often comes at a huge cost of never ending maintenance efforts and rework.
Use of a well designed framework
A reliable Test Automation Framework,
- Makes even the most complex test automation extremely easy
- Promotes reuse of critical components, increasing productivity
- Make adopting changes easy and reduce maintenance efforts
- Support any test automation tool of your choice (both current and future)
- Be adopted and used as a standard automation framework across the organization
- Allows test engineers with application/domain knowledge to automate tests irrespective of their knowhow on automation.
All of which contribute towards shortening the testing cycle time after time, during each regression.
Nothing is more important than having the right framework that will ensure test automation success. Whatever approach that you use, there are some benefits that are worth considering.
Now that we understand what to expect from successful test automation, the other relevant question is should we build or buy a framework? There is no simple answer to this one.
Let's first see what it takes to build a successful framework.
Here are a few things that have worked for me.
Clear Vision and Scope
Like any application / product development, it is extremely important to have a clear vision that will shape your framework and even set expectations clearly to begin with.
Questions such as the ones below can get you started on defining your vision
- Who will use my framework? (For e.g. automated testing tool experts, my extended framework development team, my SMEs or manual testers, etc...)
- What type of applications should I be able to automate using my framework?
- What Test automation tools should I be able to use with my Framework?
- Can my users use it with minimum learning?
Framework development Team
Your team should have a strong perspective of Software Design and Architecture. A Software architect accompanied by developers and Test Automation experts is the ideal choice.
Architects and developers ensure that they deliver a scalable and maintainable framework and the Automation experts can join the developers in implementing the framework and also provide the test automation domain knowledge required.
Usage and Adoption
A framework should be one that is easy to adopt and fairly intuitive for new users. It should bring basic standardization and uniformity to build robust and scalable test automation.
Ongoing maintenance
Building a framework is not a ‘build and forget’ activity. Framework requires constant monitoring, enhancements, modifications and maintenance to deliver expected results.
Proactive teams that understand the critical success factors of the framework need to constantly keep pace with the change requirements.
Cost Considerations
Framework development and maintenance costs need to budgeted, justified and monitored thoroughly and frequently. It’s commonly observed that the Maintenance cost of a framework outweighs the Development Cost by a wide margin. Predominant reasons are poor design and/or an unskilled development team.
Now, let's see what it takes to buy. Of course, the simple part of the answer is dollars.
With the understanding on the requirements of a well designed framework we can thoroughly evaluate a ready to use framework.
It is always a better option to first evaluate a ready framework for few simple reasons. Firstly it is ready and you can test it against your application(s) and weigh your options. Secondly, it always helps you to borrow design concepts in case you decide to build one.
Here are some guidelines to effectively evaluate a framework before you buy,
- Use some of the most complex applications and tests for evaluation.
- Make sure that you test the framework for maintenance. What happens if your test cases workflow or objects change? Adopting such changes can get pretty painful.
- Make sure that the framework is easy to use for your entire QA team and the learning curve for new adopters is not very steep.
- Don’t miss out on any of the "well designed framework" benefits that we discussed above. Check that the framework is extendable and flexible enough to handle complexities unique to your applications.
- Understanding limitations of a framework is a virtue. What appears to be a complete solution upfront can easily results in an illusion unless efforts are taken to know the shortcomings.
- Check for references and talk to others who have adopted this framework and solicit their candid feedback.