Why Automate Regression Testing?The improvements and new features we contribute to the Open Source Testing world put it at the forefront of automated regression testing tools for Web applications. When an organization chooses to build and host a Web application, automated regression testing becomes a requirement for success.
Organization's begin with a Web-based application launch and soon respond to changing user needs by maintaining and growing the site. As the function of a Web-based application grows with each succeeding release, testing needs grow geometrically. The organization adds the amount of time and resources needed to manually smoke test and regression test the features that have been implemented to the new functionality in each release. The overall new release is initially tested and debugged in order to assure the quality of the new features. Then we add the new test cases to the list of regression testing necessary in the current and future releases.
It is ever-increasing. The quality process strains our resources and impacts the schedule of each new project. Eventually project teams set priorities that typically reduce the regression testing in favor of focusing on completing the new features within a project schedule. That strategy leaves undiscovered defects introduced into existing functionality by new code development. Hopefully our users won't have a negative perception of our product and service quality. Who wants to build a business on luck?
For the past 10 years I have called for repurposing tests. Repurposing treats test scripts as software. We manage software as an organization's asset by storing it in a version control system and deploying it as a functional test, load and performance test, and production monitor. Repurposing comes with a cost: an organization needs to make a decision to build the test and maintain it.
Some test projects have chosen a test tool and used it for creating test scripts for that project. When an organization practices a "test and trash" methodology, they do not re-use them. This technique leaves little means to justify the cost of their initial creation. Worse, when the test script developer leaves, on-going support from others suffers as they learn the tool and script language to maintain the tests. Test automation is an investment that must be supported to have a return over time.
What is the Automated Regession Test Solution?Organizations succeed with automated regression testing when an organization does the following:
- Takes an inventory of the workflow present in the application
- Transforms the workflow into test scripts
- Maintains and version-controls the test scripts as software assets
- Automates the operation of test scripts based on changes to the application
- Uses the test results as inputs to management's business risk analysis
- Introductions, roles, and responsibilities
- Walk through the application(s)
- Testing requirements definition - use cases, workflow, operational test data
- Results analysis and publication - How to disseminate actionable knowledge
- Operational test data requirements
- Test operation infrastructure requirements
- Test Authoring, Test Orchestration and Test Management overview
- Identify process and next steps
- Test Plan
- Methodology Plan
- Proof of Concept (POC) plan
- Test Architecture Criteria and Recommendations
- Project Plan and Timing