Best Practices in Regression Testing

Practice 1: Regression can be used for all types of releases

Regression testing can be applied when

We need to measure the quality of product between test cycles (both planned & need based);
We are doing a major release of a product, have executed all test cycles, and are planning a regression test cycle for defect fixes; and
We are doing a minor release of a product (support packs, patches, and so on) having only defect fixes, and we can plan for regression test cycles to take care of those defect fixes.
There can be multiple cycles of regression testing that can be planned for every release. This applies if defect fixes come in phase or to take care of some defect fixes not working with a specific build.

Automation Tool Selection Recommendation

“Automated Testing” means automating the manual testing process currently in use. This requires that a formalized “manual testing process” currently exists in the company or organization. Minimally, such a process includes:
–        Detailed test cases, including predictable “expected results”, which have been developed from Business Functional Specifications and Design documentation.
–        A standalone Test Environment, including a Test Database that is restorable to a known constant, such that the test cases are able to be repeated each time there are modifications made to the application.
Information Gathering
Following are sample questions asked to tester who have been using some the testing tools:
How long have you been using this tool and are you basically happy with it?
How many copies/licenses do you have and what hardware and software platforms are you using?
How did you evaluate and decide on this tool and which other tools did you consider before purchasing this tool?
How does the tool perform and are there any bottlenecks?
What is your impression of the vendor (commercial professionalism, on-going level of support, documentation and training)?
Tools and Vendors

• Robot – Rational Software
• WinRunner 7 – Mercury
• QA Run 4.7 – Compuware
• Visual Test – Rational Software
• Silk Test – Segue
• QA Wizard – Seapine Software

Tools Overview
Robot – Rational Software
–        IBM Rational Robot v2003 automates regression, functional and configuration testing for e-commerce, client/server and ERP applications. It’s used to test applications constructed in a wide variety of IDEs and languages, and ships with IBM Rational TestManager. Rational TestManager provides desktop management of all testing activities for all types of testing.
WinRunner 7 – Mercury
–        Mercury WinRunner is a powerful tool for enterprise wide functional and regression testing.
–        WinRunner captures, verifies, and replays user interactions automatically to identify defects and ensure that business processes work flawlessly upon deployment and remain reliable.
–        WinRunner allows you to reduce testing time by automating repetitive tasks and optimize testing efforts by covering diverse environments with a single testing tool.
QA Run 4.7 – Compuware
–        With QA Run, programmers get the automation capabilities they need to quickly and productively create and execute test scripts, verify tests and analyze test results.
–        Uses an object-oriented approach to automate test script generation, which can significantly increase the accuracy of testing in the time you have available.
Visual Test 6.5 – Rational Software
–        Based on the BASIC language and used to simulate user actions on a User Interface.
–        Is a powerful language providing support for pointers, remote procedure calls, working with advanced data types such as linked lists, open-ended hash tables, callback functions, and much more.
–        Is a host of utilities for querying an application to determine how to access it with Visual Test, screen capture/comparison, script executor, and scenario recorder.
Silk Test – Segue
–        Is an automated tool for testing the functionality of enterprise applications in any environment.
–        Designed for ease of use, Silk Test includes a host of productivity-boosting features that let both novice and expert users create functional tests quickly, execute them automatically and analyze results accurately.
–        In addition to validating the full functionality of an application prior to its initial release, users can easily evaluate the impact of new enhancements on existing functionality by simply reusing existing test casts.
QA Wizard – Seapine Software
–        Completely automates the functional regression testing of your applications and Web sites.
–        It’s an intelligent object-based solution that provides data-driven testing support for multiple data sources.
–        Uses scripting language that includes all of the features of a modern structured language, including flow control, subroutines, constants, conditionals, variables, assignment statements, functions, and more.
Evaluation Criteria
Record and Playback         Object Mapping
Web Testing Object              Identity Tool
Environment Support        Extensible Language
Cost                                            Integration
Ease of Use                             Image Testing
Database Tests                     Test/Error Recovery
Data Functions                    Object Tests
Support

3 = Basic  2 = Good  1 = Excellent

Tool Selection Recommendation

Tool evaluation and selection is a project in its own right.
It can take between 2 and 6 weeks. It will need team members, a budget, goals and timescales.
There will also be people issues i.e. “politics”.
Start by looking at your current situation
– Identify your problems
– Explore alternative solutions
– Realistic expectations from tool solutions
– Are you ready for tools?

Make a business case for the tool

–What are your current and future manual testing costs?
–What are initial and future automated testing costs?
–What return will you get on investment and when?

Identify candidate tools

– Identify constraints (economic, environmental, commercial, quality, political)
– Classify tool features into mandatory & desirable
– Evaluate features by asking questions to tool vendors
– Investigate tool experience by asking questions to other tool users Plan and schedule in-house demonstration by vendors
– Make the decision

Choose a test tool that best fits the testing requirements of your organization or company.

An “Automated Testing Handbook” is available from the Software Testing Institute (www.ondaweb.com/sti), which covers all of the major considerations involved in choosing the right test tool for your purposes.

Read More about Automation Tool

Test Consulting: How to Improve a QA Area

Is your client having difficulty to measure QA performance? Is your client undecided on what test strategy to follow for new application implementation? Is your client looking for testing tool recommendations or need to improve the usage of the existing ones?
Hexaware has a dedicated Test Consulting practice where testing experts are involved in providing test consulting services to clients. During the consulting engagements, Hexaware assesses and evaluates the Approach, People and Technology and fill in the gaps by bringing in our domain experience, best practices, frameworks, tools experience, etc. The consulting services also include tools selection, tools optimization and TCoE Creation and/or optimization.
If you want to learn more about test strategy, the next information will help you to execute a test strategy engagement.
Test Strategy Approach
The first step is to identify the problem(s) that the client is facing and define the strategy objectives. After that, I recommend to follow this approach to execute a test strategy project:
Assessment Areas
As part of the information gathering phase, we leverage its proprietary ATPTM (Approach, People, and Technology) methodology to focus on the right areas and meet the strategy objectives. Hexaware’s APT™ Methodology is the foundation of all of our QA service offerings, below is the description of each component:
• The Approach component is designed to lay the foundation for the processes that each client use as part of testing
• People is the component of the IT organization with focus on testing, Hexaware analyze the groups, roles and responsibilities involved in QA and testing
• The Technology component includes the use of QA and automation testing tools for efficiency to optimize technology and lower costs.

At the end of the project, we provide to the client the following component:
•Current State: An analysis of the current state with regard to the testing objectives
•Gaps: The gaps found between the current state and the best practices and the desired state
•Recommendations: Our recommendations to close the gaps and meet the organization objectives
•Implementation Road Map: A recommended path to follow in order to implement the recommendations.
As a result of the analysis phase, we show to our clients the current state in a quantitative graph. This graph evaluates all relevant aspects of an IT organization and prioritizes each category according to the testing objectives. One example of this graph is showed below.

This was an assessment provided as part of a test strategy we created for a leading bank in Mexico for a T24 product implementation. The benefits showed in this strategy was to reduce testing cycles by 30%, automate at least 50% of the manual test cases and have a defect free implementation using robust and repeatable testing processes.
Other examples of metrics commonly used as part of the strategy objectives are the increment of automation coverage by 30%, increase productivity by 25%, reduce overall testing cost by 15%, etc.
A test consulting practice is an area full of innovation, industry best practices and shared experiences. Now with Hexaware blogs all of us will be able to formally share our experience and our colleagues can leverage them for future assignments.

Read More about QA