Performance Testing in Agile Environment
Overview
This article will help you gain a closer understanding of how you can integrate performance testing in Agile processes. Performance testing is an essential activity in all software development projects, including Agile ones. From initial planning to production analysis, application performance drives the development of better software iterations and releases. Application owners, stakeholders, developers, and testers should make performance a key consideration in every sprint, or iteration, in Agile development.
Introduction to the Approach
The approach starts by examining the software development project as a whole, the reasons why stakeholders have chosen to include performance testing in the project, and the value that performance testing is expected to add to the project. The Agile approach breaks through the barriers of conventional waterfall approaches to software development to deliver business value sooner and accelerate return on investment (ROI).
Performance testing is an integral part of Agile processes, it can help your organization develop higher quality software in less time while reducing development costs. The goal is to test performance early and often in the development effort, and to test functionality and performance in the same sprint. Performance testing is taken into consideration throughout the Agile SDLC—from the release planning stage onward.
Key considerations in Agile performance testing
In an Agile environment, it’s important for application owners to see continual improvement in an application over the course of successive sprints. They want to see a favorable trend, where each iteration of the application is better than the last. This makes it all the more important to monitor application performance trends in terms of SLO requirements. Trending reports allows you to give stakeholders regular snapshots of performance, which should ideally show that performance is getting steadily better or at least is not degrading. In addition, by looking at trending reports you do not necessarily have to study the analysis on every test run.
Agile Performance-Testing Activities
This approach can be represented by using the following nine activities.
Activity 1. Understand the Project Vision and Context
The project vision and context are the foundation for determining what performance testing is necessary and valuable. Before initiating performance testing, ensure that you understand the current project vision. Because the features, implementation, architecture, timeline, and environments are likely to change over time, you should revisit the vision regularly, as it has the potential to change as well.
Activity 2. Identify Reasons for Testing Performance
Identifying the reasons for performance testing is critical to being able to determine what performance-testing activities will add the most value to the project. Every project has different reasons for deciding to include, or not include, performance testing as part of its process. The possible reasons to make integrated performance testing a part of the project includes, Monitor resource usage trends, Measure response times, Collect data for scalability and capacity planning.
Activity 3. Identify the Value Performance Testing Adds to the Project
Strategies are intended to help focus decisions, be readily available to the entire team, include a method for anyone to make notes or comments, and be easy to modify as the project progresses.
In general, the types of information that may be valuable to discuss with the team when preparing a performance-testing strategy for a performance build includes,external resources required, risks to accomplishing the strategy, areas of concern, pass/fail criteria, completion criteria, planned variants on tests, load range.
Activity 4. Configure the Test Environment
With a conceptual strategy in place, prepare the necessary tools and resources to execute the strategy as features and components become available for test. Load-generation and application-monitoring tools are almost never as easy to implement as one expects.
Consider the following key points when configuring the test environment:
Determine how much load you can generate before the load generators reach a bottleneck.
Validate the accuracy of load test execution related to server clusters in load-balanced configuration.
Monitor resource utilization (CPU, network, memory, disk and transactions per time) across servers in the load-balanced configuration during a load test to validate that the load is distributed.
Activity 5. Identify and Coordinate Tasks
Performance test execution plans should be communicated to the team and stakeholders by using the same method(s) the strategy uses. Depending on the pace and schedule of project, there may be one execution plan per strategy, or several.
Activity 6. Execute Task(s)
When each performance build is delivered, the performance testing begins with the highest-priority task related to that build.
In general, the keys to conducting a performance-testing task includes analyze results immediately and revise the plan accordingly, record results and significant findings, record other data needed to repeat the test later.
Test execution can be viewed as a combination of the following sub-tasks:
Activity 7. Analyze Results and Report
One of the jobs of the performance specialist is to find trends and patterns in the data, which takes time. This task also tends to lead to the desire to re-execute one or more tests
Before results can be reported, the data must be analyzed. Consider the following important points when analyzing the data:
Activity 8. Consider Performance Acceptance Criteria
It generally makes sense to start identifying, or at least estimating, the desired performance characteristics of the application early in the development life cycle. This can be accomplished most simply by noting the performance characteristics that users and stakeholders equate with good performance.
Classes of characteristics that frequently correlate to a user’s or stakeholder’s satisfaction typically includes Response time, Throughput, Resource utilization.
Activity 9. Reprioritize Tasks
Based on the test results, new information, and the availability of features and components, reprioritize, add to, or delete tasks from the strategy. Some agile teams conduct periodic “performance-only” scrums or stand-ups when performance testing–related coordination, reporting, or analysis is too time-consuming to be handled in the existing update structure.
Summary
Performance testing in an agile project environment allows you to manage the testing in a highly flexible way. In particular, this approach allows you to revisit the project vision and reprioritize tasks based on the value they add to the performance test at a given point in time.
Overview
This article will help you gain a closer understanding of how you can integrate performance testing in Agile processes. Performance testing is an essential activity in all software development projects, including Agile ones. From initial planning to production analysis, application performance drives the development of better software iterations and releases. Application owners, stakeholders, developers, and testers should make performance a key consideration in every sprint, or iteration, in Agile development.
Introduction to the Approach
The approach starts by examining the software development project as a whole, the reasons why stakeholders have chosen to include performance testing in the project, and the value that performance testing is expected to add to the project. The Agile approach breaks through the barriers of conventional waterfall approaches to software development to deliver business value sooner and accelerate return on investment (ROI).
Performance testing is an integral part of Agile processes, it can help your organization develop higher quality software in less time while reducing development costs. The goal is to test performance early and often in the development effort, and to test functionality and performance in the same sprint. Performance testing is taken into consideration throughout the Agile SDLC—from the release planning stage onward.
Key considerations in Agile performance testing
In an Agile environment, it’s important for application owners to see continual improvement in an application over the course of successive sprints. They want to see a favorable trend, where each iteration of the application is better than the last. This makes it all the more important to monitor application performance trends in terms of SLO requirements. Trending reports allows you to give stakeholders regular snapshots of performance, which should ideally show that performance is getting steadily better or at least is not degrading. In addition, by looking at trending reports you do not necessarily have to study the analysis on every test run.
Agile Performance-Testing Activities
This approach can be represented by using the following nine activities.
Activity 1. Understand the Project Vision and Context
The project vision and context are the foundation for determining what performance testing is necessary and valuable. Before initiating performance testing, ensure that you understand the current project vision. Because the features, implementation, architecture, timeline, and environments are likely to change over time, you should revisit the vision regularly, as it has the potential to change as well.
Activity 2. Identify Reasons for Testing Performance
Identifying the reasons for performance testing is critical to being able to determine what performance-testing activities will add the most value to the project. Every project has different reasons for deciding to include, or not include, performance testing as part of its process. The possible reasons to make integrated performance testing a part of the project includes, Monitor resource usage trends, Measure response times, Collect data for scalability and capacity planning.
Activity 3. Identify the Value Performance Testing Adds to the Project
Strategies are intended to help focus decisions, be readily available to the entire team, include a method for anyone to make notes or comments, and be easy to modify as the project progresses.
In general, the types of information that may be valuable to discuss with the team when preparing a performance-testing strategy for a performance build includes,external resources required, risks to accomplishing the strategy, areas of concern, pass/fail criteria, completion criteria, planned variants on tests, load range.
Activity 4. Configure the Test Environment
With a conceptual strategy in place, prepare the necessary tools and resources to execute the strategy as features and components become available for test. Load-generation and application-monitoring tools are almost never as easy to implement as one expects.
Consider the following key points when configuring the test environment:
Determine how much load you can generate before the load generators reach a bottleneck.
Validate the accuracy of load test execution related to server clusters in load-balanced configuration.
Monitor resource utilization (CPU, network, memory, disk and transactions per time) across servers in the load-balanced configuration during a load test to validate that the load is distributed.
Activity 5. Identify and Coordinate Tasks
Performance test execution plans should be communicated to the team and stakeholders by using the same method(s) the strategy uses. Depending on the pace and schedule of project, there may be one execution plan per strategy, or several.
Activity 6. Execute Task(s)
When each performance build is delivered, the performance testing begins with the highest-priority task related to that build.
In general, the keys to conducting a performance-testing task includes analyze results immediately and revise the plan accordingly, record results and significant findings, record other data needed to repeat the test later.
Test execution can be viewed as a combination of the following sub-tasks:
- Coordinate test execution and monitoring with the team.
- Validate tests, configurations, and the state of the environments and data.
- While the test is running, monitor and validate scripts, systems, and data.
- Log start and end times, the name of the result data, and so on.
Activity 7. Analyze Results and Report
One of the jobs of the performance specialist is to find trends and patterns in the data, which takes time. This task also tends to lead to the desire to re-execute one or more tests
Before results can be reported, the data must be analyzed. Consider the following important points when analyzing the data:
- Analyze the captured data and compare the results against the metric’s acceptable or expected level to determine whether the performance of the application being tested shows a trend toward or away from the performance objectives.
- If the test fails, a diagnosis and tuning activity are generally warranted.
- If you fix any bottlenecks, repeat the test to validate the fix.
- Modify the test to get new, better, or different information if the results do not represent what the test was defined to determine.
- Use current results to set priorities for the next test.
- Collecting metrics frequently produces very large volumes of data.
Activity 8. Consider Performance Acceptance Criteria
It generally makes sense to start identifying, or at least estimating, the desired performance characteristics of the application early in the development life cycle. This can be accomplished most simply by noting the performance characteristics that users and stakeholders equate with good performance.
Classes of characteristics that frequently correlate to a user’s or stakeholder’s satisfaction typically includes Response time, Throughput, Resource utilization.
Activity 9. Reprioritize Tasks
Based on the test results, new information, and the availability of features and components, reprioritize, add to, or delete tasks from the strategy. Some agile teams conduct periodic “performance-only” scrums or stand-ups when performance testing–related coordination, reporting, or analysis is too time-consuming to be handled in the existing update structure.
Summary
Performance testing in an agile project environment allows you to manage the testing in a highly flexible way. In particular, this approach allows you to revisit the project vision and reprioritize tasks based on the value they add to the performance test at a given point in time.
0 comments:
Post a Comment