Thursday, November 6, 2008

LOOKUP FOR PERFORMANCE TESTING

Performance Testing is done to investigate the behavior of your application/product at various workloads over duration of time. Performance Testing aims at identifying the bottlenecks of components (For e.g. servers, network, hardware etc.) involved in your application/product. Analyzing the performance of your application should be rendered before deploying to production. Verbalizing about analysis doesn’t need any effort. Instead, committing to performance testing and producing the analysis report, which replicates the actual behavior of the application requires an effort. In the process of the measuring the performance of the application, it is necessary to evaluate the performance of application under normal load, under anticipated future load, under highly abnormal peak load and also under uninterrupted, sustained load. Innumerable factors influence the performance of the application. Primary concerns in the performance of the application may include Response Times, Request/Response rate and Data Transfer rate.

More often, performance testing is started without investigating pre-requisites. It is expedient, to have extensive information about the architecture of your application/product, bandwidth of network, network topology, volume handled by your application/product, optimal hardware for your application/product and benchmarking you product. Also, ensure that the testing environment should replicate the anticipated live system in all the respects including the hardware. Generally, after investing lot of efforts to analyze the performance, testers realize that test environment doesn’t replicate the live system either in architecture or in network or in hardware ending up in wasting their time, effort and resources.
The following needs to be investigated in depth before originating the performance testing.
1. Which would be the best tool for your application?
Perpetually, automated test tools are used to evaluate the performance of your application/product. Hundreds of tools are available in the market having their own technical limitations. A careful effort has to go into deciding which tool would be more suitable for analyzing the performance of you application/product. Ensure the tool of your selection is compatible with the communication protocol of your application, compatible with the servers involved in your applications architecture. The tool should be able to monitor servers to evaluate the performance. More often, all the tools will give the basic parameters (like response times and transaction times) that are required to analyze the performance of your application/product.
2. What is the type of recording to be used?
Depending on the architecture of the application and network topology recording the traffic can be done in 3 methods.
¨ API Recording should be done when you want to record the calls between the specific client application and the server. It is recommended if you are accessing secure data from a web server, because this method captures the information before it reaches the wire and is encrypted. It can record requests from multiple clients simultaneously, as long as single application process is issuing the requests.
¨ Network Recording should be done to record the TCP/IP traffic on the LAN using the “promiscuous” mode of your computer’s Ethernet adapter. This method is recommended when the client is running on Non-Windows platform.
¨ Proxy Recording should be done to record the raw IP packets off the wire. This method lets you capture IP packets that may not be visible during the network recording.
3. For how many users (virtual user simulation)?
Before inaugurating the performance scripts recording, presumptions should be made in the number of users that will be operating at an instant of time on your postproduction application. Fairly, the same number or a few more should be considered as the number of virtual users to analyze the performance of your application.
4. What are the parameters required?
As parameters play a vital role in the analysis of performance testing, the tester should decide much in advance about the parameters he requires. Basic parameters include response times, request/response rate, data transfer rate, concurrent connections etc.
5. What is the volume of your application?
Volume of the servers is one of the significant concerns in performance testing. Ensure that the volume of the servers in the test environment is same as the anticipated volume under production. If the volume of the servers’ enlarge over a period of time, benchmark the anticipated volume with certain time intervals. After benchmarking, evaluate the performance of application under normal volume, under anticipated future volumes and under highly abnormal peak volumes benchmarked. Also, you have to think about the accessibility of this volume under unpredictable market conditions.
6. Whether to monitor the server resources?
It’s always suggested to monitors all the servers (database servers, application servers, web servers etc.) involved in the architecture of your application. Frequently, it has been observed that the bottlenecks are in the servers involved of your application or in your network. The tool you are using should support monitoring the servers of your application. Drilling down to the results acquired by the tool can identify the bottlenecks. After identifying, tune the application to boost the performance of your application/product.
7. Whether to monitor the network resources?
One of the important components to boost the performance is network topology and its bandwidth. The bandwidth should be made as high as possible, which results in the better performance of your application/product. Monitoring the network is recommended for your application if no bottlenecks are found in the servers.
8. Whether to monitor the hardware resources?
There might be a bottleneck in the hardware resources of your application. The CPU cycles and memory might degrade the performance of your application if the hardware resources are inefficient. Monitor the hardware resources if the servers and the network are found efficient.
9. What is the optimal hardware for the application?
Identify the best hardware that can be used to run your application, which is free from the bottlenecks like low memory, less number of CPU Cycles etc. Identifying and defining the optimal hardware for the application is also equally important as part of this activity.

The other concern is Think Time. In performance testing, the think time should be ignored. The script should perform the action immediately without any delay as soon as the pre-requisite is satisfied. For example, if the action to be performed is “click on the link” of Home Page to navigate to another page – the prerequisite will be the complete download of Home Page.

For every important functional action, a transaction should be defined to obtain the resultant parameters for this transaction. The transactions should be modular & independent so that a series of transaction calls will add one to your scripts count.

After the completion of recording the script (which should be a series of transactions), ensure that the script executes for a single user. After the successful execution for one user then try to customize the script to the extent possible so that the transactions are dynamic at the server end. Prior to the execution of the workloads the maximum number of users that can operate on your application should be benchmarked. Decide on the workload that you want to set for multiple users. Workload/Scenario is the behavior of the users during runtime of the script. Various combinations of workloads should be covered as a process of performance testing. Few are mentioned below which are the basic scenarios/workloads
¨ to queue the user(s) one after the other to operate on the scenario
¨ to force in the minimum, average and maximum number of virtual users at the beginning of the scenario and permitting them to run till the end of the scenario
¨ to increase a consistent number of virtual users at periodic intervals of time till the maximum users operate on the scenario
¨ to force the users in a irregular manner to operate on the scenario without maintaining consistency in both the number of users and time intervals

These scenarios/workloads should be executed for a fixed amount of time, till the completion of the scenario, for an ample time and for the maximum time possible (which should be in the multiples of 12 hours). During the process of running the above workloads it is recommended to balance the load across the network. The load can be balanced by distributing the run time users to use the resources of other machines that are involved in the network. For many tools to balance the load, installation of the tool agents on the machines you want to utilize as a resource is a pre-requisite. After the successful execution of the scenario, the tool extracts the basic parameters and the parameters requested by you. It gives you in the form of a report (with graphs for some tools). The analysis part of performance testing should be carried out according to your experiences. The report generated by the tool is very generic and should be taken as a base line but not as a performance report.

1 comment:

Ananth said...

Helpful post. Surely a reference to decide on the performance testing tool.