Original text: http://www.cnblogs.com/Chilam007/p/6445165.html

Introduction to analysis

The analyzer is a component that analyzes the test result data. It is one of the three major components of LR. It saves a large number of data graphs used to analyze the performance test results. However, it is not necessary to analyze each view. Select relevant data views for analysis, and the analysis results can generate test reports in different formats.

1. Setting options

How are the data in the analysis obtained? In fact, when the scenario is running, by default, all the vuser information is saved on the load machine of the vusr. These data will be automatically sorted or merged only after the scenario runs, and all the information and data of the vusers on the load machine will be transferred to the result directory. By default, in the controller controller, results->auto collate results is selected, which means that by default, when the scene runs, the data will be automatically collated or merged. But when this item is not selected, after the scene runs, you can select results->collate results in the controller for manual sorting.

Before performing data view analysis, it is necessary to set some options involved in the analysis diagram to filter out unnecessary data and facilitate data analysis.

Here are some common setting options:

1. Result directory settings

Select results->results setting in the controller controller, as shown in Figure 1:

a. A result file is generated every time the scene is executed. The result file is named res followed by a sequence number (such as res0), and the sequence number is incremented by 1 each time it is executed;

b. Every time a scenario is executed, the result after execution will overwrite the previous result. (In order to prevent accidents, the first result saving method is generally selected).

Figure 1 (Results Directory Setup)

2, result collection settings

When LR processes these data, you can view the summary of these data. How to view the summary data needs to be set in the result collection option, as shown in Figure 2:

figure 2

data source：

a. Indicates viewing only summary data. If checked, analysis also processes data for advanced purposes such as filtering and grouping. And the data aggregation item cannot be set.

b. Indicates that only full processed data is viewed, but summary data is not displayed.

c. Indicates that the summary data is viewed while the full data is being processed.

data aggregation：

a. Indicates that data is aggregated using built-in data aggregation formulas to optimize performance

b. Indicates that only web-related data is aggregated using the built-in data aggregation formula

c. Indicates user-defined to set aggregated data, as shown in Figure 3:

aggregate data (available only for complete data): used to set the type of aggregated data. This is only applicable to complete data. You can select the type of graph that requires aggregated data, which can reduce the capacity of the database.

select the graph properties to aggregate: Specifies the graph properties to aggregate.

web data aggregation only: Indicates that only the data of the web is aggregated, and the granularity of the aggregation can be set here.

image 3

3、configure measurements 设置

When analyzing the graph, it is found that the amplitude of the Y-axis of the analysis graph is too small, you can set it here, and perform an appropriate zoom-in or zoom-out operation on the Y-axis, as shown in Figure 4:

a. Manually set a scale value; b. Use the optimized automatic scale to display each metric in the graph; c. Set all metric scales in the graph to 1; d. View the trend of all metrics.

The default value is 1, that is, it is displayed at the original scale. Sometimes the amplitude of the waveform display is too small, so the zoom ratio needs to be adjusted appropriately.

Figure 4

4. Set filter conditions

As shown in Figure 5: In the process of recording the test script, if the think time is not ignored when executing the script, the analysis analyzer will include the think time in the statistical analysis results, so that when the think time exists at the beginning of the user transaction and the Between the end, the related transaction statistics will be affected. Therefore, in many cases, it is necessary to filter the user's thinking time, just delete the include think time option in the drop-down box, and the thinking time will be automatically filtered out in the analysis results.

Figure 5

Second, the analysis diagram

The analysis analyzer provides rich analysis graphs, as shown in Figure 6:

There are 8 common types:

1) vuses graph: During the execution of the scheme, vusers generate data when executing transactions. Use the vusers graph to determine the overall behavior of vusers during scenario execution. It shows the vuser status and the number of vusers that completed the script. Mainly includes running vuser graph and vuser summary graph

2) Error map: Error information when the main statistical scheme is executed. It mainly includes four kinds of graphs: error statistics (by description), error per second (by description), error statistics and error per second.

3) Transaction graph: Describes the transaction performance and status during the entire script execution process. Mainly include the average transaction response time graph, the number of transactions per second graph, the total number of transactions per second, the transaction summary graph, the transaction performance summary graph, the transaction response time (under load) graph, the transaction response time (percentage) graph, and the transaction response time (distribution) graph. )picture.

4) Web resource graph: It mainly provides some information about the performance of the web server. Use the web resource graph to analyze the number of hits per second, server throughput, HTTP status codes returned from the server, HTTP responses per second, page downloads per second, server retries per second, and summary of server retries during a scenario run , connections, and connections per second.

5) Web page segmentation graph: It mainly provides some information to evaluate whether the page affects the transaction response time. Includes page breakdown, page component breakdown, page component breakdown (over time), page download time breakdown, page download time breakdown (over time), and downloaded components graph

6) System resource graph: It mainly monitors the usage of system resources during the operation of the scenario. Can monitor windows resources, UNIX resources, SNMP resources, Antara Flame Thrower resources and SiteScope resources

7) Web server resource graph: It is mainly used to capture the information of the web server when the scene is running. Mainly used to analyze microsoft IIS server, apache server, iplanet/netscape server and iplanet (SNMP) server.

8) Database server resource graph: It mainly displays the statistical information of the database server. Currently supports DB2, oracle, sql server and sybase databases.

Image 6

Summary report

1. Summary part

2. Statistics part

3. Transaction Statistics Section

The first line counts the number of all transactions passed, failed, and stopped when the scenario is running. The table shows the details of all transactions executed:

1) transaction name (transaction name); 2) minimum (the shortest time for the transaction to run); 3) average (the average time for the transaction to run); 4) maximum (the longest time for the transaction to run);

5) std.deviation (standard deviation): The variance describes the deviation of a group of data from its average value. The larger the variance value, the more discrete the group of data, and the stronger the volatility; otherwise, it indicates that the group of data is more discrete The more aggregated, the less volatility;

6) 90 percent: This value is not displayed when the controller runs the scene, because it is the result of statistics for the entire series of data. Indicates the time spent in 90% of the execution of a transaction. For example, if a transaction is executed 100 times, sort the response time of these 100 transactions in ascending order, and the 90% is the 90th transaction running time;

7) pass (number of transactions passed); 8) fail (number of transactions that failed); 9) stop (number of transactions stopped): When executing a scenario, if the user manually stops the execution of the scenario, the transaction does not have its own state, then it is the stop state.

Note: The transaction pass rate must be greater than 95%, that is, the failure rate should be less than 5%, because if the transaction failure rate is too high, it means that customers are prone to errors when using the system, so no matter how short the transaction response time is If it meets the requirements, because the most basic needs of customers have not been met, and the functions cannot be handled correctly, it is impossible to talk about performance.

4. SLA

SLA (service level agreement) can define performance testing goals and measure performance during the performance testing process. During the performance testing process, the LR will collect and save performance-related data. When analyzing the running results, the analyzer points The collected data is compared with the metric data defined in the SLA, and the analysis results are displayed in the analyzer. The three states of the SLA are:

a.pass: Indicates that SLA obtains the test data, and the data meets the target requirements; b.fail: Indicates that SLA obtains the test data, but the test results do not meet the target requirements; c.no data: Indicates that the SLA did not obtain the test data Test data, so there is no way to tell if it passed or failed.

The SLA configuration steps are as follows:

1. Click the button shown in Figure 7 in the summary view:

Figure 7

2. Click new to define the SLA target, as shown in Figure 8:

Figure 8

3. Set the target to be measured. Here, the transaction response time is taken as an example, as shown in Figure 9.

There are two ways to target the transaction response time, one is to measure it as a percentage (that is, what percentage of the transaction response time cannot exceed the target time); the other is to measure the average transaction response time. Let's introduce these two methods in turn.

Figure 9

4. Select the transaction. (Note: The transaction must be inserted in the script, otherwise the transaction to be measured cannot be selected when selecting the transaction in this step), as shown in Figure 10:

Figure 10

5. Set the percentage threshold. If the transaction response time is measured in percentage mode, as shown in Figure 11:

Figure 11

In this step, you need to set the percentage and transaction response time threshold. The set percentage is 90%, and the transaction response time is 2s, that is, as long as 90% of the transaction response time does not exceed 2s, then the SLA report result is PASS, otherwise the result is FAIL. As shown in Figure 12.

Figure 12

In percentage mode, you can enter the analysis transaction interface on the analyze transaction button to view detailed analysis information, as shown in Figure 13:

Figure 13

The following is a measure of the average transaction response time:

1. Set the load standard. If you choose to measure by average transaction response time, as shown in Figure 14:

Select the load standard, that is, what indicator is used to measure the change of transaction response. Taking the number of running virtual users as an example, it is necessary to set the response time of the transaction under different numbers of running virtual users.

Figure 14

2. Set the threshold.

After selecting the load standard, you need to set the transaction response time under different load standard conditions. Here, you need to set the transaction response time under different running virtual users, as shown in Figure 15.

It is set that when the number of virtual users is less than 10, the transaction response time should not exceed 1s, and when the number of virtual users is greater than 10, the transaction response time should not exceed 1.5s.

Figure 15

The settings are all completed here. It can be seen that SLA is essentially a goal and a means of measuring whether the test results achieve the goal. It is very similar to the setting of the target scene, and the principle is almost the same.

After completing the SLA settings, the analyzer will display the results of each metric transaction in different time domains, as shown in Figure 16:

Figure 16

Here you can select different transactions and different time domains for detailed analysis, take viewing air ticket information as an example for analysis, click the analyze transaction button, the analyzer will display the detailed information of the transaction, and the detailed analysis information mainly includes transaction summary information, transaction Correlation, error messages and snapshot views.

1) Transaction summary information

2) Transaction-related information (mainly including related information that may need to be associated when analyzing transactions: some error information when the script is running, system resource consumption, web resource consumption, and database resource consumption.)

Note: My report only shows the consumption of web resources, in fact, there are several other kinds of information mentioned above.

3) Error information (mainly displays the error information that occurs during the entire scenario running process, which is similar to the error output information generated during the scenario running process. It records the error type, error code, transaction name, script, and error code line in detail. number, which virtual user made an error during the running process, etc.).

Note: Because there is no error during the running of my script, there is no error message displayed in the report, but you can check it yourself.

4) Snapshot view (mainly describing the transaction response time in the analyzed time domain), as shown in Figure 17.

The abscissa represents the execution time of the scenario, and the ordinate represents the transaction response time. There are three curves in the figure. The red one represents the number of virtual users when the scenario is running, the green one is the transaction response time when the scenario is running, and the black one represents the threshold defined by the SLA. If the green line exceeds the black line, the SLA at that point has failed, and the SLA status will be set to failed. Otherwise, it is successful, and the status of the SLA will be set to passed.

Figure 17

Note: The display of results in percentage mode and average mode will be slightly different, and you can operate and analyze by yourself.

Five, HTTP response statistics

HTTP is a communication protocol that allows Hypertext Markup Language (HTML) documents to be transferred from a web server to a web browser. HTML is a markup language used to create documents that contain links to related information. You can click a link to access other documents, images, or multimedia objects and obtain additional information about the linked item. (About the meaning of HTTP request response mechanism and HTTP response status code, you can Baidu by yourself, so I won't talk about it here.)

Analysis common graph analysis

The resource usage is rarely analyzed in the LR analyzer, because LR is rarely used to monitor system resource usage during performance testing, especially UNIX, LINUX and ALX operating systems, LR is rarely used to monitor, More monitoring is with the help of third-party tools. Of course, if the server is a Windows operating system, it is relatively simple to use LR for monitoring.

1. vuser diagram

It shows the vuser status and the number of vusers that completed the script. Use these graphs in conjunction with transaction graphs to determine the impact the number of vusers has on transaction response times.

The X-axis represents the elapsed time since the scenario started running, the Y-axis represents the number of vusers in the scenario, and the vuser graph shows the number of vusers executing vuser scripts and their status in each second during the test. Can help determine the vuser load on a server in any given environment. By default, this graph only shows running vusers.

2. Click-through rate graph

Displays the number of HTTP requests submitted by vusers to the web server per second during the scenario running. Use this graph to evaluate vuser generation and load in terms of hits. This graph is typically viewed alongside the Average Transaction Response Time graph to observe the impact of hits on transaction performance . The X-axis represents the time the program has taken since it started running, and the Y-axis represents the number of hits on the server.

Note: The click rate does not measure the real processing power of the server, nor can it measure the processing power of the server only by the click rate, because even if the server has a bottleneck, it will still affect the change of this value, because LR is actually recorded by an agent The tool records the requests submitted during the recording process into scripts, and simulates the user to resubmit these requests during playback. Then LR can count HTTP requests at the time of submission, and then generate a click-through rate view. But this does not mean that the click rate view drawn by LR is necessarily correct. If the actual HTTP request submitted by the client is 2000/s, but the value drawn by the click rate view is 1000/s, this indicates that the request submitted by the client It is not completely sent to the server at all, so this situation is most likely a timeout of the request at the gateway, because each gateway port has a maximum value that it is allowed to access, and when this value is too large, the gateway will also queue up Phenomenon, if the queue is too long, some requests will be timed out, and finally the value of the click rate will be incorrect.

3. Average transaction response time graph

Displays the average time the plan took to execute transactions during its run. The X-axis represents the elapsed time since the scenario started running, and the Y-axis represents the average time (s) it took to execute each transaction. The average transaction response time most directly reflects the performance of the transaction. Generally, the average transaction response time graph is compared with the vuser graph to observe the impact of vuser operation on transaction performance. You can right-click and select show transaction breakdown tree to view the time spent on each page of sub-transactions or all transactions.

The average transaction response time graph directly reflects the performance of the system, which is also the performance in the eyes of customers. When necessary, the response time of the business must be clearly defined. When analyzing, the response time is generally analyzed first. When the average transaction response time meets the definition , it only means that the response time can meet the requirements, but this does not mean that the system meets the customer requirements, because the transaction response time calculated by LR is not necessarily correct, so when the transaction response time meets the requirements, it is necessary to analyze some other data. , what needs to be determined is whether the business has been successfully completed, if the business has been successfully completed, and the transaction response time meets the requirements, so as to indicate that the transaction response time meets the customer's requirements; if the average transaction response time fails to meet the requirements, it is necessary to further Analyze what causes the transaction response time to be too long, so as to further optimize the performance of the system.

4. Throughput graph

Displays the throughput per second on the server during the scenario run. Throughput is measured in bytes and represents the amount of data a vuser can get from the server in one second. With this graph, you can evaluate the load generated by vusers based on server throughput. You can compare it with the average transaction response time graph to see the impact of throughput on transaction performance.

The X-axis represents the elapsed time since the scenario started running, and the Y-axis represents the server's throughput in bytes. Throughput directly reflects the processing capacity of the server. If the value of the throughput processed by the server is larger, it indicates that the server has a stronger ability to process services. The value of throughput can be found only after two tests, that is, the inflection point of throughput must be found during the test process, so as to find the maximum throughput of the server when processing services, that is, the maximum processing capacity of the server.

Analysis in LoadRunner 11