1. What is software performance and performance testing
Definition : The performance of software is a non-functional characteristic of software. It is not concerned with whether the software can complete a specific function, but the timeliness displayed when the function is completed.
It can be seen from the definition that performance focuses on the non-functional characteristics of the software, so generally speaking, the time to intervene in performance testing is after the completion of functional testing. In addition, the timeliness in the definition shows that performance is also an indicator, which can be measured by time or other indicators. Usually, we use certain tools or means to detect whether certain indicators of the software meet the requirements, which is performance testing.
Definition of performance testing : refers to testing various performance indicators of the system by simulating various normal, peak and abnormal load conditions through automated testing tools.
2. Types of Performance Tests
- Benchmark test: When applying low pressure to the system, check the operating status of the system and record the relevant numbers as a basic reference
- Load test: refers to continuously increasing the pressure on the system or increasing the duration under a certain pressure until one or more items of the system
- The performance index reaches a safety critical value, for example, a certain resource has reached a saturated state.
- Stress testing: Stress testing is to evaluate the operation of the system when the system is at or above the expected load. The focus is on the processing capacity of the system under peak load or beyond the maximum load.
- Stability test: When a certain business pressure is applied to the system, the system is run for a period of time to check whether the system is stable.
- Concurrency test: Test whether there are deadlocks or other performance problems when multiple users access the same application, the same module or data records at the same time,
3. Basic concepts of performance testing
1. Response time
a) Definition:
The response time is the time from when the user sends a request to when the user receives the response data returned by the server.
b) Critical path:
The figure below shows the path of an http request. The request will be sent to the web server through the network for processing. If you need to operate the DB, it will be forwarded to the database by the network for processing, and then return the value to the web server. The web server will finally pass the result data through the network. returned to the client.
c) Calculation method:
As shown in the figure, the client’s request response time = (N1 + N2 + N3 + N4 + N5 + N6) + (A1 + A2 + A3 + A4 + A5 + A6), namely: (network time + application processing time)
d) Response time-load correspondence:
Inflection point description in the figure:
1. Sudden increase in response time
2. It means the limit reached by one or more resources of the system
3. Inflection points can usually be used for performance test analysis and positioning
2. Throughput
a) Definition:
The number of client requests processed by the system per unit time
b) Calculation unit:
Generally, the number of requests/second is used as the unit of throughput, and it can be expressed by the number of pages/stopwatch.
In addition, from a business point of view, the number of visitors/day or the number of page views/day can also be used as the unit.
c) Calculation method:
Throughput = (number of requests) / (total time).
d) Throughput-load correspondence:
Inflection point description in the figure:
1. Throughput gradually reaches saturation
2. It means the limit reached by one or more resources of the system
3. Inflection points can usually be used for performance test analysis and positioning
3. Concurrent number:
-
Number of concurrent users: The number of users who submit requests to the system at the same time at a certain physical moment. The submitted requests may be for the same scenario or function, or different scenarios or functions.
-
Number of online users: the number of users accessing the system within a certain period of time, these users do not necessarily submit requests to the system at the same time
-
Number of system users: total user data registered in the system
The relationship between the three: the number of system users >= the number of online users >= the number of concurrent users
4. Resource utilization
a) Definition:
Refers to the degree of usage of different system resources, usually measured as a percentage of the maximum occupied
b) Server resources that usually require attention are as follows:
1. CPU: Just like the human brain, it is mainly responsible for the judgment of related matters and the actual processing mechanism
2. Memory: the memory block area in the brain, the place where the information collected by the eyes, skin, etc. is recorded for the CPU to judge, but it is temporary and the access speed is fast. If the computer is turned off or the power is turned off, the data here will disappear .
3. Disk IO: The memory block in the brain, which saves important data (permanently saved, will not be lost when turned off or powered off, and the speed is slow), so that these data can be used again in the future.
4. Network:
c) Resource Utilization-Load Correspondence:
Inflection point description in the figure:
1. The usage of a recommended resource on the server is gradually reaching saturation
2. Inflection points can usually be used for performance test analysis and positioning
5. Other commonly used concepts:
a) TPS：
Transactions Per Second, the number of transactions per second
b) Thinking time:
The pause time after each user operation, or the interval between operations, during which there is no pressure on the server
c) Number of clicks:
The number of HTTP requests submitted by users to the WEB server per second. This indicator is a unique indicator of WEB applications: WEB applications are in the "request-response" mode. Once a user sends an application, the server must process it once, so a click is the smallest unit of transaction that a WEB application can handle. If each click is defined as a transaction, click rate and TPS are a concept. It is easy to see that the greater the hit rate, the greater the pressure on the server. The click-through rate is just a performance reference indicator, it is important to analyze the impact of the click. It should be noted that the click here does not refer to a click operation of the mouse, because in one click operation, the client may send multiple HTTP requests to the server.
d）PV：
Visiting a URL generates a PV (Page View, page views). The total daily PV of each website is an important indicator to describe the size of a website.
UV：
As an independent user, all pages that visit the site are counted as a UV (Unique Visitor, user visit)