With the popularity of the Internet and a huge user base, user experience is very important. Performance is an index, which is the compliance degree of the software system for timeliness. For a product, the timeliness of performance is measured in terms of response time and throughput, where the response time is the sum of an http complete request process time. For users, response time is the basic end-to-end user experience.

(1) Response time from the user's perspective:

• 1-3 seconds is an excellent performance

• 3-5 seconds, acceptable, intermediate performance

• More than 5 seconds, unacceptable

Response time = network time + application processing time, the response time is the sum of a http complete request process time, the sum of request and response time

(2) The utilization rate of the system from the perspective of operation and maintenance

cpu usage
memory usage
average load

When the cpu or memory usage is greater than 70%, there may be a slow access speed of the deployed program

Mysql and other database attention indicators

All data comes from the database:

1. The number of connections: the number of connections is leaked (the number of connections is used, but not released) [if the channel of the database is not released and occupied, the connection will not be connected, and the database will be paralyzed]

2. Read and write speed: IOPS

JVM (for java technology line): CPU and memory usage of Java applications

512M，CPU：1G

Memory overflow/memory leak: OOM

The maximum capacity of the system: including cpu, jvm

The maximum number of tasks processed by the system:

100 million, but the payment is made at the same time, 50 million can be paid at the same time, and the other 50 million can be processed in a queue. What is the queuing strategy? ----"

Set according to the priority of the thread. Each task has a priority, then the maximum priority is 10, the minimum is 1, and the default is 5

When the system resources are not tight, the priority is not particularly prominent, but when the system resources are particularly tight, then the one with the higher priority will be executed first

(3) Development Perspective Concerns

Response time = network time + application processing time, the response time is the sum of a http complete request process time, the sum of request and response time

The focus on development will be more comprehensive. After all, the code is written by programmers. The details can be summarized as follows:

• Response time for front and back interactions

• Parameter setting of middleware (how much memory is allocated, how many tasks are executed at most)

• Memory release leaks

• Leaked connections

• Are there any unreasonable memory usage patterns

• Whether there is an unreasonable thread synchronization method (A and B are two tasks, the task is a thread, and the task conflicts. Processing method: lock mechanism, B cannot operate when A operates)

• Whether there is unreasonable competition for resources in the system

• System Architecture & Code Structure

(4) WEB front-end focus on performance

The so-called front-end performance is currently one of the more popular technologies in performance testing. The specific points of concern are summarized as follows:

• Browser resource loading (HTML parsing, image resource loading, CSS file resource loading)

• Whether the optimization of the front-end caching technology is reasonable

• Time-consuming interaction between front-end and back-end

(5) Focus on performance from the testing perspective

The perspective that users focus on is full-stack. It is necessary to consider the product experience from the perspective of the user, and also monitor and pay attention to the perspective of operation and maintenance and development. Therefore, the specific job responsibilities of testing in performance testing can be summarized as follows:

• Design reasonable scenarios and test cases to verify the resource data of the system

• Verify that the architecture meets the requirements under high concurrency

• Provide architects and developers with reasonable value ranges for middleware configuration parameters

• Use technical means to monitor the system, DB, middleware, and full link monitoring to monitor system resource conditions

1. Performance testing terminology [interview]

1. Response time:

Response time = network time + application processing time. Response time is the sum of a complete http request process time, the sum of request and response time.

The time for an operation to complete, that is, the time for the server to return the response data to the client after the client sends a request to the server. It includes the time used for waiting and serving, and also includes the time used to return the result. Response time = network time + application processing time,

For example:

vue--->nginx :web server

django:app-serving

MySQL:data server

2. Number of concurrent users

The core of performance testing is to verify whether the current system can support the access of existing users, that is to say, how many users the system can bear to access the system at the same time period,

The number of concurrent users can be said: no matter from the perspective of business or the pressure on the server, it describes the clients who send requests to the client at the same time. Sometimes it is also called "concurrency test". This mainly reflects the maximum number of concurrent accesses that the server can sustain.

The core of performance testing is to verify how many people can access the system at the same time, throughput and response time

3.IOPS

This term is mainly applied to databases, that is, the number of input/output operations that occur per second, and is a measure of data transfer. It is used for reading and writing of disks. IOPS refers to the number of reads and writes per second, which is reflected on the hard disk and the database read and write speed.

4.TPS/QPS

A transaction refers to an operation or a combination of a group of operations. TPS counts the number of transactions processed per second, that is, the number of transactions that the system can process per second. QPS refers to the query rate per second, such as the measure of how much traffic the DB server handles within a specified time

5. Throughput

Throughput refers to the speed of operation, that is, the number of operations per second or the number of transactions per second. Or it can also be said that the number of client requests per unit time reflects how many requests the system can process per second

The difference between throughput and concurrency testing: 【Important】

Concurrency test: Send requests to the system at the same time at the same time

Throughput: the number of tasks that the system can handle within the time period

6. Delay

Delay is the time used to describe the waiting service in the operation, that is, the response time used to wait for the server to return to the client after the client sends a request

7. Utilization

For resources requested by a service, the usage rate describes how busy the resource is within a given time interval. For storage resources, usage refers to the consumed storage capacity.

8. Saturation

Refers to the alignment workload that a certain resource cannot satisfy the service

9. Bottleneck [important]

In system performance, a bottleneck is a resource that limits system performance. This is also a very core part of the performance testing process

10. Think Time

The English name of thinking time is Think Time, also known as sleep time. From a business perspective, it refers to the interval between each request when the user is operating.

2. Performance testing theory

(1) Understanding of resource scheduling:

crm: customer management system

The smallest unit of an operating system in a computer is a process, and the smallest unit of operation of a process is a thread. For example, DingTalk is a process, but it can do many things. It is a multi-threaded program. When a thread is executing, resources are allocated by the operating system. The operating system allocates resources through the time slicing of the operating system combined with the scheduling algorithm, and decides which thread will be executed first and which will not be executed first. In the scheduling process, the algorithm has a preemptive algorithm scheduling. According to the priority of tasks, resources with high priority are allocated first, and resources with low priority are allocated later. Such as Linux, Mac, java. The highest thread priority is 10, the lowest is 1, and the default is 5

Scheduling strategy:

At the operating system level, professionally speaking, the scheduler can be understood as: CPU time is allocated to active processes and threads, and a set of priority mechanisms are maintained, so that more important tasks can be executed faster, the scheduler Will track all ready-to-run processes. Scheduling policies (schedulers) can dynamically give priority to improving the performance of workloads.

Workload type:

• CPU-intensive: The application performs heavy calculations, usually runs for a long time, and consumes a large amount of CPU. Mainly refers to the calculation program, the program has a large amount of data to filter out specific data.

• IO-intensive: The application program performs I/O, does not perform much calculations, and will occupy a large amount of memory resources. The smallest granularity of the system is threads, which means that the most fine-grained system scheduling is the scheduling of threads. Mainly refers to the reading and writing program, the program has a large number of file Io reading and writing, SMS marketing coupon distribution.

(2) Waiting queue

In the program, the waiting queue will be involved. Whether it is synchronous interaction or asynchronous interaction, its maximum queue will be involved. The core idea of this design is to prevent the server from running without In the case of a queue, an avalanche will eventually lead to a paralysis of the server, so waiting in the queue is very core and very necessary. In the waiting queue, the test needs to obtain the following performance data:

1. What is the value of the queue setting? What is the maximum task that can be run?

2. It is necessary to test the queuing strategy mechanism, that is to say, simulate a large number of programs to queue up, and then after a task is executed, the queue position will be released, and the waiting ones can be entered and executed immediately. In the middle, it is designed to be first-in-first-out or first-in-last-out, as well as the design strategy of thread priority

3. When threads are queuing, what is the maximum waiting time? That is to say, it is impossible for a thread to be waiting forever. How long does it take to wait? What is the mechanism of thread processing? The technical term for this time is: access waiting time

4. The complete time of a thread is composed of three parts, response time: the time when the client initiates the request + access waiting time + logic execution time + time to return to the client. Generally, in the test, you can set the name of each thread as uuid, so that it is independent. You can use this uuid to let the development students cooperate to output the time output of each stage, and then you can get the specific information of each stage. It's time, and then judge whether the time is optimized according to the time.

Queue: Queue Specification: First In First Out ----"First In First Out

Queue method:

put: Put data into the queue

get: Take out the data in the queue

empty: determine whether the queue is empty

Queue Notes:

1. When the queue is full, it cannot be put into it again. If it is a program, it will be stuck

2. When the queue is empty, if the data is taken out, it will also be stuck

Producer consumer mode: implemented by queue: Queue

Producer (Producer): Continuously produces data, the method used is put

Consumer (Consumer): Continuous consumption of data, the method used is get

3. Performance test model (steps) [interview]

The performance test model can also be understood as a performance test step, which is summarized as follows:

• Pre-test preparation

Sort out the pre-test goals, such as how much throughput, testers lead and other relevant people to finalize
What is the target reference for performance testing? Product requirements documents and product user volume and system architecture.

*For example, Taobao performance test scenario: how many people log in at the same time, product response time, how many people pay, receive discounts, and receive SMS marketing at the same time

• Introduction of test tool technology

LoadRunner: An industrial-level performance testing tool developed by HP, which is a paid performance testing tool
JMeter: 100% performance testing tool developed using Java language, which is open source, and Internet companies basically use this performance testing tool
Locust: It is a performance testing tool designed based on Python coroutines, which requires writing Python code for performance testing
Gatling: It is a high-performance performance testing tool under the microservice architecture: developed by HP (less used)

• Performance testing plan

Specify the task start and end time for each performance test
Identify who is responsible for each task
Tools for clear performance testing

• Test design and development

Use selected tools or code to design scenarios and write code to test the designed performance test cases

• Test execution and management, data collection

Multiple test executions, collecting data for each performance test. Including system resources: CPU and memory, database: number of connections and iops, jvm resources: memory resources. Response Time: Max, Min, Median, Average, Median, 90%, 95%, 99%, Throughput
Data visualization after data collection

• test analysis

The actual collected results get the final conclusion, such as throughput response time, etc., and compare with the expected goal to judge whether the goal is achieved

System resources: CPU and memory, database: number of connections and iops, jvm resources: memory resources

[Interview: What preparations need to be made before the performance test]

Sort out the pre-test goals, such as how much throughput, testers lead and other relevant people to finalize
What is the target reference for performance testing? Product requirements documents and product user volume and system architecture.

*For example, Taobao performance test scenario: how many people log in at the same time, product response time, how many people pay, receive discounts, and receive SMS marketing at the same time

【Interview: During the performance test, what should I do if the development says that I need to add a server】

Can be added, production, development, testing are added, all consistent. It won't work if only development is added.

Notes on performance testing:

1. The configuration of the test environment server must be consistent with the configuration of the production environment server, so that the conclusions drawn have reference value

[Interview 1: What are the indicators of performance testing]

Including system resources: CPU and memory, database: number of connections and iops, jvm resources: memory resources. Response Time: Max, Min, Median, Average, Median, 90%, 95%, 99%, Throughput

[Interview 2: What tools are used for daily performance testing]

LoadRunner: An industrial-level performance testing tool developed by HP, which is a paid performance testing tool
JMeter: 100% performance testing tool developed using Java language, which is open source, and Internet companies basically use this performance testing tool
Locust: It is a performance testing tool designed based on Python coroutines, which requires writing Python code for performance testing
Gatling: It is a high-performance performance testing tool under the microservice architecture: developed by HP (less used)

[Interview 3: How is performance testing done]

1. Combining the product requirements document, the user volume of the product and the system architecture, sort out the pre-test goals, such as the throughput, and the testers will lead and finalize with other relevant people

2. Draw up a performance test plan, including

Specify the task start and end time for each performance test
Identify who is responsible for each task
Tools for clear performance testing (below)

LoadRunner: An industrial-level performance testing tool developed by HP, which is a paid performance testing tool
JMeter: 100% performance testing tool developed using Java language, which is open source, and Internet companies basically use this performance testing tool
Locust: It is a performance testing tool designed based on Python coroutines, which requires writing Python code for performance testing
Gatling: It is a high-performance performance testing tool under the microservice architecture: developed by HP (less used)

3. Review the test points of the test plan with relevant personnel, and make further revisions and confirmations

4. Execute the test and collect the data

Multiple test executions, collecting data for each performance test. Including system resources: CPU and memory, database: number of connections and iops, jvm resources: memory resources. Response Time: Max, Min, Median, Average, Median, 90%, 95%, 99%, Throughput
Visualization of collected data

5. Final test analysis

The actual collected results get the final conclusion, such as throughput response time, etc., and compare with the expected goal to judge whether the goal is achieved

[Interview 4: What is a queue]

In the program, the waiting queue will be involved. Whether it is synchronous interaction or asynchronous interaction, its maximum queue will be involved. The core idea of this design is to prevent the server from running without In the case of queues, there will be an avalanche and eventually the server will be paralyzed. In the waiting queue, the test needs to obtain the following performance data:

1. What is the value of the queue setting? What is the maximum task that can be run?

Queue: Queue Specification: First In First Out ----"First In First Out

Queue method:

put: Put data into the queue

get: Take out the data in the queue

empty: determine whether the queue is empty

Queue Notes:

1. When the queue is full, it cannot be put into it again. If it is a program, it will be stuck

2. When the queue is empty, if the data is taken out, it will also be stuck

[Interview 5: What is the producer consumer model]

Producer consumer mode: implemented by queue: Queue

Producer (Producer): Continuously produces data, the method used is put

Consumer (Consumer): Continuous consumption of data, the method used is get

Queue: Queue Specification: First In First Out ----"First In First Out

【Interview 6: What is the difference between IO-intensive and CPU-intensive】

Workload type:

Software testing resource library sharing

[Message: Software testing and learning, free access]

write at the end

Finally, let me say that if you want to take testing as a long-term career goal, you need to keep learning all the time. To make yourself competitive, no matter how many years you work now, as long as you take action, you will already have an advantage. That's it, I wish everyone a promotion and salary increase in 2023, and those who haven't joined can get an offer from the company they like as soon as possible, and everything goes smoothly.

Who knows! Performance testing can learn so quickly. . . . . .

Table of contents

foreword

(1) Response time from the user's perspective:

(2) The utilization rate of the system from the perspective of operation and maintenance

Mysql and other database attention indicators

JVM (for java technology line): CPU and memory usage of Java applications

The maximum number of tasks processed by the system:

(3) Development Perspective Concerns

(4) WEB front-end focus on performance

(5) Focus on performance from the testing perspective

1. Performance testing terminology [interview]

2. Performance testing theory

(1) Understanding of resource scheduling:

Scheduling strategy:

(2) Waiting queue

3. Performance test model (steps) [interview]

System resources: CPU and memory, database: number of connections and iops, jvm resources: memory resources

[Interview: What preparations need to be made before the performance test]

【Interview: During the performance test, what should I do if the development says that I need to add a server】

[Interview 1: What are the indicators of performance testing]

[Interview 2: What tools are used for daily performance testing]

[Interview 3: How is performance testing done]

[Interview 4: What is a queue]

[Interview 5: What is the producer consumer model]

【Interview 6: What is the difference between IO-intensive and CPU-intensive】

Software testing resource library sharing

write at the end

Guess you like