The goal of software performance testing is to identify any performance bottlenecks in the application. The performance of a software system depends not only on the design and coding of the system itself, but also on the operating environment on which the system depends. The operating environment of the system will depend on some key factors, such as: system architecture, hardware configuration, network bandwidth, supporting software such as database and middleware, and the size of the external load.
The improvement of system performance is a continuous improvement process of testing, adjusting, retesting, and readjusting... that is, performance tuning, which adjusts software design, code, and system configuration according to the results of performance testing.
Types of performance testing
Performance tests mainly include 6 types, as shown in the following table.
| type |
illustrate |
| load test |
Determine how the software behaves at a given time as the workload increases, be it concurrent users, number of transactions, software behavior, etc. Continuously increase the pressure on the system under test until performance indicators (such as response time) exceed expected indicators or some resource usage has reached saturation. The processing limit of the system can be found to provide data for system tuning. |
| pressure test |
Measure the software stability under higher traffic load (more users, transactions, etc.) than normal working parameters, that is, the system is in a certain saturated state, such as CPU, memory, disk space and other hardware resources are saturated, the system can The ability to handle sessions, and whether the system is error-prone. |
| stability test |
Evaluate the long-term performance of software under a fixed regular workload. In other words, it determines how long the software can sustain a constant workload to provide long-term sustainability. During testing, the testing team monitors KPIs such as memory leaks, memory usage, low memory, etc. The stability test also analyzes the response time and throughput after long-term use to see whether these indicators are consistent. |
| spike test |
Spike testing is a type of stress test that measures software performance under significant, sudden increases in workload, such as the number of concurrent users. Whether the software can handle sudden increases in workload repeatedly and rapidly. |
| capacity test | Tests the efficiency of software when processing large amounts of data to check for data loss, system response time, data storage reliability, etc. |
| scalability test | Measure the effectiveness of software at handling increasing workloads. Scalability testing can be performed by gradually adding data volumes or users while monitoring software performance. |
Today, with the development of DevOps, performance testing has risen to software system life cycle performance engineering. This article introduces performance testing tools and solutions from five aspects: client performance testing, server performance testing, application performance monitoring of distributed systems, and full-link stress testing of distributed systems.
Google Lighthouse
PerfDog
Monkey
Money runner
mobileperf
Pyroscope
MemoryLeakDetector
JMeter
LoadRunner
Webload
Gatling
k6
Vegata
Locust
Skywalking
Pinpoint
Client Performance Testing Tool
1）Google Lighthouse
Lighthouse is an open source tool developed by Google to analyze web application and page performance. Lighthouse analyzes web applications and web pages, collects modern performance indicators and insights about developer best practices, and allows developers to optimize and improve websites based on the generated evaluation pages to improve user experience.
Lighthouse is integrated directly into the chrome developer tools, under the 'Audits' panel.
2) PerfDog
PerfDog is a mobile full-platform iOS\Android performance testing tool platform. It can quickly locate and analyze performance problems, improve the performance and quality of APP applications and games. The mobile phone does not need ROOT/jailbreak, and the mobile phone hardware, games and applications do not need to be changed. It is extremely simplified and instant Plug and play.
Official website address:
https://wetest.qq.com/product/perfdog
3) Monkey
Monkey is a command-line tool provided by Android SDK, which is easy to use and run on any version of Android emulator and physical device. Monkey will send a pseudo-random stream of user events, suitable for stress testing the app.
4) Monkeyrunner
The MonkeyRunner tool provides multiple APIs. Through the monkeyrunner API, you can write a Python program to simulate the operation and control of the Android APP, test its stability and record the problems that occur easily by taking screenshots.
5) mobileperf
Tmall team's open-source PC Android performance stability test tool can collect Android performance data: cpu memory fluency fps logcat log traffic process thread number process startup log, mobileperf also supports native monkey test.
download link:
https://github.com/alibaba/mobileperf
6) Pyroscope
Pyroscope is an open source continuous analysis platform. It can help you find performance problems and bottlenecks in the code, and the cause of high CPU utilization. And help you understand the call tree of the application, providing rich diagrams and call tree display.
Official website address: https://pyroscope.io
7) MemoryLeakDetector
MemoryLeakDetector is a local memory leak monitoring tool developed by the Xigua video android team. It has the advantages of simple access, wide monitoring range, excellent performance and good stability. It is widely used in the native memory leak management of ByteDance's main application.
Official website address: https://github.com/bytedance/memory-leak-detector
Server performance testing tool
8) JMeter
JMeter is a Java-based stress testing tool developed by the Apache organization. Used for stress testing software, it was originally designed for web application testing, but has since expanded to other testing areas. It can be used to test static and dynamic resources such as static files, Java servlets, CGI scripts, Java objects, databases, FTP servers, and more.
JMeter can be used to simulate huge loads on servers, networks or objects, to test their strength and analyze overall performance under different stress categories.
Official website address: https://jmeter.apache.org/
9) LoadRunner
LoadRunner is a load testing tool that predicts system behavior and performance. By simulating tens of millions of users implementing concurrent loads and real-time performance monitoring to identify and troubleshoot issues, LoadRunner enables testing of entire enterprise architectures.
Official website address:
https://www.microfocus.com/zh-cn/portfolio/performance-engineering/overview
10) WebLOAD
WebLOAD is an enterprise-level load and performance testing tool for web applications that provides performance, integrity, and scalability testing, and can simulate thousands of users simultaneously, so you can test heavy traffic loads and report on applications Weaknesses, constraints, and performance bottlenecks in .
Use WebLOAD for website load testing, continuous testing, cloud load testing, and more. The tool can generate load from the cloud or from a local machine and provides an integrated development environment (IDE) for visually recording, editing and debugging test scripts.
Official website address: https://www.radview.com/
11) Gatling
Gatling is an open source tool for high-performance server performance testing developed based on Scala. It is also a powerful load testing tool that works out of the box. Gatling is mainly used to measure HTTP-based servers, such as web applications, RESTful services, etc. Gatling is the tool of choice for load testing against any HTTP server.
Official website address: https://gatling.io/
12) k6
k6 is a high-performance load testing tool and a high-performance tool designed to run tests under high load in pre-production and QA environments, scriptable in JavaScript. It is a developer-centric (of course, testers can also use it, because it is really convenient), free and open source load testing tool, designed to make performance testing productive and pleasant experience, can maximize Reduce system resource consumption.
Official website address: https://k6.io/
13) Vegata
Vegeta is a multifunctional HTTP load testing tool written in Go language, providing command line tools and a development kit.
Official website address:
https://github.com/tsenart/vegeta
14) Locust
Locust is an open source stress testing tool developed in Python that supports distribution. It can load test web interfaces by writing python scripts. Locust can support thousands of concurrent user visits on a single machine, and because of its support for distributed operation, theoretically, Locust can support extremely high concurrent tests while using fewer stress machines.
Official website address: https://locust.io/
Performance monitoring tools for distributed systems
In a distributed system with a microservice architecture, when a client initiates a request, it often calls multiple services, involving multiple middleware, and the system is distributed on multiple servers. Therefore, when a performance bottleneck occurs in the system , fault diagnosis becomes very complicated. The application performance monitoring (APM) tool of the distributed system locates the performance bottleneck on the link by tracing and analyzing the service call chain.
Online performance monitoring refers to monitoring the actual data of system performance with the help of monitoring tools. Because it is real data, it is more objective and analytically valuable than the test results obtained by generating load through tools in the R&D environment.
15) SkyWalking
SkyWalking is an excellent open source APM tool in China, which provides an intuitive observation platform for distributed systems, used to collect, process and visualize data from services and cloud-native infrastructure, through monitoring, alarming, visualization and distributed tracking Features such as microservices, distributed, and containerized system architectures provide observability (observability). It can observe distributed systems across different clouds, and it supports the next-generation distributed architecture Service Mesh from SkyWalking 6.
Official website address:
http://skywalking.apache.org/
16）Pinpoint
Pinpoint is an APM (Application Performance Management) tool for large-scale distributed systems, written in Java/PHP. Pinpoint provides a solution to help analyze the overall structure of the system, and by tracking transactions in distributed applications, analyze how components in the system are connected to each other, for full-link monitoring of large distributed systems, and to obtain different services Between the service and the database, as well as the call relationship of the methods inside the service, you can also monitor the call duration, availability and memory of the method.
download link:
https://pinpoint-apm.github.io/pinpoint/
A full-link stress testing platform for distributed systems
Full-link stress testing refers to simulating massive user requests and data access production environments in real business scenarios, conducting comprehensive and realistic stress tests on the entire business link, finding performance bottlenecks in distributed systems in advance and continuously optimizing them practice. At present, most enterprises use open-source tools Gatling and JMeter to build pressure measurement clusters for full-link pressure measurement . At the same time, there are also commercial full-link stress testing solutions in China, such as the Perfma full-link stress testing solution, JD.com’s ForceBot platform, Meituan’s Quake, AutoNavi’s TestPG, ByteDance’s Rhino, Alimama’s MagicOTP and performance The test platform ACP, and Ali's Amazon, PTS and JVM-SANDBOX platforms.
END meager strength
Finally, I would like to thank everyone who has read my article carefully. Seeing the fans’ growth and attention all the way, there is always a need for reciprocity. Although it is not a very valuable thing, you can take it away if you need it:
These materials should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey. I hope it can help you too!
加入我的软件测试交流群：110685036免费获取~（同行大佬一起学术交流，每晚都有大佬直播分享技术知识点）Software testing interview applet
The software test question bank maxed out by millions of people! ! ! Who is who knows! ! ! The most comprehensive quiz mini program on the whole network, you can use your mobile phone to do the quizzes, on the subway or on the bus, roll it up!
The following interview question sections are covered:
1. Basic theory of software testing, 2. web, app, interface function testing, 3. network, 4. database, 5. linux
6. web, app, interface automation, 7. performance testing, 8. programming basics, 9. hr interview questions, 10. open test questions, 11. security testing, 12. computer basics
method of obtaining:
