Preface
This article mainly briefly summarizes the principles of performance testing tools and how to select them. Performance testing is different from functional testing. The execution of performance testing is the repetition and concurrency of basic functions. Multiple users need to be simulated. Indicator parameters need to be monitored when the performance test is executed. At the same time, the results of the performance test are not so obvious and the data needs to be analyzed. These characteristics determine that performance testing is more suitable to be completed through tools.
1. A brief discussion on why tools are needed
Let’s take a look at the definition of a tool: It originally refers to an appliance needed for work, and later extended to a means to achieve, complete or promote something. (---Explanation from Baidu)
1. From the perspective of human evolution, being able to make and use tools is the most fundamental difference between humans and apes, because tools can help us improve productivity and efficiency.
2. Imagine what would happen if you didn’t use tools for performance testing?
We can analyze it from the perspective of the definition of performance testing. Performance testing refers to testing various performance indicators of the system through automated testing tools to simulate a variety of normal, peak, and abnormal load conditions. If you do not use tools, relying solely on manual performance testing will have the following disadvantages:
a) Testing requires a lot of resources:
In order to simulate multiple loads and concurrent scenarios, multiple people need to work together. Usually, the test does not have a lot of resources, and even if there are resources, the manual effect will be greatly reduced, and some scenarios may even be impossible to complete manually.
b) Very poor repeatability:
Performance testing often requires repeated tuning and test execution. Without the help of tools, it is unimaginable to rely solely on manual labor.
c) Poor test accuracy:
Due to the need to simulate a variety of load and concurrency scenarios, if it is operated manually, errors will inevitably occur. Compared with tools or programs, this error will be larger and will have a greater impact on the test results.
d) Poor collection, organization and presentation of results:
If there are no tools, manual data collection will have larger errors than tools.
2. The relationship between performance testing and performance testing tools
1. Performance testing is divided into system testing from the testing stage, and it has no direct relationship with the specific tools used. Using tools is just a method and means to improve the efficiency and accuracy of performance testing. In essence, there is no substantial difference from using tools when doing other things.
2. Performance testing is not equal to Loadrunner. LR is just one of the performance testing tools, and it is not omnipotent. In some cases, it may not come in handy.
3. The difference between automated testing tools and performance testing tools: Performance testing tools are generally based on communication protocols (conventions followed by clients and servers to exchange information). They do not care about the UI of the system, while automation uses object recognition technology. , pay attention to the UI interface. It is impossible or difficult to create load with automation, but it is easy with protocols.
现在我也找了很多测试的朋友，做了一个分享技术的交流群，共享了很多我们收集的技术文档和视频教程。
如果你不想再体验自学时找不到资源，没人解答问题，坚持几天便放弃的感受
可以加入我们一起交流。而且还有很多在自动化，性能，安全，测试开发等等方面有一定建树的技术大牛
分享他们的经验，还会分享很多直播讲座和技术沙龙
可以免费学习！划重点！开源的！！！
qq群号：110685036
3. Reference for selecting performance testing tools:
Usually in a company or project, when we choose any tool, we will do some research, the purpose is to choose the tool that is suitable for the company or project. Performance testing tools are no exception, and can usually be considered from the following aspects:
1. Cost:
a) Tool cost: Tools are usually divided into two types: commercial closed (shou) source (fei) and non-commercial open (mian) source (fei). Commercial tools usually have more powerful functions and are charged. Because of the fee, after-sales service can be provided. If If something goes wrong, professionals will help. Open source tools are usually free and have limited functions, and the organization that maintains the tools is also voluntary, so if you encounter problems you need to solve them yourself, there is no dedicated person to provide services. The specific choice of commercial or open source tools needs to be considered based on the company's situation, such as company size, the cost it is willing to bear, and the overall situation of the project. Generally speaking, large companies can usually afford the cost of tools and will consider purchasing commercial tools. Small companies may choose open source tools due to financial pressure.
b) Learning cost: Using any tool requires learning, which will incur learning costs (such as time). Therefore, we also need to consider the learning costs of project team members when selecting tools. If there are two tools A and B that can meet the testing needs of the project team, if tool A is used by most people, and tool B is used by only a very small number of people, then it is recommended that tool A be given priority. Usually, it is best for testers to be familiar with a process commercial (performance) tool, an open source free (performance) tool, and also need to be familiar with common (performance) script development languages, etc. This is a basic requirement.
2. Supported protocols:
Performance testing is usually very closely related to the protocol. For example, the B/S system usually uses the http protocol to exchange information between the client and the server provider, and the C/S system usually uses the socket protocol to exchange information. When choosing a tool, you need to consider the protocol your project uses. Whether a testing tool can meet the testing needs and achieve satisfactory test results is the most basic issue to consider when choosing a testing tool.
3. Vitality:
There are many performance testing tools nowadays, such as LR and jmeter, which are more popular tools, and there is a lot of relevant information on the Internet, but some niche tools may have less information on the Internet. If you encounter a relatively technical problem during the use of the tool, when asking for solutions or help, public tools will be relatively more advantageous. After all, the more people use it, the more information there is, so you can try it yourself. The problem you encountered may have been encountered and solved by others before. Even if no one has encountered it before, because there are many people using it and researching it, I believe there will always be experts who can help solve it through the help of the community or forum.
4. Cross-platform:
Needless to say, this point goes without saying. Just look at why JAVA has always been so popular.
4. Common performance testing tools:
Performance testing tools, in theory, all tools used in the performance testing process can be called performance testing tools, and are usually divided into the following categories:
illustrate:
- Server-side performance testing tools: need to support generating stress and load, recording and generating scripts, setting up and deploying scenarios, generating concurrent users and exerting continuous pressure on the system.
- Web front-end performance testing tool: You need to pay attention to the processing process of client tools such as browsers on the specific pages that need to be displayed.
- Mobile terminal performance testing tools: Same as web terminal performance testing tools, they also need to care about the page processing process. In addition, they also need specific data collection functions, such as: recording of mobile phone CPU, memory, power, startup time and other data.
- Resource monitoring tool: This is mainly able to collect data during the performance test process and provide a good way to display the results.
5. Characteristics of common performance testing tools
- JMeter: It uses a multi-threaded model and is highly scalable, but the manufacturing pressure is not that high. It is very suitable for pressing some Tomcat services or some back-end interfaces. The disadvantage of JMeter is that the pressure value cannot be accurately controlled and it is difficult to adapt to high concurrency situations. Moreover, because it is written in JAVA, it consumes resources.
- LoadRunner: More like a simulator, it is more suitable for front-end construction of more complex scenarios, such as simulating 100 user login scenarios. LoadRunner provides good support for non-technical personnel. LoadRunner does not work with backend interfaces.
The following table is a comparison table between JMeter and LoadRunner:
| describe | JMeter | LoadRunner |
| Architecture principles | Through the intermediate agent, monitor and collect instructions from concurrent clients, generate scripts from them, send them to the application server, and then monitor the application server feedback process. | Same JMeter |
| Install | Simple, just decompress, more flexible | The LoadRunner installation package is relatively large, installation is troublesome, and the tool itself is relatively bulky. |
| Supported protocols | Supports multiple protocols: HTTP, HTTPS, SOAP, FTP, Database via JDBC, JMS, etc., but it is not comprehensive enough compared to LR. For this reason, jemter is relatively more flexible and lightweight | It supports many protocols and is relatively comprehensive, but because of this the tool itself appears to be bulky and not flexible enough. |
| Script recording | Provides a function to use local ProxyServer (proxy server) to record and generate test scripts, and also supports badboy recording and then generating JMeter scripts | The built-in recording function is powerful and can be recorded and played back directly. |
| Concurrency model | Increase concurrent users by increasing the number of thread groups or setting the number of loops | Supports multiple concurrency models by selecting what kind of scenario you want to set up in the scenario and then selecting the number of virtual users |
| Distributed testing | Support, multiple agents can be set up to achieve concurrent pressure on multiple machines through remote control | Same JMeter |
| Resource monitoring | Implemented through JMeterPlugins plug-in and ServerAgent | Comes with resource monitoring function |
| Report analysis | Generate HTML reports by integrating with Ant | Supports generating HTML and Word reports |
| Virtual IP | not support | support |
| Internet speed simulation | not support | support |
| Scalability | Open source, the source code can be modified according to needs | Implemented through extended function library |
| learning cost | Mainly self-study information from the official website | There are a lot of online materials and related training. If you buy the genuine version, there is also technical support. |
Finally, I would like to thank everyone who read my article carefully. Looking at the increase in fans and attention, there is always some courtesy. Although it is not a very valuable thing, if you can use it, you can take it directly!
Software Testing Interview Document
We must study to find a high-paying job. The following interview questions are from the latest interview materials from first-tier Internet companies such as Alibaba, Tencent, Byte, etc., and some Byte bosses have given authoritative answers. After finishing this set I believe everyone can find a satisfactory job based on the interview information.
