Performance testing concepts
Generally speaking, performance is an indicator that indicates the degree to which a software system or component meets its timeliness requirements; secondly, performance is a characteristic of a software product that can be measured in time. The timeliness of performance is measured in response time or throughput. Response time refers to the time it takes for the server to respond to a request.
System performance includes two dimensions: time and space. Time refers to the response time of customer operation services, and space refers to the consumption of system resources when the system executes client requests. The performance that customers care about is only the performance of time. Customers do not care about the causes of performance problems, but performance test engineers and system engineers must pay attention to the usage of system resources.
Performance testing is the end-to-end performance indicator of the test system (that is, the process in which the client sends a request and returns the information to the server). Currently, it is impossible to design a system with quantitative performance in the system design stage, that is, the development engineer cannot guarantee the performance of the designed system. How the performance performs depends on testing to verify the performance. Performance testing is a series of testing processes. Before the system is put into the market, the performance of the system must be tested, otherwise it cannot be determined whether the performance of the system can meet the needs of customers.
At the same time, I have also prepared a software testing video tutorial for everyone (including interviews, interfaces, automation, performance testing, etc.), which is below. If you need it, you can watch it directly, or you can directly click on the small card at the end of the article to get the information document for free.
Where to watch software testing video tutorials:
Bilibili Fengshen’s interface testing tutorial, complete 70 projects in 30 days (including automated testing and performance testing), you will be employed immediately after completing the course, and you will be free of charge forever!
Performance test automation
There are many difficulties in performing performance testing manually, as shown in the figure.
1. Consume a large amount of testing resources (mainly including the cost of testers and testing machines).
2. How to ensure concurrent testing?
3. How to collect test results?
4. How to repeat this test?
Performance testing tools need to be used to replace manual testing, that is, the process of performance testing automation. Automated performance testing is a method that leverages people, processes, and technology to reduce risks during application deployment, upgrades, or patch deployment. At its core, automated performance testing applies production workloads to systems prior to deployment while measuring system performance and end-user experience.
An automated performance testing solution should be able to solve the following problems:
1. Simulate hundreds or thousands of users interacting with the system without excessive hardware requirements.
2. Measure end-user response time.
3. Monitor load system components.
Efficient automated testing solutions (i.e. performance testing tools) typically use four main components to build and run tests:
1. Virtual user generation program used to capture end-user business processes into automated scripts.
2. Monitor, used to organize, drive, manage and monitor workloads.
3. Load generation program, used to run virtual users during execution.
4. Analysis engine for viewing, analyzing and comparing results.
Well-structured performance testing should be able to answer the following questions:
1. Is the application responsive enough to target users?
2. Can the application handle the expected user load and more?
3. Is the application capable of handling the large number of transactions required by the business?
4. Is the application stable under expected or unexpected user loads?
5. Do users have a positive experience when put into use (i.e. fast response time)?
Mainstream performance testing tools
There are many performance testing tools on the market at present. The mainstream performance testing tools include LoadRunner, QALoad, SilkPerformer and Rational Performance Tester. These are all load performance testing tools, and their principles are the same. The first is the recording script. Through the recording script, the performance testing tool obtains the content sent by the client to the server through the protocol; then through the playback script, the recorded content is played back to simulate multiple users sending requests to the system under test at the same time. To achieve the purpose of concurrent testing; finally, the performance testing tool saves the collected test data to the database, and generates relevant views through the analyzer to achieve the purpose of performance testing.
Here is a brief introduction to these mainstream performance testing tools:
(1)LoadRunner
LoadRunner is a load testing tool that predicts system behavior and performance. It can simulate the concurrent load of tens of millions of users and monitor system performance in real time to confirm and find problems. LoadRunner enables testing of entire enterprise architectures. By using LoadRunner, enterprises can minimize testing time, optimize performance and accelerate the release cycle of application systems.
The latest version is version 11.0, and the manufacturer Mercury was acquired by HP in 2006.
The protocols supported by LoadRunner are as follows:
1)自定义：C Vuser、VB Vuser、VBScript Vuser、http://VB.NET Vuser、Java Vuser、JavaScript Vuser。
2)电子商务：Action Message Format(AMF)、AJAX(Click and Script)、File Transfer Protocol (FTP)、Flex、Listing Directory Service(LDAP)、Microsoft .NET、Web(Click and Script)、Web(HTTP/HTML)、Web Services。
3) Client/server: DB2 CLI, Domain Name Resolution (DNS), Informix, MS SQL Server, ODBC, Oracle (2-Tier), Sybase CTlib, Sybase DBlib, Windows Sockets.
4 )ERP/CRM ： Oracle NCA 、Oracle Web Applications 11i 、PeopleSoft Enterprise 、Peop leSoft-Tuxedo、SAP-Web、SAP(Click and Script)、SAPGUI、Siebel-Web。
5) Mail services: Internet Messaging (IMAP), MS Exchange (MAP), Post Office Protocol (POP3), Simple Mail Transfer Protocol (SMTP).
6) Middleware: Tuxedo, Tuxedo 6.
7) Wireless services: i-Mode, Multimedia Messaging Service (MMS), WAP.
8) Streaming media: Media Player (MMS), Real.
(2)QALoad
QALoad is an automated load testing tool for client/server systems, enterprise resource planning (ERP), and e-commerce applications. QALoad, part of QACenter, thoroughly measures an application's scalability and performance through repeatable, real-world testing. QACenter brings together a complete set of cross-enterprise automated testing products designed to improve software quality. QACenter can automate testing tasks across multiple platforms throughout the development lifecycle. During the preparation period for production, QALoad can simulate hundreds or even thousands of users executing key services concurrently to complete application testing, and optimize system performance based on discovered problems to ensure successful deployment of applications. The latest version is version 5.6, and the manufacturer is Compuware.
The protocols supported by QALoad are as follows:
Communication layer: Winsock, IIOP, WWW, WAP, Net Load.
数据层：ODBC、MS SQL Server、Oracle、Oracle Forms Server、Sybase、DB2、ADO。
Application layer: SAP, Tuxedo, Uniface, QARun, Java.
(3)Silk Performer
SilkPerformer is the industry's leading application performance testing solution. It supports the current mainstream application platforms in the industry, uses thousands of virtual users to simulate various real load scenarios that may be encountered in the production environment, and helps users quickly locate possible performance bottlenecks. , while providing diagnosis and analysis capabilities to help development and testing teams quickly fix application performance problems, provide strong information support for application release decisions, and accelerate product release. The latest version is version 7.0, and the manufacturer is Micro Focus.
The protocols supported by SilkPerformer are as follows:
1) Mainstream database access protocols: ODBC, ADO, Oracle OCI, IBM CLI.
2) Mainstream protocols: HTTP(S), SMTP/POP, MAPI, FTP, LDAP, WAP, MMS, Radius, TCP/IP, UDP, SSL, SOAP (XML), i-Mode.
3) Streaming media technology: Macromedia Flex/AMF, Streaming (MS, Real).
4) Mainstream interfaces and application frameworks: CORBA (IIOP), EJB (IIOP, RMI), (D)COM (COM+, MTS), ActiveX, DLL, BEA Tuxedo (ATMI, JOLT), Oracle Forms, .NET Framework, J2EE /Java Framework, VB6 Framework.
5)ERP/CRM 系统：SAP、PeopleSoft、Siebel、Oracle Applications。
6)其他：Outlook Web Access、MS .NET SOAP Stack、Apache Java SOAP Stack、Chordiant、E.piphany、Lawson、SSPS ShowCase、Amdocs Clarify。
(4)Rational Performance Tester
Rational Performance Tester (RPT for short) is also a performance testing tool suitable for performance and reliability testing of Web-based applications. Rational Performance Tester combines ease of use with in-depth analysis capabilities to simplify test creation, load generation, and data collection to help ensure that applications perform reliably and support thousands of concurrent users. The latest version is version 8.1, and the manufacturer is IBM Rational.
The protocols supported by Rational Performance Tester include: HTTP, SAP, Siebel, SIP, TCP Socket, Citrix, etc.
Rational Performance Tester can choose 4 types of recorders when recording scripts: RPT HTTP recording, SAP recording, SDK Smaple Socket recording and Citrix recording.
Summarize
PS: Here is a collection of self-study tutorials for software testing. It should be very helpful for those who are developing in the testing industry. In addition to basic introductory resources, bloggers also collect a lot of advanced automation resources. From theory to practice, only by integrating knowledge and action can you truly master it. The full set of content has been packaged on the network disk, and the total content is close to 500 G.
☑ 240 episodes - a complete set of video courses from zero to mastery
☑ [Courseware + Source Code] - complete supporting tutorials
☑ 18 sets - source code of practical testing projects
☑ 37 sets - testing tool software package
☑ 268 - real interview questions
☑ 200 templates - Interview resume template, test plan template, software test report template, test analysis template, test plan template, performance test report, performance test report, performance test script case template (complete information)
These materials should be the most comprehensive and complete preparation warehouse for friends who do [software testing]. This warehouse has also accompanied me through the most difficult journey. I hope it can also help you! Everything must be done early, especially in the technical industry, where technical skills must be improved.
