1. Basic concepts of CS/CSS system architecture
1.1 System architecture definition
Although the B/S structure and J2EE structure have become more and more popular models, applications based on the traditional C/S structure are still widely used in various industries. Especially the counter-core accounting system of commercial banks in the financial industry. On the one hand, traditional commercial banks generally have a large number of character terminals and other devices that need to be reused. On the other hand, they also have a large number of intensive high-counter services that require high real-time performance. They use traditional C/S-based structure or C The application efficiency of the /S/S structure is more guaranteed.
The C/S structure is the CLIENT/SERVER structure. The traditional C/S structure is generally divided into two layers: client and server. The basic working principle of this structure is that the client program sends an SQL request to the data server, and the server returns the data and results. The client is responsible for implementing user interface functions and encapsulating part of the application logic. The server-side database server mainly provides data storage functions, but also provides part of the application logic through triggers and stored procedures.
The C/S/S structure is a three-tier structure of client/application server/database server, with an application server added in the middle, which usually implements application logic and is a bridge connecting the client and the database server. It responds to requests sent by users to perform certain business tasks and deals with database servers. In terms of technical implementation, it usually selects middleware products, such as BEA's TUXEDO and IBM's CICS. (In fact, the application of J2EE architecture also belongs to this three-layer or multi-layer structure, which is not included here.)
Compared with the two-layer C/S structure, the three-layer or multi-layer C/S structure has its advantages mainly in: enhanced security, improved efficiency, easy maintenance, scalability, shareability, and good openness.
1.2 System Architecture
1.3 Characteristics of performance testing in CS/CSS system architecture
1.3.1 Performance Influencing Factors of CS/CSS System Architecture
Due to the following characteristics of the CS/CSS system, it is very difficult for test engineers to perform performance testing on a CS/CSS system:
-
Various parts of the entire system use multiple operating systems, with differences in performance;
-
Multiple databases are used in each link of the entire system architecture, and there are also differences in performance;
-
There are multiple applications, belonging to multiple types, distributed on different devices, including self-developed applications and third-party applications;
-
The equipment and components in the system are connected and communicated through different protocols;
-
The system has many internal interfaces and many performance bottlenecks; the overall performance of the system often depends on the worst part; separate testing and joint testing are required
-
The performance index of the system is not only related to the application system architecture, but also related to the business model of specific industry applications;
-
The industry application adopting this architecture is often a 7×24 hour system;
-
Industry applications using this architecture may have many high cabinet businesses, which will affect the selection and conversion of performance measurement items;
-
Each link basically communicates by exchanging data messages, and the format is often more complicated.
Therefore, such a system is a test for the depth and breadth of the test engineer's knowledge. For such a system, how to use what kind of test strategy, how to analyze test requirements, how to select the conversion calculation model of performance measurement items, how to determine the test content and rounds, how to design performance test cases, etc., and plan and implement performance tests Many other problems in, all need to follow a systematic method to solve.
1.3.2 Basic strategy of performance test in CS/CSS system architecture
1. Determine the scope of testing
First, you can analyze the most likely bottlenecks in the stress test, so as to purposely adjust the test strategy or test environment so that the stress test results truly reflect the performance of the software. For example, server hardware limitations, database access performance settings, etc. often become important factors that restrict software performance, but these factors are obviously not the most concerned by users. We must set up some settings to minimize the impact of these factors before testing. .
In addition, users are more concerned about the performance of which link in the entire system will affect the scope of work. If some links are brand-new systems, and some links are already mature systems with only minor changes, it is possible that the partial performance test of the brand-new system needs to be systematic and comprehensive.
2. Analyze the customer's performance test requirements
Has the customer clearly proposed performance indicators, or only provided user usage and historical transaction flow data, and we need to calculate the performance benchmark ourselves? Is the purpose of performance testing to verify system performance or to determine the ideal configuration of the target system? Do I still need to use the test results to predict the processing power of different models? Questions such as whether it is required to arrange the performance tuning process in each round of performance testing need to be answered in a targeted manner.
3. Plan and plan for performance testing
In the test plan and program, attention should be paid to the solution of the problems raised in the test requirement analysis stage.
4. The determined test pass criteria, performance test plan, and results must be approved by the customer
To confirm with the customer, what is the standard for the system's performance indicators; for the plan and results of each part and step in the performance test, and even the performance test process, it is necessary to determine whether the customer is required to confirm and sign according to its importance. Obtaining the recognition of customers is the most important.
1.3.3 Performance measurement and performance detection in CS/CSS system
Performance measurement
1. The performance measurement must be carefully planned before the performance test starts:
The software performance measurement technology has a wide range. It can include performance measurement techniques such as logging, event counting, event duration, and sampling.
-
Determine the strategy for performance measurement: what are we going to test?
-
Plan what measurement tools to use in performance testing.
2. Representativeness of measurement
- The measurement results must reflect the important factors affecting performance: workload, software, and computer system environment.
3. Repeatability of measurement
- Ability to control workload, software, and computer system environment, so that the test process can be repeated.
Performance detection technology
When performing performance measurements, standard commercial tools can be used, but often the data provided by standard tools cannot meet the requirements. Performance probing is to insert code probes at key points of the program to measure the execution characteristics of the software. So as to achieve the following goals:
-
Performance data acquisition is more convenient
-
Increased data level of detail
-
Data collection methods are more controllable
According to the recommendations of SPE (Software Performance Engineering), software detection requirements should be regarded as an integral part of the software architecture. Design software probes when designing software.
Therefore, in the performance test process of the planning project, it is recommended to consider the island performance detection requirements when designing the software, so as to prepare for better performance measurement in the performance test.
1.3.4 Types of performance test under CS/CSS system
Generalized performance testing includes many types. Such as:
-
Scalability/load testing: By continuously increasing the pressure on the system under test, until performance indicators such as response time exceed a predetermined indicator or a certain resource has reached a saturated state. This test can find the processing limit of the system and provide data for system tuning.
-
Performance testing: Test whether the performance of the system meets the production performance requirements by simulating the business pressure of the production operation and the combination of usage scenarios. For example, the actual production structure test is used to find the maximum throughput and the best response time to ensure the stability and safety of the line.
-
Configuration testing: Find the optimal allocation principle of system resources through testing.
- Concurrency testing: Test whether there are deadlocks or other performance problems when multiple users access the same application, the same module, or data record at the same time.
-
Stress testing: Test the system in a certain saturation state, such as when the CPU and memory are saturated, the session capacity that the system can handle, and whether the system will encounter errors.
-
Volume testing: The maximum session capacity that the test system can handle.
-
Reliability testing: Run for a period of time by loading the system with certain business pressure (for example, the utilization rate of resources is 70-90%).
-
Failover testing: For a system with redundant backup and load balancing, this test is used to verify whether the user can continue to use the system if a local failure occurs in the system, and how much the user will be affected.
In the actual performance test under the CS/CSS system, the selection and combination of performance test types need to be carried out according to the specific situation.
1.3.5 Components of performance test under CS/CSS system
Usually in a CS/CSS system, it is divided into user interface layer, service logic layer and data service layer, which correspond to clients, application servers, and database servers respectively. For example, in financial industry applications, the client carries counter services and is deployed at branches (including character tellers or graphic tellers), as well as self-service services deployed on self-service equipment; application servers are mainly used for routing functions and services Processing functions, and the role of channel integration; and the core business processing system includes trading platforms, business logic, core processing, data processing, etc.
Because the business logic is distributed in different links, the system has many internal interfaces and many performance bottlenecks, and the overall performance of the system often depends on the worst part. Therefore, the overall performance test of the entire system may need to do its own internal performance test for each link.
For example, the following is an example of CS/CSS system financial industry application:
In order to test the performance of the entire system, it is necessary to conduct internal performance tests for each component in advance, such as the pressure test of the back-end host, the pressure test of the SNA gateway, the pressure test of the large front-end system, the pressure test of the front-end system, and the external system access Stress testing and so on.
In the internal stress test conducted this time, in order to eliminate the influence of other parts of the system, it is necessary to isolate their respective parts, and the drivers and piles are replaced by software testing tools or self-made programs. In the internal pressure test of each part, the various performance test types described in the previous section can be used for performance measurement according to the specific situation.
2. Calculation model of metric items for performance testing in CS/CSS system architecture
2.1 Define metric items
When conducting model analysis for performance testing, first determine the key performance goals. It should be obtained through communication with customers. These goals should be to solve the performance problems that customers care about. That is to say, the customer's performance testing needs are embodied as key performance goals. Performance goals should be clear and measurable. For example: support 2000 concurrent users; continuous operation for 30 days without downtime, etc.
After clarifying the key performance objectives and the pass/fail criteria of the performance test, it is necessary to define how to measure the key performance objectives and the pass/fail criteria of the performance test. The standard items of measurement will affect the choice of test methods and test tools. For example, if you want to measure the concurrent response time of 100 users, you must specify which of the following standard items to measure:
-
Response time for each concurrent user
-
In the case that 99 users have already accessed, the response time of the 100th user
-
Both indicators must be measured
2.2 Performance benchmark and test intensity estimation
In fact, key performance goals are not always easy to clarify. Customers often only have some historical data and some expected proportions of business growth and so on. But these data are also very useful to us. They can be used as performance benchmarks for our design and testing.
For the performance test, the performance benchmark of the design must first be proposed when designing. The so-called performance benchmark is to think: how many people use this system? What is the maximum number of users when using it? What is the time interval between users during peak periods, and what is the system response time at what level? How big is the data growth? To what order of magnitude and time is the data growth unbearable for hardware? What are the network implementation conditions? How to control CPU and memory growth during processing? All these, and then conditionally optimize the coding implementation and hardware configuration according to the performance benchmark during the design, and the test only verifies whether the system reaches the expected design goal.
But the current situation is often: after the design is designed to require a performance test, what to test and then what is, like an exam without a standard answer but requiring everyone to pass. Or, although the customer has clearly put forward the key performance indicators, but did not consider the design, rely on the performance test to give the actual performance data, and then compare and optimize. Performance testing is based on performance benchmarks, especially based on the key performance targets obtained through calculation and conversion, in order to get the expected results. This point requires test engineers to repeatedly inculcate this concept to designers, otherwise, performance testing research, including tool research, will always stay in the discussion stage. Don't consider the optimization problem after the coding is completed. If the optimization is not completed after the project is implemented, it will be difficult to ensure customer satisfaction.
Design without a goal, just like the traffic problem between cities, we complain about the lack of vision of the builders, and the lack of imagination of software designers. For the conversion of performance benchmarks to key performance goals, use the following examples to illustrate.
There are 200 users using client software for business processing (the concurrency must be at least 200 concurrent), and the total business volume processed through the software each year is 20 million transactions per year.
The following assumptions are used when estimating the test intensity:
-
The business volume of the whole year is concentrated in 10 months, 20 working days per month, 8 hours per working day;
-
Using the 80-20 principle, 80% of the business in each working day is completed within 20% of the time, that is, 80% of the business is completed within 1.6 hours every day;
Estimated result of test pressure:
Approximately 20 million transactions were processed last year, 15% of which required 3 requests to the application server for each transaction; 70% of the services required 2 requests to the application server for each transaction; the remaining 15% of the services Each business submits a request to the application server. According to previous statistics, the annual business increase is 15%. Taking into account the needs of business development in the next three years, the test needs to be carried out at twice the existing business volume.
The total number of requests per year is: (2000*15% 3+2000 70% 2+2000 15%*1)*2="80 million times/year.
The number of requests per day is: 8000/200="400,000 times/day.
The number of requests per second is: (400000 80%)/(8 20%*3600)=55.6 times/second.
Under normal circumstances, the ability of the application server to process requests should be at least 56 times per second.
Or, ignore the number of submitted requests, and define the number of transactions per second, and "throughput" based on the number of business transactions.
If you consider the increase in transaction volume in the next few years (an annual increase of 15%), it can be summarized as:
Throughput: (T4 80%) /(1.6 3600)
-
T4 = T0 * (1 + 15%) ^ 4
-
T0: The current average daily trading volume is 20 million
-
T4: Average daily transaction volume in the next 4 years
In addition, sometimes the determination of key performance indicators is related to specific applications. For example, in the core business system of the financial industry application, there will be batch processing such as daily settlement and monthly settlement, and it is often used how many hours of batch processing is to characterize performance indicators.
2.3 Conversion of metric items and available measurement data items
Only the use of clear and collectable data can truly reflect the performance of the system. E.g:
-
The number of successful transactions per second
-
Response time of a single client (using the difference in timestamp, the time when the request is sent and the time when the response is received)
-
CPU usage
-
Network traffic occupancy rate
-
Memory usage
-
Hard disk usage
2.4 Decomposition of stress
For a complex system composed of many links, if you want to simulate the actual environment for the overall joint performance test, you need to decompose the overall pressure at various levels.
For example, the figure below is a joint performance test of an actual system. The maximum throughput of the background host system is 480 transactions per second. . Since the communication gateway and the host have a one-to-one relationship in this environment, the pressure of the communication gateway must reach 480 pens/sec. A communication gateway corresponds to three front-end processors, so each front-end processor must reach a throughput of 160 transactions per second to the host to make the entire system run at full capacity. Consider other levels and analogy. Since there are other service systems besides the host, an "X" is added to the pressure of the front-end processor to represent the pressure required by other service systems. When there is a shortage of equipment at a certain level and the pressure cannot actually be achieved, it is often necessary to use software means to directly apply pressure on the upper level to solve it.
3. Key points for planning and implementation of performance testing in CS/CSS system architecture
3.1 Human resource plan in the test plan
Because of the more complex types of software testing in the performance test, the human resource plan in the performance test plan is also more important. It is necessary to fully consider the human resource requirements of various tasks such as test organization, test program writing, test design, implementation and execution, test reports, and so on.
Under normal circumstances, the performance testing of an engineering project consists of the following roles and responsibilities:
-
Test Analyst: Responsible for analyzing test strategies, writing test plans, formulating test plans, and organizing tests;
-
Test engineer: design and implement test cases; coordinate the environment; analyze test results and write test summary reports.
-
Tester (usually also a test engineer): test execution; record the test process, collect and organize test record data.
-
Software engineer (sometimes concurrently by a test engineer): Responsible for writing and debugging client test software and simulation software; database management system installation, ofs configuration and system embedded data preparation.
-
System engineer (sometimes concurrently by a test engineer): responsible for hardware maintenance and operating system installation and configuration for testing.
-
Senior test person in charge: responsible for approving the test plan and test summary report.
-
Client: If necessary, you can participate in the test and put forward specific test requirements; you can request to suspend the test; important test conclusions should be signed for approval.
3.2 Choose suitable test tools for the project
In the process of performance testing, there must be suitable testing tools to support.
The testing tools used for performance testing include:
-
Load simulation tools
-
Performance measurement tools
System-level measurement tools: CPU and memory usage statistics
Statistics: Response time, throughput
Profiling class: Code-level measurement tools, such as counting the number of function calls
For testing tools, the requirements of each specific project are different, and there is no general solution. Moreover, the introduction of tools requires investment of time, capital and manpower.
The choice of test tool needs to consider the needs of the system under test in the performance test and the functions that the test tool needs to complete. General options include:
-
Real production system
-
Commercial stress testing tools
-
Custom stress testing tools
The first option is limited to factors such as resources and accuracy, which are generally not used in stress testing, and will not be discussed here. The main considerations for the choice of the latter two options include:
-
Whether it can meet the needs of the stress test as a simulation program and load simulation
-
Can provide detailed and accurate performance measurement data
-
The limiting factors in the cost of testing tools include time and money
-
The test team's mastery of the test tools
There are many good automated performance testing tools. Such as: MI's Loadrunner, MI's Astra LoadTest, Empirix's E-load, Rational TeamTest, etc. Among them, MI's Loadrunner is the most famous and commonly used.
Sometimes in the process of performance testing, self-compiled or customized stress testing tools are also used, mainly based on the following reasons:
First of all, due to the characteristics of the system under test, it is difficult to find commercial test tools that meet the needs of the simulation program. Even if there are test tools close to this demand, taking into account the cost and training time factors, it will increase the risk of the test process.
Secondly, sometimes due to the maturity of related technologies, there are no uncontrollable risks in the design, implementation, or mastery of the test tools when choosing a customized stress test tool; and the timeliness of the test needs can be met at any time during the test process. , Customized testing tools also have unparalleled advantages.
Finally, considering the productization of the front-end system in the future, the need for further testing of the system will continue for a long time. Customized testing tools can better and better meet this demand. At the same time, for projects that use the same system architecture as the target system, you can learn from the idea of this customized testing tool to quickly build a new testing tool.
3.3 Test preparation
The preparation for performance testing can include the preparation of test data, test tool preparation, test environment preparation, and trial execution.
3.3.1 Preparation of test data
For industry applications, especially financial industry applications, the most important thing in the preparation of test data is the selection of transactions. The selection of transactions is based on the following principles:
Internal stress test: Try to select a few transactions that consume the most system resources, and cover all transaction forms (such as conversational, batch, asynchronous, etc.), so that it is possible to maximize the detection of the performance bottleneck of this part;
Overall joint stress test: Generally, the overall joint stress test needs to fully simulate the actual production situation, so the sampling selection of transactions is relatively complicated. It is usually necessary to collect the current transaction volume and predict the performance test transaction volume, and more importantly, determine the distribution of the transaction sending ratio.
For example, the distribution of the proportion of transactions sent for an actual financial project:
First select the top 50 transactions in the actual original sending ratio. Then add up all its proportions (must be less than 1) as the new base, and divide by the original proportions of each transaction respectively to get the proportion of simulated sending using the tool.
In addition, the main actual transaction data is usually extracted from the database using scripts; it may also be necessary to use certain rules to create some data.
3.3.2 Preparation of test tools
Generally, sufficient time should be reserved for the preparation and learning of test tools. For commercial testing tools, it is usually necessary to record and modify scripts, and to set scenes; for self-editing tools, there must be a design and coding process. Moreover, it usually has its own configuration file and data file used, which requires configuration or data format conversion.
3.3.3 Test environment preparation
The problem to be noted here is that embedded test data may need to be generated in the back-end (stub) database to ensure that it is sufficient and reproducible or easy to clean. In addition, after the test environment is prepared, the process of checking the environment before each execution is also very important, which can avoid a lot of useless work.
4. Performance Test Report
4.1 Arrangement and analysis of test structure data
Writing of test report documents
1. The test file that should be generated during the test
Test record (signed by the person in charge of the test and the person who participated in the test);
Test summary report.
2. What must generally be included in the test summary report
The name of the tested software, test items, and test environment;
The conclusion of the stress test of the tested software: response time, maximum/minimum number of concurrency, number of failures, maximum/minimum time of normal continuous operation, the relationship between concurrent number and failure, etc.
In short, it must contain pre-defined key performance index data, change curve analysis and conclusions.
5. Issues that need attention in performance testing under CS/CSS system architecture
5.1 Pay attention to influencing factors such as the license of the middleware and the number of users of the database
Sometimes the test result does not meet the expectations and it is not necessarily the problem of the tested object. It may be caused by insufficient middleware license or the limitation of the number of concurrent users of the database system used. Sometimes it also requires comprehensive analysis due to problems with configuration items. .
5.2 Data aggregation problem
The data selected in the performance test should be dispersed in difference, similar to the random difference distribution of actual production data, and fully test the state of the system under different data. If a single data is used for testing, on the one hand, local functions of the system may not be used, resulting in unreliable performance test data; in addition, if the system has optimized processing functions for the same data processing, the performance test data may also be unreliable.
END
Official account: Programmer Erhei, get software testing resources (interview questions, PDF documents, video tutorials)
Good things should be shared with friends