I. Background and purpose test
In the beginning the demand for research, testers need to clear the purpose of the test, then the project itself must first clear case, there will be different for different purposes of the project, depending on the project can generally be divided into the following six cases:
1, completely new system
Completely new system means that the system does not test business data as a reference, there is no business personnel can effectively be estimated volume of business, you can not convert the traffic index value (TPS / QPS), then we usually test for the following purposes:
- Best test application system processing power and extreme processing power for the development and assessment of the demand side.
- Test application system is stable
- Test application system is reliable
- Test application system bottlenecks, and provide advice and other expansion
2, transformation of the new system
The transformation of the new system in general can be analyzed through the old system, the volume of business, combined with operational staff evaluated based on business developments. Testing purposes generally as follows:
- Test application system meets the current traffic demand.
- Test application system meets traffic requirements N years later
- Test application system is stable
- Test application system is reliable
- Test application system bottlenecks, and provide advice and other expansion
3, system upgrade
Upgrade system may be upgraded for defect repair, there may be changes to upgrade your business needs. Not done before such as performance testing, the new system set up in accordance with the transformation of the way purposes. As already done performance typically tested for the following purposes:
- Best test applications or extreme processing power and before the test results were analyzed.
- Test application system is stable
4, operation of the system online
System requirements to run online non-functional testing is generally encountered online non-functional testing defect, it is difficult to reproduce. Usually the test object as follows:
- Simulate the production environment operation, non-functional testing to find flaws with the development of optimized retest.
5, the procurement systems
Procurement systems require non-functional testing is generally a number of manufacturers of products for testing, comparison of test results are given, usually for testing purposes as follows:
Obtaining a fixed number of concurrent users, manufacturers comparison of the index (the TPS, response times, CPU usage, error rate, etc.)
6, the hardware and software selection
Select non-functional testing hardware and software, mainly in the case of formula keeping the other cases, performance comparison of the situation.
Acquiring different hardware and software processing capability, according to the service circumstances, select the highest cost-effective hardware and software.
Second, the scope of the test confirmed
Test range mainly in the research process, the system under test to confirm the relationship with neighboring systems to graphically show the way, and at a significant mark labeling system under test and the way the text description of the transaction examined specific and which systems interact.
Third, the test system architecture analysis
The main object of analysis architecture is required to confirm the language used to develop test systems / development framework, communication mechanisms / protocols, the middleware, the database, whether to adopt the cluster application, whether using a database cluster, whether there is backup mechanism (Hot Standby / cold standby) , load balancing mechanism. To set a specific follow-up testing strategy, environment deployment, monitoring means laying the groundwork.
Fourth, business model analysis
Analysis of the main business model in order to get a more realistic simulation is run online, thus ensuring the test coverage. By reference into the service data (production logging operation) and no service data according to the system in both cases the reference
Fifth, there is a reference data service
Service data may be analyzed with reference to the system of extracting business model as follows:
Examples
| Case No. | Interface name |
Transactions accounting |
| 1 |
create_instant_trade (Prompt arrival) |
6% |
| 2 |
create_ensure_trade (secured transactions) |
5% |
| 3 |
create_split_trade (split) |
3% |
| 4 |
create_settle (settlement) |
5% |
| 5 |
create_refund (refund) |
0.5% |
| 6 | query_trade (transaction inquiries) | 80.5% |
Sixth, no business data
Communication with business people or developers, to guide business people based on business use case, probably projected business model. Specific methods are as follows
- Confirmation test business transactions, business transactions selection rule as follows:
- More frequent use of the transaction (business people provided based on usage)
- But frequent use is not particularly large amount of data involved in the transaction transaction (business people with developers based on usage provided)
- Trading complex logic high transaction (developers according to code logic)
- Recognizing the business transaction accounted for:
- Business people estimated the proportion of transactions each selected according to usage
- If you do not use the service, you can refer to peer (non-functional testers and make recommendations based on the relevant conduct of business confirmed)
Seven, user analysis
| Demand category | Demand Highlights | Performance specification requirements described |
|
User analysis |
The number of users demand |
The number of each category of users of the system analysis |
| User demand category | ||
| The system supports a maximum number of users concurrent operation | Refers to the number of concurrent users at the same time, the number of users log into the system and business operations. | |
| The maximum number of users supported by the system connection | The system is able to access (or log) at the same time the maximum number of connected users. In general, not all access (or log) real users in real-time operation, the part of the user access (or log) system, temporarily not operational, so that users accustomed to the operation requirements of the system under test to provide additional concurrent access capability of the system. |
Eight, test indicators
Performance tests commonly used indicators confirm the following:
| Categories |
Index entry |
Index values |
Remark |
| Business class |
System capacity (TPS) |
Pareto rule |
The peak of TPS |
| General transaction response time |
<3 seconds |
To-end response time |
|
| Complex transaction response time |
<5 seconds |
For a large table query transaction |
|
| Transaction success rate |
>=99.9% |
|
|
| system resource |
CPU usage |
<=80% |
|
| Memory Usage |
|
No clear upward trend. |
|
| stability |
operation hours |
24 hours |
CPU usage, memory usage, disk busy rate is relatively stable, no significant fluctuations. |
- System capacity (TPS) analysis methods:
Communication and estimated peak day trading volume of S1 (unit: T), to obtain the peak time period S2 (unit: hour), if needed to meet the operational capacity N years later, the business needs to be given a hundred percent increments S3.
Capability to meet the current service processing time T1 (unit: Pen / sec) = (S1 * 0.8) / (S2 * 0.2 * 36000);
N satisfy the service processing capability of T2 (unit: Pen / sec) = T1 * (1 + S3) ^ N;
- Response time obtain:
Response time indicators are generally divided into classes based on the system mode interfaces, client access category, a specific time index and client access classes are according to the service by the service personnel, the specified response time,
Typically standard: 1-3 seconds through; 3-5 seconds by cautious; can not be more than 5 seconds. Response time generally interface class criteria: billing categories: single query 200 ms: 500 ms
Nine, the amount of embedded data analysis
The amount of data object embedded data is embedded according to the production conditions and traffic database table data storage, authentication, database performance on their business transaction performance data in the case where a certain amount. Generally divided into a lead from its own configuration data and production data.
Production derivative desensitization mainly to production data into the test database, such data more realistic.
The main structure is based on the number of their own data multiplied by the amount of daily trading volume of the storage time table or statistical line mode (need to know about the transaction involves table data storage), the basic data structure, data types, and should meet the requirements of the order, to avoid data hotspot
query_balance
| can | service | database | 数据表 | 操作 | 线上数据量 | 当前测试环境 |
|
下单 |
mag | |||||
| ma-web | member库 | m_member_identity 会员查询 member_account 会员账户信息表 |
query | 5320w | 3053w | |
| dpm | dpm库 | dpm_outer_account 外部户
|
query | 10450w | 5735w | |
| vouch | vouch库 | trade_vouch 交易凭证 | insert | 11559w | 2722w | |
| tss | tss库 | trade_order 交易订单 acq_trade_order_ext 收单交易扩展 split_party分账信息表 |
insert | 9152w 10670w 3181w
|
2491W 3200W 3260w
|