Partial overview of web performance testing In general, the processing of a web request consists of the following steps:
(1) The client sends a request
(2) The web server receives the request and processes it;
(3) The web server obtains data from the DB;
(4) The webserver generates the user's object (page) and returns it to the user. The time from the beginning of sending the request to the client to the last byte is called the response time (the third step is not included in each request processing).
1. Transaction
In the web performance test , a transaction represents a process of "sending a request from the user -> the web server receives the request and processes it -> the web server obtains data from the DB -> generates the user's object (page) and returns it to the user", Typical response times are for transactions.
2. Request response time
The request response time refers to the time from a request initiated by the client to the end of the client receiving the response returned from the server. In some tools, the response is usually called "TTLB". That is, "time to last byte", which means the time from when a request is initiated to when the client receives the last byte of the response. The unit of response time is generally "seconds" or "milliseconds". A formula can express: response time = network response time + application response time. The standard can refer to the foreign 3/5/10 principle:
(1) Within 3 seconds, the page responds to the user and displays it, which can be considered as "very good";
(2) Within 3 to 5 seconds, the page responds to the user and displays it, which can be considered "good";
(3) Within 5~10 seconds, the page responds to the user and displays it, which can be considered as "reluctantly acceptable";
(4) More than 10 seconds will make people a little impatient, and the user is likely not to continue to wait;
3. Transaction response time
A transaction may be composed of a series of requests. The response time of a transaction is mainly aimed at users. It is a macro concept and is proposed to explain the business response time to users . For example , the response time of an inter-bank withdrawal transaction is determined by a series of The request is composed of . The transaction response time is a parameter that directly measures the performance of the system .
4. Number of concurrent users
Concurrency is generally divided into two situations. One is concurrency in the strict sense, that is, all users do the same thing or operation at the same time, which generally refers to doing the same type of business. For example, in the credit card approval business, a certain number of advocates submit the completed approval business at the same time; there is also a special case, that is, all users perform exactly the same operation, for example, in the credit card approval business, all users can apply together business, or modify the same record.
Another type of concurrency is broad scope concurrency. The difference between this concurrency and the previous concurrency is that although multiple users have issued requests or performed operations on the system, these requests or operations can be the same or different. For the whole system, there are still many users operating the system at the same time, so it also belongs to the category of concurrency.
It can be seen that the latter concurrency includes the former concurrency. And the latter kind of concurrency is closer to the actual usage of users, so for most systems, only a small number of users are " strictly concurrent". For WEB performance testing, these two concurrency situations generally need to be tested, and the usual practice is to perform concurrency testing in a strict sense first. User concurrency in the strict sense generally occurs in modules that are used more frequently. Although the probability of occurrence is not very large, once a performance problem occurs, the consequences are likely to be fatal. Concurrency testing in the strict sense is often associated with functional testing , because exceptions encountered in concurrent functions are usually program problems, and this testing is also part of robustness and stability testing.
Number of concurrent users: There are 2 common misconceptions about the number of concurrent users. A wrong view is to interpret the number of concurrent users as the total number of users using the system, on the grounds that these users may use the system at the same time; there is a more correct view that the number of online users is understood as the number of concurrent users. In fact, online users are not necessarilyconcurrent with other users. For example, users who are browsing web pages have no impact on the server . However, the number of online users is one of the main criteria for calculating the number of concurrent users.
5. Throughput
Refers to the total amount of data transmitted on the network during a performance test. Throughput/transmission time is the throughput rate.
6、 TPS（transactionper second）
The number of transactions or transactions that the system can process per second. It is an important indicator to measure the processing power of the system.
7. CTR
The number of HTTP requests submitted by users to the WEB server per second . This indicator is a unique indicator of WEB applications: WEB applications are in the "request-response" mode. Once a user sends an application, the server will process it once, so clicking is a WEB application can The smallest unit of transactions processed. If each click is defined as a transaction, the CTR and TPS are a concept. It is easy to see that the higher the CTR, the greater the pressure on the server. The CTR is only a performance reference indicator, important is to analyze the impact of clicks. It should be noted that the click here does not refer to a single-click operation of the mouse, because in a single-click operation, the client may issue multiple HTTP requests to the server.
8. Resource utilization
Refers to the degree of use of different system resources, such as server CPU utilization, disk utilization, etc. Resource utilization is the main basis for analyzing system performance indicators and improving performance, so it is the focus of WEB performance testing .
Resource utilization is mainly aimed at WEB servers, operating systems , database servers, networks, etc., and is the main reference for testing and analyzing bottlenecks. In the WEB performance test, the corresponding parameters are collected and analyzed as needed.
Common indicators (referring to the required test items for Web application servers and database servers)
| index |
illustrate |
| ProcessorTime | When the server CPU utilization rate reaches 70% on average, the service is close to saturation. |
| Memory Available Mbyte | The number of available memory, if the memory is found to change during the test, pay attention to it, if it is a memory leak, it is more serious |
| Physicsdisk Time | Physical disk read and write time |
Web server metrics
| index |
illustrate |
| Requests Per Second（Avg Rps） | Average number of responses per second = total request time / seconds |
| Avg time to last byte per terstion （mstes） | The average number of iterations of business scripts per second, some people will confuse the above |
| Successful Rounds | successful request |
| Failed Requests | failed request |
| Successful Hits | successful clicks |
| Failed Hits | Failed Clicks |
| Hits Per Second | hits per second |
| Successful Hits Per Second | Successful clicks per second |
| Failed Hits Per Second | Failed hits per second |
| Attempted Connections | number of links attempted |
Database Server Performance Metrics
| index |
illustrate |
| User 0 Connections | The number of user connections, that is, the number of connections to the database |
| Number of deadlocks | 数据库死锁 |
| Butter Cache hit | 数据库Cache的命中情况 |
系统的瓶颈定义
| 性能项 |
命令 |
指标 |
| CPU限制 | vmstat | 当%user+%sys超过80%时 |
| 磁盘I/O限制 | Vmstat | 当%iowait超过40%(AIX4.3.3或更高版本)时 |
| 应用磁盘限制 | Iostat | 当%tm_act超过70%时 |
| 虚存空间少 | Lsps，-a | 当分页空间的活动率超过70%时 |
| 换页限制 | Iostat, stat | 虚存逻辑卷%tm_act超过I/O(iostat)的30%，激活的虚存率超过CPU数量(vmstat)的10倍时 |
| 系统失效 | Vmstat, sar | 页交换增大、CPU等待并运行队列 |
稳定系统的资源状态
| 性能项 |
资源 |
评价 |
| CPU占用率 | 70% | 好 |
| 85% | 坏 | |
| 90%+ | 很差 | |
| 磁盘I/0 | <30% | 好 |
| <40% | 坏 | |
| <50%+ | 很差 | |
| 网络 | <30%带宽 | 好 |
| 运行队列 | <2*CPU数量 | 好 |
| 内存 | 没有页交换 | 好 |
| 每个CPU每秒10个页交换 | 坏 | |
| 更多的页交换 | 很差 |
通俗理解：
日访问量
常用页面最大并发数
同时在线人数
访问相应时间
案例：
最近公司一个项目，是个门户网站，需要做性能测试，根据项目特点定出了主要测试项和测试方案：
一种是测试几个常用页面能接受的最大并发数(用户名参数化，设置集合点策略)
一种是测试服务器长时间压力下，用户能否正常操作(用户名参数化，迭代运行脚本)
一种则需要测试服务器能否接受10万用户同时在线操作，如果是用IIS做应用服务器的话，单台可承受的最大并发数不可能达到10万级，那就必须要使用集群，通过多台机器做负载均衡来实现；如果是用websphere之类的应用服务器的话，单台可承受的最大并发数可以达到10万级，但为性能考虑还是必须要使用集群，通过多台机器做负载均衡来实现；通常有1个简单的计算方式，1个连接产生1个session，每个session在服务器上有个内存空间大小的设置，在NT上是3M，那么10万并发就需要300G内存，当然实际使用中考虑其他程序也占用内存，所以准备的内存数量要求比这个还要多一些。还有10万个用户同时在线，跟10万个并发数是完全不同的2个概念。这个楼上已经说了。但如何做这个转换将10万个同时在线用户转换成多少个并发数呢？这就必须要有大量的历史日志信息来支撑了。系统日志需要有同时在线用户数量的日志信息，还需要有用户操作次数的日志信息，这2个数据的比例就是你同时在线用户转换到并发数的比例。另外根据经验统计，对于1个JAVA开发的WEB系统（别的我没统计过，给不出数据），一般1台双CPU、2G内存的服务器上可支持的最大并发数不超过500个（这个状态下大部分操作都是超时报错而且服务器很容易宕机，其实没什么实际意义），可正常使用（单步非大数据量操作等待时间不超过20秒）的最大并发数不超过300个。假设你的10万同时在线用户转换的并发数是9000个，那么你最少需要这样的机器18台，建议不少于30台。当然，你要是买个大型服务器，里面装有200个CPU、256G的内存，千兆光纤带宽，就算是10万个并发用户，那速度，也绝对是嗖嗖的。
另外暴寒1下，光设置全部进入运行状态就需要接近6个小时。具体的可以拿1个系统来压一下看看，可能会出现以下情况：
1、服务器宕机；
2、客户端宕机；
3、从某个时间开始服务器拒绝请求，客户端上显示的全是错误；
4、勉强测试完成，但网络堵塞或测试结果显示时间非常长。假设客户端和服务器之间百兆带宽，百兆/10000=10K，那每个用户只能得到10K，这个速度接近1个64K的MODEM上网的速度；另外以上分析全都没考虑系统的后台，比如数据库、中间件等。
1、服务器方面：上面说的那样的PC SERVER需要50台；
2、网络方面：按每个用户50K，那至少5根百兆带宽独享，估计仅仅网络延迟就大概是秒一级的；
3、如果有数据库，至少是ORACLE，最好是SYSBASE，SQLSERVER是肯定顶不住的。数据库服务器至少需要10台4CPU、16G内存的机器；
4、如果有CORBA，那至少再准备10台4CPU、16G内存的机器；再加上负载均衡、防火墙、路由器和各种软件等，总之没个1000万的资金投入，肯定搞不定。
这样的门户系统，由于有用户权限，所以并不象jackie所说大多是静态页面。但只要是多服务器的集群，那么我们就可以通过1台机器的测试结果来计算多台机器集群后的负载能力的，最多额外考虑一下负载均衡和路由上的压力，比如带宽、速度、延迟等。但如果都是在1台机器上变化，那我们只能做一些指标上的计算，可以从这些指标上简单判断一下是否不可行，比如10万并发用户却只有1根百兆带宽，那我们可以计算出每个用户只有1K带宽，这显然是不可行的。但实际的结果还是需要测试了才知道，毕竟系统压力和用户数量不是线性变化的。
这一类系统的普遍的成熟的使用，以及很多软件在方案设计后就能够大致估算出系统的性能特点，都导致了系统在软件性能方面调优的比例并不大（当然不完全排除后期针对某些代码和配置进行优化后性能的进一步提高），更多的都是从硬件方面来考虑，比如增加内存、硬盘做RAID、增加带宽、甚至增加机器等。
网络技术中的10M 带宽指的是以位计算， 就是 10M bit /秒 ，而下载时的速度看到的是以字节（Byte）计算的，所以10M带宽换算成字节理论上最快下载速度为： 1.25 M Byte/秒!