Outline
Performance testing, the indicators for monitoring the server is also very important aspect. By monitoring the server performance analysis, you can locate performance bottlenecks.
Back-end performance indicators CPU, memory, network, I / O, etc.
analysis of idea
- Overall system CPU utilization
- Memory Usage
- Disk I / O utilization and latency
- Network utilization
CPU Positioning Analysis
CPU utilization greater than 50%, need to pay attention; more than 70%, need to pay close attention; more than 90%, the situation is more serious.
Monitoring command: vmstat, sar, dstat, mpstat, top, ps
| Types of | Measurement Methods | Metrics |
| Utilization | 1, vmstat statistics 1-% idle 2, sar -u statistical 1-% idle 3, dstat statistical 1-% idl 4, mpstat -P ALL statistics 1-% idle |
Note> = 50% Alarm> = 70% Serious> = 90% |
| Full | 1, vmstat the r values> cpu logical number of stars 2, SAR -q, " the runq-SZ "> logical CPU number of stars
|
When the run queue is greater than 1, it has demonstrated a certain load |
Memory location analysis
When the physical memory is not enough, it will use the swap partition, so the performance testing process need to focus on the use of swap and mem.
Not enough physical memory, replaced a lot of memory to swap space, and CPU bottlenecks could lead to I / O's.
Monitoring command: vmstat, sar, dstat, free, top, ps, etc.
| Types of | Measurement Methods |
Labeling measure |
| Occupancy rate | 1, free view usage 2、vmstat 3 sar -r 4、ps |
Note> = 50% Alarm> = 70% Serious> = 80% |
| Full | 1, vmstat the si / so ratio, swapd accounting 2, sar -W view times missing pages 3、dmesg | grep killed |
1, so a large value, and has high proportion swapd, memory has been saturated 2, sar command times mean missing pages and more memory is saturated 3, not enough memory OOM mechanism will trigger the kernel |
Network location analysis
Monitoring command: sar, ifconfig, netstat, and dev rate view of the net.
通过查看发现收发包的吞吐率达到网卡的最大上限，网络数据报文有因为这类原因而引起的丢包、阻塞等现象都证明当前网络可能存在瓶颈。
为了减小网络对性能测试的影响，一般我们都在局域网中进行测试执行。
| 类型 | 度量方法 | 衡量标准 |
| 使用情况 | 1、sar -n DEV 的收发计数大于网卡上限 2、ifconfig RX/TX宽带超过网卡上限 3、cat /proc/net/dev的速率超过上限 4、nicstat的util基本满负荷 |
1、收发包的吞吐率达到网卡上限 2、有延迟 3、有丢包 4、有阻塞 |
| 满载 | 1、ifconfig dropped 有计数 2、netstat -s "segments retransmited"有计数 3、sar -n EDEV，rxdrop/s txdrop/s有计数 |
有丢包统计 |
| 错误 | 1、ifconfig，“errors” 2、netstat -i，RX-ERR TX-ERR 3、sar -n EDEV，rxerr/s txerr/s 4、ip -s link， “errors” |
错误有计数 |
IO定位分析
I/O读写频繁的时候，如果I/O得不到满足会导致应用的阻塞。
需要考虑I/O的TPS、平均I/O数据、平均队列长度、平均服务时间、平均等待时间、IO利用率（磁盘Busy Time%）等指标
监控命令：sar、iostat、iotop
| 类型 | 度量方法 | 衡量标准 |
| 使用情况 | 1、iostat -xz，“%util” 2、sar -d，“%util” 3、cat /proc/pid/sched | grep iowait |
注意>=40% 告警>=60% 严重>=80% |
| 满载 | 1、iostat -xnz，“avgqu-sz ”>1 2、iostat await>70 |
IO疑似满载 |
| 错误 | 1、dmseg 查看io错误 2、smartctl /dev/sda |
有错误信息 |