Use tc to simulate network delay and packet loss under Linux

1. Introduction to analog delayed transmission

netem and tc: netem is a network simulation function module provided by Linux 2.6 and above kernel versions. This function module can be used to simulate complex Internet transmission performance in a local area network with good performance, such as low bandwidth, transmission delay, and packet loss. Many Linux distributions that use the Linux 2.6 (or above) kernel have this kernel feature enabled, such as Fedora, Ubuntu, Redhat, OpenSuse, CentOS, Debian, etc. tc is a tool in the Linux system, the full name is traffic control (flow control). tc can be used to control the working mode of netem. That is to say, if you want to use netem, you need at least two conditions, one is that the netem function in the kernel is included, and the other is to have tc.

It should be noted that the flow control introduced in this article can only control the packet sending action, but not the packet receiving action. At the same time, it takes effect directly on the physical interface. If the physical eth0 is controlled, then the logical network card (such as eth0: 1) will also be affected. Impact, on the contrary, if you do control on the logical network card, the control may be invalid. (Note: Multiple NICs in the virtual machine can be regarded as multiple physical NICs in the virtual machine).

tc qdisc add dev eth0 root netem delay 100ms

//该命令将 eth0 网卡的传输设置为延迟 100 毫秒发送

In a more realistic situation, the delay value will not be so accurate, there will be certain fluctuations, we can use the following situation to simulate the delay value with volatility:

tc qdisc add dev eth0 root netem delay 100ms 10ms

//该命令将 eth0 网卡的传输设置为延迟 100ms ± 10ms (90 ~ 110 ms 之间的任意值)发送

You can further strengthen the randomness of this fluctuation:

tc qdisc add dev eth0 root netem delay 100ms 10ms 30%

//该命令将 eth0 网卡的传输设置为 100ms , 同时大约有 30% 的包会延迟 ± 10ms 发送

2. Simulate network packet loss

tc qdisc add dev eth0 root netem loss 1%

//该命令将 eth0 网卡的传输设置为随机丢掉 1% 的数据包

You can also set the success rate of packet loss:

tc qdisc add dev eth0 root netem loss 1% 30%

//该命令将 eth0 网卡的传输设置为随机丢掉 1% 的数据包, 成功率为 30%

3. Delete the relevant configuration on the network card

Change add in the previous command to del to delete the configuration

tc qdisc del dev eth0 XXXXXX(自己加的配置)

//该命令将 删除 eth0 网卡的相关传输配置

At this point, we can already simulate a certain network delay and packet loss in the test environment through TC. The following are more applications and introductions about tc.

4. Repeat the simulation package

tc qdisc add dev eth0 root netem duplicate 1%

//该命令将 eth0 网卡的传输设置为随机产生 1% 的重复数据包

5. Corruption of simulated data packets

tc qdisc add dev eth0 root netem corrupt 0.2%

//该命令将 eth0 网卡的传输设置为随机产生 0.2% 的损坏的数据包(内核版本需在 2.6.16 以上)

6. Analog data packets are out of order

tc qdisc change dev eth0 root netem delay 10ms reorder 25% 50%

//该命令将 eth0 网卡的传输设置为有 25% 的数据包(50%相关)会被立即发送，其他的延迟10 秒

In the new version, the following commands will also disrupt the order of sending packets to a certain extent:

tc qdisc add dev eth0 root netem delay 100ms 10ms

7. View the configured network conditions

tc qdisc show dev eth0

//该命令将查看并显示 eth0 网卡的相关传输配置

8. Introduction to TC flow control

In Linux, TC has two control methods CBQ and HTB. HTB is designed to replace CBQ. It is a hierarchical filtering framework.

TC includes three basic building blocks: queue discipline qdisc (queueing discipline), class (class) and classifier (Classifiers)

(1) Queuing discipline in TC

Used to achieve the transmission and reception speed of the control network. Through the queue, Linux can cache the network data packets, and then according to the user's settings, smooth network traffic without interrupting the connection (such as TCP) as much as possible.

It should be noted that linux does not control the receive queue well, so we generally only use the send queue, that is, "controlling sending does not control receiving". It encapsulates the other two main TC components (classes and classifiers). If the kernel needs to send a data packet through a network interface, it needs to add the data packet to the queue according to the qdisc (queuing rule) configured for this interface. Then, the kernel will take as many data packets from qdisc as possible and give them to the network adapter driver module.

The simplest QDisc is pfifo. It does not do any processing on incoming data packets. The data packets pass through the queue in a first-in, first-out manner. However, it saves packets that the network interface cannot handle for a while.

Queue rules include FIFO (first-in first-out), RED (random early detection), SFQ (random fair queue) and token bucket (Token Bucket), class-based queue (CBQ), CBQ is a super queue, that is, it can contain Other queues (even other CBQ).

(2) Class in TC

class is used to represent the control strategy. Obviously, many times, we may have to implement different flow control strategies for different IPs. At this time, we have to use different classes to represent different control strategies.

(3) Filter rules in TC

filter is used to classify users into specific control strategies (ie different classes). For example, we want to implement different control strategies (A, B) for the two IPs xxa and xxb. At this time, we can use filter to classify xxa into control strategy A, xxb into control strategy B, and filter division flag Bits can be implemented with u32 marking function or IPtables set-mark (mostly using iptables for marking) function.

At present, the filters that TC can use are: fwmark classifier, u32 classifier, route-based classifier and RSVP classifier (respectively used for IPV6, IPV4), etc. The fwmark classifier allows us to use Linux netfilter code to select traffic, while the u32 classifier allows us to select traffic based on the ANY header. It should be noted that filters are inside QDisc and they cannot be used as subjects.

(4) TC application process

Data packet-> iptables (when passing iptables, iptables sets different mark according to different ip-> TC (class)-> TC (queue)

(5) Application

It is assumed that the eth0 bit is the external network interface of the server. Before starting, first clear all eth0 queue rules

tc qdisc del dev eth0 root 2> /dev/null > /dev/null

1) Define the topmost (root) queue rule and specify the default category number

tc qdisc add dev eth0 root handle 1: htb default 2

2) The definition of the 1: 1 category (speed) of the first layer was originally to define more leaf categories of the second layer, but for now, it is enough in this application.

tc class add dev eth0 parent 1:1 classid 1:2 htb rate 98mbit ceil100mbit prio 2 
tc class add dev eth0 parent 1:1 classid 1:3 htb rate 1mbit ceil 2mbit prio 2

Note: The above is the speed we control the output server, one is 98M, one is 2M.

rate: is the bandwidth value guaranteed by a class. If there is more than one class, to ensure that the sum of all subclasses of the parent class is less than or equal
prio: competitive when used to indicate the bandwidth borrowing, PRIO smaller, the higher priority, the more competitive
ceil: a class is the maximum energy The resulting bandwidth value

At the same time, in order not to make a session permanently occupy bandwidth, the random fair queue sfq is added.

tc qdisc add dev eth0 parent 1:2 handle 2: sfq perturb 10 
tc qdisc add dev eth0 parent 1:3 handle 3: sfq perturb 10

3) Set filter

The filter can use its own u32 or iptables to mark it.
Specified in the root class 1: 0, the 192.168.0.2 filter uses the 1: 2 rule to give him a speed of 98M. The writing is as follows:

tc filter add dev eth0 protocol ip parent 1:0 u32 match ip src 192.168.0.2 flowid 1:2
tc filter add dev eth0 protocol ip parent 1:0 u32 match ip src 192.168.0.1 flowid 1:3

If it is all ip written as follows:

tc filter add dev eth0 protocol ip parent 1: prio 50 u32 match ip dst 0.0.0.0/0 flowid 1:10

//使用 Iptables 来配合过滤器

You can also use this method, but you need to use the following iptables command to mark

tc filter add dev eth0 parent 1: protocol ip prio 1 handle 2 fw flowid 1:2 
tc filter add dev eth0 parent 1: protocol ip prio 1 handle 2 fw flowid 1:3

iptables only needs to be marked

iptables -t mangle -A POSTROUTING -d 192.168.0.2 -j MARK --set-mark 10 iptables -t mangle -A POSTROUTING -d 192.168.0.3 -j MARK --set-mark 20

(6) TC controls the highest speed

Rate ceiling The rate limit
parameter ceil specifies the maximum bandwidth that a class can use to limit how much bandwidth the class can borrow. The default ceil is the same as the rate.

This feature is useful for ISPs because they generally limit the total number of users being served even if other users do not request service. (ISPS wants users to pay more for better service). Note that root classes are not allowed to be borrowed, so ceil is not specified.

Note: The value of ceil should be at least as high as the rate of the class it belongs to, that is to say, ceil should be at least as high as any of its subclasses.

(7) Burst burst

Network hardware can only send one packet at a time. It only depends on the rate of one piece of hardware. Link sharing software can use this capability to dynamically generate multiple connections running at different speeds. So rate and ceil are not an instant measure, just an average of packets sent at a time. The actual situation is how to make a class with a small flow rate provide other classes with the maximum rate at a certain time.

The burst and cburst parameters control how much data can be easily sent to other classes at the maximum speed of the hardware. If cburst is less than a theoretical data packet, the burst formed by it will not exceed the ceil rate, so is the maximum rate of TBF in the same method.

You may ask why bursts are needed. Because it can easily increase the response speed on a very crowded link. For example, WWW traffic is bursty. You visit the homepage to get and read suddenly, and burst will "charge" again in your free time.

Note: burst and cburst must be at least as large as the values of their subclasses.

(8) TC command format

Join

tc qdisc [ add | change | replace | link ] dev DEV [ parent qdisc-id | root ] [ handle qdisc-id ] qdisc[ qdisc specific parameters ]
tc class [ add | change | replace ] dev DEV parent qdisc-id [ classid class-id ] qdisc [ qdisc specific parameters ]
tc filter [ add | change | replace ] dev DEV [ parent qdisc-id | root ] protocol protocol prio priorityfiltertype [ filtertype specific parameters ] flowid flow-id

display

tc [-s | -d ] qdisc show dev DEV 
tc [-s | -d ] class show dev DEV 
tc filter show dev DEV

View the status of TC

tc -s -d qdisc show dev eth0
tc -s -d class show dev eth0

Delete tc rule

tc qdisc del dev eth0 root

Examples

1) Use TC download to limit a single IP for speed control

tc qdisc add dev eth0 root handle 1: htb r2q 1 
tc class add dev eth0 parent 1: classid 1:1 htb rate 30mbit ceil 60mbit 
tc filter add dev eth0 parent 1: protocol ip prio 16 u32 match ip dst 192.168.1.2  flowid 1:1

You can limit the download speed of 192.168.1.2 to 30Mbit up to 60Mbit, where r2q refers to the root without default, so that the bandwidth of the entire network is not limited.

2) Use TC to control the speed of the entire IP

tc qdisc add dev eth0 root handle 1: htb r2q 1 
tc class add dev eth0 parent 1: classid 1:1 htb rate 50mbit ceil 1000mbit 
tc filter add dev eth0 parent 1: protocol ip prio 16 u32 match ip dst 192.168.111.0/24 flowid 1:1

You can limit the bandwidth of 192.168.111.0 to 255 to 3000k, and the actual download speed is about 200k. In this case, all machines on this segment share the 200k bandwidth.

You can also join an sfq (random fair queue)

tc qdisc add dev eth0 root handle 1: htb r2q 1 
tc class add dev eth0 parent 1: classid 1:1 htb rate 3000kbit burst 10k 
tc qdisc add dev eth0 parent 1:1 handle 10: sfq perturb 10 
tc filter add dev eth0 parent 1: protocol ip prio 16 u32 match ip dst 192.168.111.168 flowid 1:1

sfq, he can prevent an ip in a segment from occupying the entire bandwidth.

3) Use TC to control the external speed of the server to 10M

As follows, I want to manage a server, can only send out 10M data

tc qdisc del dev eth0 root 
tc qdisc add dev eth0 root handle 1:htb 
tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit ceil 100mbit 
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 10mbit ceil 10mbit 
tc qdisc add dev eth0 parent 1:10 sfq perturb 10
tc filter add dev eth0 protocol ip parent 1: prio 2u32 match ip dst 220.181.xxx.xx/32 flowid 1:1

The above, let 220.181.xxx.xx / 32 run the default, mainly to make this ip connection not controlled

tc filter add dev eth0 protocol ip parent 1: prio 50 u32 match ip dst 0.0.0.0/0 flowid 1:10

By default, all traffic will pass through this

Reference source: http://blog.csdn.net/weiweicao0429/article/details/17578011

Old manon

Published 13 original articles · Likes6 · Visitors 10,000+

Private letter concerns