iperf3 specified network card test --bind-dev parameter details

The latest version 3.13 of iperf3 has been installed, and it is found that there is an additional parameter support --bind-dev. Those who pay attention to this parameter will know that there is a -B parameter in the old version of iperf3, which can specify which IP to use for testing on a multi-IP host. This blog has a specific analysis, see iPerf3 -B parameter detailed graphic analysis

1. New and old versions of iperf3 (iperf3 3.13 vs. iperf3 3.9) parameter update

• The description of the -B parameter in the old version:
  select the sending port by specifying the ip address, but we can know from the above detailed analysis that this -B can only specify which IP address to send the message through, and cannot pass this parameter Change the physical network card that sends the message, which physical network card the host uses to send data, ultimately depends on the routing table of the host (this problem exists for the host with multiple network cards belonging to the same IP subnet) . If it is necessary to change the physical network card for sending, we also need to change the matrix setting of the routing table entry to ensure that the packet is sent out from the specified port.

       -B, --bind host
              bind to the specific interface associated with address host.

• Instructions for the -B parameter and the newly added –bind-dev in the new version:
  The new version adds parameters for specifying the physical network card. There are two ways:
• Through --bind-dev followed by a network card name (the name of the network card displayed by ifconfig), explicitly specify which physical network card to send from,
• Through -B hostIP[%dev] or --bind hostIP[%dev], use the optional parameters in [] to specify.

       -B, --bind host[%dev]
              bind  to the specific interface associated with address host.  If an optional interface is specified, it is treated as a shortcut for --bind-dev dev.  Note that a percent sign and inter‐
              face device name are required for IPv6 link-local address literals.

       --bind-dev dev
              bind to the specified network interface.  This option uses SO_BINDTODEVICE, and may require root permissions.  (Available on Linux and possibly other systems.)

2. Look directly at the conclusion

If you don't want to read the detailed process, you can directly read the following conclusions:

1. Specifying a physical network card allows the host to send data from the specified physical network card
2. After specifying the physical network card, it will automatically use the IP address corresponding to the physical network card to send

3, --bind-dev dev detailed explanation

1. Let's take a look, this machine has two network cards, the IP addresses belonging to the same subnet are eno1 and enp2s0, and the corresponding IP addresses are 192.168.3.238 and 192.168.3.107 respectively. Before starting the test, the packets of TX were 1.2G and 370.4G respectively.
2. Use the iperf3 --bind-dev command to send a package of 1.09Gbytes through enp2s0 for testing.
   After the test is complete, use the ifconfig command again to check. You can see that the TX data transmission volume of the eno1 network card remains unchanged at 1.2GB, while the TX data of enp2s0 The amount sent increased from 370.4G to 371.7G, indicating that the data was sent from the enp2s0 network card.
3. Conversely, through the iperf3 --bind-dev command, eno1 sends a package of 1.09Gbytes for testing, and we will find that: After the test is completed, use the ifconfig command to check again, and you can see that the TX data transmission volume of the enp2s0 network card remains at 371.7GB There is no change, but the TX data transmission volume of eno1 increases from 1.2G to 2.4G, indicating that the data is sent from the eno1 network card.

In conclusion, if you directly specify the physical network card, you can automatically switch to the IP address of the specified physical network card for sending .

xxx@xxx-pc:~/iperf3/iperf$ ifconfig
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.238  netmask 255.255.255.0  broadcast 192.168.3.255
        inet6 fe80::a6fc:b416:ea9c:df2f  prefixlen 64  scopeid 0x20<link>
        ether 8c:dc:d4:41:8b:ac  txqueuelen 1000  (以太网)
        RX packets 369667  bytes 26117275 (26.1 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 807538  bytes 1224949754 (1.2 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 20  memory 0xf7f00000-f7f20000

enp2s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.107  netmask 255.255.255.0  broadcast 192.168.3.255
        inet6 fe80::98ed:685c:da1d:183e  prefixlen 64  scopeid 0x20<link>
        ether a0:36:9f:56:95:be  txqueuelen 1000  (以太网)
        RX packets 7713629  bytes 1981189364 (1.9 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 248973591  bytes 370498351533 (370.4 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0xf7d00000-f7dfffff

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (本地环回)
        RX packets 17648  bytes 2096944 (2.0 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 17648  bytes 2096944 (2.0 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

xxx@xxx-pc:~/iperf3/iperf$ /usr/local/bin/iperf3 -c 192.168.3.60 --bind-dev enp2s0

我们从服务端可以看到，收到来自己192.168.3.107的1.09G的报文。
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.3.107, port 47464
[  5] local 192.168.3.60 port 5201 connected to 192.168.3.107 port 47476
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  92.6 MBytes   776 Mbits/sec
[  5]   1.00-2.00   sec   112 MBytes   940 Mbits/sec
[  5]   2.00-3.00   sec   112 MBytes   940 Mbits/sec
[  5]   3.00-4.00   sec   112 MBytes   940 Mbits/sec
[  5]   4.00-5.00   sec   112 MBytes   940 Mbits/sec
[  5]   5.00-6.00   sec   112 MBytes   940 Mbits/sec
[  5]   6.00-7.00   sec   112 MBytes   940 Mbits/sec
[  5]   7.00-8.00   sec   112 MBytes   941 Mbits/sec
[  5]   8.00-9.00   sec   112 MBytes   939 Mbits/sec
[  5]   9.00-10.00  sec   112 MBytes   941 Mbits/sec
[  5]  10.00-10.17  sec  19.5 MBytes   941 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.17  sec  1.09 GBytes   924 Mbits/sec                  receiver
-----------------------------------------------------------

xxx@xxx-pc:~/iperf3/iperf$ ifconfig
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.238  netmask 255.255.255.0  broadcast 192.168.3.255
        inet6 fe80::a6fc:b416:ea9c:df2f  prefixlen 64  scopeid 0x20<link>
        ether 8c:dc:d4:41:8b:ac  txqueuelen 1000  (以太网)
        RX packets 371228  bytes 26246399 (26.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 807585  bytes 1224955508 (1.2 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 20  memory 0xf7f00000-f7f20000

enp2s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.107  netmask 255.255.255.0  broadcast 192.168.3.255
        inet6 fe80::98ed:685c:da1d:183e  prefixlen 64  scopeid 0x20<link>
        ether a0:36:9f:56:95:be  txqueuelen 1000  (以太网)
        RX packets 8073124  bytes 2004918105 (2.0 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 249795878  bytes 371728640852 (371.7 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0xf7d00000-f7dfffff

xxx@xxx-pc:~/iperf3/iperf$ /usr/local/bin/iperf3 -c 192.168.3.60 --bind-dev eno1

我们从服务端可以看到，收到来自己192.168.3.238的1.09G的报文。
Accepted connection from 192.168.3.238, port 45136
[  5] local 192.168.3.60 port 5201 connected to 192.168.3.238 port 45142
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  95.5 MBytes   801 Mbits/sec
[  5]   1.00-2.00   sec   111 MBytes   934 Mbits/sec
[  5]   2.00-3.00   sec   111 MBytes   934 Mbits/sec
[  5]   3.00-4.00   sec   111 MBytes   934 Mbits/sec
[  5]   4.00-5.00   sec   111 MBytes   934 Mbits/sec
[  5]   5.00-6.00   sec   111 MBytes   934 Mbits/sec
[  5]   6.00-7.00   sec   111 MBytes   934 Mbits/sec
[  5]   7.00-8.00   sec   111 MBytes   934 Mbits/sec
[  5]   8.00-9.00   sec   111 MBytes   934 Mbits/sec
[  5]   9.00-10.00  sec   111 MBytes   934 Mbits/sec
[  5]  10.00-10.14  sec  15.9 MBytes   934 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.14  sec  1.09 GBytes   921 Mbits/sec                  receiver


xxx@xxx-pc:~/iperf3/iperf$ ifconfig
demo-oai: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.70.129  netmask 255.255.255.192  broadcast 192.168.70.191
        ether 02:42:11:7b:67:e4  txqueuelen 0  (以太网)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:bf:65:22:bd  txqueuelen 0  (以太网)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.238  netmask 255.255.255.0  broadcast 192.168.3.255
        inet6 fe80::a6fc:b416:ea9c:df2f  prefixlen 64  scopeid 0x20<link>
        ether 8c:dc:d4:41:8b:ac  txqueuelen 1000  (以太网)
        RX packets 726265  bytes 51096987 (51.0 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1614509  bytes 2449795540 (2.4 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 20  memory 0xf7f00000-f7f20000

enp2s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.107  netmask 255.255.255.0  broadcast 192.168.3.255
        inet6 fe80::98ed:685c:da1d:183e  prefixlen 64  scopeid 0x20<link>
        ether a0:36:9f:56:95:be  txqueuelen 1000  (以太网)
        RX packets 8080367  bytes 2005367711 (2.0 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 249808572  bytes 371729998079 (371.7 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0xf7d00000-f7dfffff

4. -B hostIP[%dev] or --bind hostIP[%dev]

It seems that the binding relationship between the specified IP address and the physical network card will be checked, and an error will occur if it is not the same. (I haven’t analyzed why the error occurs, because when [%dev] is not specified, the host program will not bind the IP address to the physical network card. The host always uses the specified IP, but it decides which network cards are used according to the routing table to send ). The network card configuration and routing table entries are as follows.

xxx@xxx-pc:~/iperf3/iperf$ /usr/local/bin/iperf3 -c 192.168.3.60 -B 192.168.3.238%eno1------通过网瞳eno1发送
xxx@xxx-pc:~/iperf3/iperf$ /usr/local/bin/iperf3 -c 192.168.3.60 -B 192.168.3.238%enp2s0----出错，连接超时
xxx@xxx-pc:~/iperf3/iperf$ /usr/local/bin/iperf3 -c 192.168.3.60 -B 192.168.3.107%enp2s0---通过网卡enp2s0发送
xxx@xxx-pc:~/iperf3/iperf$ /usr/local/bin/iperf3 -c 192.168.3.60 -B 192.168.3.107%eno1------出错，连接超时


xxx@xxx-pc:~/iperf3/iperf$ ifconfig
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.238  netmask 255.255.255.0  broadcast 192.168.3.255
        inet6 fe80::a6fc:b416:ea9c:df2f  prefixlen 64  scopeid 0x20<link>
        ether 8c:dc:d4:41:8b:ac  txqueuelen 1000  (以太网)
        RX packets 2143213  bytes 150333835 (150.3 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3228464  bytes 4899509947 (4.8 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 20  memory 0xf7f00000-f7f20000

enp2s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.107  netmask 255.255.255.0  broadcast 192.168.3.255
        inet6 fe80::98ed:685c:da1d:183e  prefixlen 64  scopeid 0x20<link>
        ether a0:36:9f:56:95:be  txqueuelen 1000  (以太网)
        RX packets 8818872  bytes 2054024464 (2.0 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 253112865  bytes 376654961590 (376.6 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0xf7d00000-f7dfffff


xxx@xxx-pc:~/iperf3/iperf$ route
内核 IP 路由表
目标            网关            子网掩码        标志  跃点   引用  使用 接口
default         _gateway        0.0.0.0         UG    100    0        0 enp2s0
default         _gateway        0.0.0.0         UG    101    0        0 eno1
link-local      0.0.0.0         255.255.0.0     U     1000   0        0 enp2s0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.168.3.0     0.0.0.0         255.255.255.0   U     100    0        0 enp2s0
192.168.3.0     0.0.0.0         255.255.255.0   U     101    0        0 eno1
192.168.70.128  0.0.0.0         255.255.255.192 U     0      0        0 demo-oai
192.168.71.128  0.0.0.0         255.255.255.192 U     0      0        0 rfsim5g-public
192.168.72.128  0.0.0.0         255.255.255.192 U     0      0        0 rfsim5g-traffic