In Linux, the error driver failed programming external connectivity on endpoint: xxxx is reported when Docker starts the service

Environment: Rocky 8.6( CentOS 7), firewalld,Docker

PS: If the firewall is turned off, maybe I won't encounter this problem all the time, and I lost a bunch of hair again.

An error will be reported if and only when the port of the server needs to be occupied. If there is no need to map the port, there will be no corresponding problems .
For example: docker run -d --name nginx-test1 nginxThis kind of port is not mapped.
docker run -d --name nginx-test1 -p 80:80 nginxIf the port is mapped, you need to operate the firewall (if enabled).

1. Scene

Because a service on the server needs to expose the port , the firewall has been operated and restarted; then the services running Dockeron server cannot be used normally.
When executing docker run xxxand , docker start xxxxthe following error will be reported:

docker: Error response from daemon: driver failed programming external connectivity 
on endpoint xxxx(d3e43fa36d34987312e2c3ffd30d3ce3d1892d43ef951241fc2ae320ffeb1330):  
(iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 9905 -j DNAT 
--to-destination 172.18.0.2:9005 ! -i docker0: iptables: No chain/target/match by that name.
 (exit status 1)).

2. Reason

DockeriptablesFor the relationship with , please refer to the official document: https://docs.docker.com/network/iptables/

dockerWhen the service starts, dockerthe service will automatically iptablesregister twoDOCKER-USER chains named and to allow communication between the ports exposed by the service management .DOCKERdocker容器（containner）

When the firewall firewalldis restarted, the chain will be lost, docker runand the above error will appear when it is used.

3. Solutions

3.1 Docker restart (recommended)

This method is the most simple and rude, because it will be automatically registered at Dockerstartup . Isn’t it nice! ! !

systemctl restart docker
# 或
service docker restart 

[Note] The container with --restart=alwaysthe parameter will be automatically started as Docker. If it is not set, it needs to be restarted manually.
If you want to set automatic startup for a container, you can set it with the following command:

# 更新已有的容器：xxx 为容器的标识或容器的名称
docker container update --restart=always xxxx

# 启动时设置
 docker run -d --name nginx-test1 -p 80:80 --restart=always nginx

3.2 Restoring iptables rules (not recommended)

In fact, this method is to back up the rules of the firewall through iptables-savethe command , and then iptables-restorerestore it through ; but it is not very practical in the process of practice.

• When iptablesthere are no Dockertwo , then the ports exposed by all services will not take effect, and the service will be interrupted (at this time, it is better to restart directly).
• Even if it is backed up, if there is a change in the Dockerexposed service, you need to manually handle the difference, which is very troublesome.
• If there is no backup at all, manually add the corresponding chain. If many services Dockerare running , it will be very laborious to add (still don’t bother)
• If you back up regularly, you can only use this method (but since the service has been interrupted, it is better to restart it directly).

The complexity of a iptablesrule directly persuades:

# Generated by iptables-save v1.8.4 on Thu Jan 26 14:58:33 2023
*filter
:INPUT ACCEPT [105973064:35160153136]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [96468849:80475545565]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER -d 172.18.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8805 -j ACCEPT
-A DOCKER -d 172.18.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8801 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
COMMIT
# Completed on Thu Jan 26 14:58:33 2023
# Generated by iptables-save v1.8.4 on Thu Jan 26 14:58:33 2023
*security
:INPUT ACCEPT [105973069:35160156762]
:FORWARD ACCEPT [11122602:8643145221]
:OUTPUT ACCEPT [96468861:80475547136]
COMMIT
# Completed on Thu Jan 26 14:58:33 2023
# Generated by iptables-save v1.8.4 on Thu Jan 26 14:58:33 2023
*raw
:PREROUTING ACCEPT [117095673:43803302209]
:OUTPUT ACCEPT [96468861:80475547136]
COMMIT
# Completed on Thu Jan 26 14:58:33 2023
# Generated by iptables-save v1.8.4 on Thu Jan 26 14:58:33 2023
*mangle
:PREROUTING ACCEPT [117095673:43803302209]
:INPUT ACCEPT [105973071:35160156988]
:FORWARD ACCEPT [11122602:8643145221]
:OUTPUT ACCEPT [96468861:80475547136]
:POSTROUTING ACCEPT [107591469:89118692771]
COMMIT
# Completed on Thu Jan 26 14:58:33 2023
# Generated by iptables-save v1.8.4 on Thu Jan 26 14:58:33 2023
*nat
:PREROUTING ACCEPT [288081:17293388]
:INPUT ACCEPT [287965:17286375]
:POSTROUTING ACCEPT [1756221:105814926]
:OUTPUT ACCEPT [1756221:105814926]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.18.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.18.0.2/32 -d 172.18.0.2/32 -p tcp -m tcp --dport 8805 -j MASQUERADE
-A POSTROUTING -s 172.18.0.3/32 -d 172.18.0.3/32 -p tcp -m tcp --dport 8801 -j MASQUERADE
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A DOCKER -i docker0 -j RETURN
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 8805 -j DNAT --to-destination 172.18.0.2:8805
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 8801 -j DNAT --to-destination 172.18.0.3:8801
COMMIT
# Completed on Thu Jan 26 14:58:33 2023

Personal blog: Roc's Blog