Solve the problems related to cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5).
Errors in the service log are similar to the following:
2023-04-13 10:58:15.701 19582 INFO werkzeug [-] 127.0.0.1 - - [13/Apr/2023 10:58:15] “POST /NetworkDriver.RevokeExternalConnectivity HTTP/1.1” 200 -
2023-04-13 10:58:15.711 19582 INFO werkzeug [-] 127.0.0.1 - - [13/Apr/2023 10:58:15] “POST /NetworkDriver.Leave HTTP/1.1” 200 -
2023-04-13 10:58:16.075 19582 WARNING pr2modules.ipdb.main [-] Deprecation warning https://docs.pyroute2.org/ipdb_toc.html
2023-04-13 10:58:16.230 19582 ERROR kuryr_libnetwork.controllers [-] net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory
net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5)
PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory
PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
ovs-vsctl: no port named qvo1161b338-11
2023-04-13 10:58:16.422 19582 INFO werkzeug [-] 127.0.0.1 - - [13/Apr/2023 10:58:16] “POST /NetworkDriver.DeleteEndpoint HTTP/1.1” 200 -
2023-04-13 10:58:17.428 19582 INFO werkzeug [-] 127.0.0.1 - - [13/Apr/2023 10:58:17] “POST /IpamDriver.ReleaseAddress HTTP/1.1” 200 -
2023-04-13 10:58:38.572 19582 INFO werkzeug [-] 127.0.0.1 - - [13/Apr/2023 10:58:38] “POST /IpamDriver.RequestAddress HTTP/1.1” 200 -
2023-04-13 10:58:39.035 19582 ERROR kuryr_libnetwork.controllers [-] /usr/local/libexec/kuryr/ovs: line 50: brctl: command not found
/usr/local/libexec/kuryr/ovs: line 51: brctl: command not found
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory
net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5)
PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory
PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
2023-04-13 10:58:39.037 19582 INFO tool [-] 127.0.0.1 - - [13/Apr/2023 10:58:39] “POST /NetworkDriver.CreateEndpoint HTTP/1.1” 200 -
2023-04-13 10:58:39.373 19582 INFO tool [-] 127.0.0.1 - - [13/Apr/2023 10:58:39] “POST /NetworkDriver.Join HTTP/1.1” 200 -
2023-04-13 10:58:39.378 19582 INFO tool [-] 127.0.0.1 - - [13/Apr/2023 10:58:39] “POST /NetworkDriver.ProgramExternalConnectivity HTTP/1.1” 200 -
2023-04-13 10:58:39.484 19582 INFO tool [- ] 127.0.0.1 - - [13/Apr/2023 10:58:39] “POST /NetworkDriver.EndpointOperInfo HTTP/1.1” 200 -
2023-04-13 10:58:39.656 19582 INFO tool [-] 127.0.0.1 - - [13/Apr/2023 10:58:39] “POST /NetworkDriver.EndpointOperInfo HTTP/1.1” 200 -
2023-04-13 10:58:53.189 19582 INFO werkzeug [-] 127.0.0.1 - - [13/Apr/2023 10:58:53] “POST /NetworkDriver.RevokeExternalConnectivity HTTP/1.1” 200 -
2023-04-13 10:58:53.197 19582 INFO werkzeug [-] 127.0.0.1 - - [13/Apr/2023 10:58:53] “POST /NetworkDriver.Leave HTTP/1.1” 200 -
2023-04-13 10:58:53.588 19582 ERROR kuryr_libnetwork.controllers [-] net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory
net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5)
PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory
PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
2023-04-13 10:58:53.758 19582 INFO werkzeug [-] 127.0.0.1 - - [13/Apr/2023 10:58:53] “POST /NetworkDriver.DeleteEndpoint HTTP/1.1” 200 -
2023-04-13 10:58:54.883 19582 INFO werkzeug [-] 127.0.0.1 - - [13/Apr/2023 10:58:54] “POST /IpamDriver.ReleaseAddress HTTP/1.1” 200 -
2023-04-13 10:58:55.265 19582 INFO werkzeug [-] 127.0.0.1 - - [13/Apr/2023 10:58:55] “POST /IpamDriver.RequestAddress HTTP/1.1” 200 -
2023-04-13 10:58:56.621 19582 ERROR kuryr_libnetwork.controllers [-] /usr/local/libexec/kuryr/ovs: line 50: brctl: command not found
/usr/local/libexec/kuryr/ovs: line 51: brctl: command not found
net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory
net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5)
PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory
PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)
2023-04-13 10:58:56.624 19582 INFO tool [-] 127.0.0.1 - - [13/Apr/2023 10:58:56] “POST /NetworkDriver.CreateEndpoint HTTP/1.1” 200 -
2023-04-13 10:58:56.965 19582 INFO tool [-] 127.0.0.1 - - [13/Apr/2023 10:58:56] “POST /NetworkDriver.Join HTTP/1.1” 200 -
2023-04-13 10:58:56.970 19582 INFO tool [-] 127.0.0.1 - - [13/Apr/2023 10:58:56] “POST /NetworkDriver.ProgramExternalConnectivity HTTP/1.1” 200 -
2023-04-13 10:58:57.097 19582 INFO tool [- ] 127.0.0.1 - - [13/Apr/2023 10:58:57] “POST /NetworkDriver.EndpointOperInfo HTTP/1.1” 200 -
2023-04-13 10:58:57.226 19582 INFO tool [-] 127.0.0.1 - - [13/Apr/2023 10:58:57] “POST /NetworkDriver.EndpointOperInfo HTTP/1.1” 200 -
Solution:
Install missing dependencies such as:
yum -y install libibverbs libmlx5
Restart the corresponding service, for example:
systemctl restart kuryr-libnetwork