[Docker] docker creates a container -m stepping on the pit record

1. Environment

ubuntu20.04
docker 
nvidia-docker

2. Problem

Start multiple processes, and other processes will be killed together and killed.

The phenomenon is similar to the blog https://blog.csdn.net/Tilyp/article/details/103367159 , but no solution is seen. It may take 10 hours (multiplayer power).

3. Analysis

1) When the two processes are started separately, keep flashing to see if the memory has been used up, or if there is free memory:

cat /proc/meminfo | grep Mem

2) Analyze the memory occupied by a process:

First use nvidia-smi to view the process pid:

nvidia-smi

Then use the following command to view the process memory usage:

ps -aux | grep your_pid

Or check the memory usage of the previous processes:

ps aux | sort -k4nr | head -10

3) Think of an article I saw before, about the problem of memory limitations. Docker resource limit reference:

https://www.icode9.com/content-4-827661.html

4. Solve

Set the -m parameter to a larger value. solved the problem. Set from 8GB to 16GB or 32GB. This is related to the local maximum memory, which is troublesome, so set it directly to the maximum.

nvidia-docker run -p 10001:22 -m 32GB ...

5. Leave a question

In other equipment, using the same configuration, there is no error, but in the new machine, the problem is still uncertain.

Could it be the system or the updated docker? ? ?