Run the python program in the background on the server

write on top

It has been pycharm to run the code on the server, but there is a problem: the program stops running when the local computer sleeps. But the deep learning code ran all day long, and it was uncomfortable after
I accidentally
closed it a few times. The code has not finished running but due to abnormal termination problems such as network instability, so the code can be executed in the background through the nohup command, and it is also convenient to run multiple programs at the same time.

You can also use it to download packages like torch that need to be installed for a few hours

Reference: https://blog.csdn.net/huavhuahua/article/details/121145512
https://blog.csdn.net/m0_38024592/article/details/103336210
https://blog.csdn.net/Yonggie/article/ details/120267134

environment

install nohup

pip install nohup

Direct run code error: nohup: failed to run command `python3': No such file or directory
Note: You still need to switch to the corresponding conda environment, then cd to the code directory, and finally run the code

code example

nohup executes in the background, ignores input and saves output to specified file

nohup python -u myfile.py >> myresult.out 2>&1 &

Among them, myfile.py is the program that needs to be run, myresult.out is the output file, and myresult.out is saved in the same path as the .py file by default.

Python instructions can also be replaced with other program execution instructions to run programs in other languages, and the rest remain unchanged

Nohup executes the py file in the background, and enters both the normal log and the error log of the execution into myresult2.out

The -u parameter makes python not enable buffering. Solve the problem that the output of python is buffered, causing nohup.out to see the output immediately.

nohup command

Nohup refers to running continuously, which is an abbreviation of no hang up, which means uninterrupted and does not hang up. When running a process, if you don't want it to close when you log out of your account, you can use nohup.

Nohup is not specified, so the output will go to nohup.out

& Background process

& for running in the background
Use & at the end of the program to make the program run automatically.

2>&1 redirect error content to standard output

0 means stdin standard input, user keyboard input content
1 means stdout standard output, content output to the display
2 means stderr standard error, error content
2>&1 is a whole, > cannot have spaces on the left and right, and the error content is redirected to input to standard output.

The following two codes are the same, except that 1 (standard input) is omitted, and the following my.log 2>&1 will input 2 (error content) to standard output, and then the previous standard output will be input to my.log again, Meaning errors and standard content will be output to my.log

nohup python my.py >> my.log 2>&1 &
nohup python my.py 1 >> my.log 2>&1 &

Run the myfile.py script continuously in the python environment, and redirect the output of the script to my.log (>> means append, if you use >, the content will be cleared)

nohup python my.py >> my.log 2>&1 &

The code can be split into two pieces. Right now:

# 等价于下面两行的内容
nohup python my.py 1>> my.log &
nohup python my.py 2>> my.log &

• The above code is to output both error and standard to my.log, the original code is just simplified.
  If it is 2>1, it is to output the error content to file 1.
  Add & in 2>&1 to distinguish files 1 and 1 (standard output)
• nohup tomcat.sh > /dev/null 2>&1 & is the same, throw all errors and standards into /dev/null, and destroy them all.

View current python-related processes

Linux can view various processes through the relevant parameters of the ps command, but viewing all processes is often equal to finding a needle in a haystack, so viewing python-related processes is more practical when running code

ps -ef |grep python

You can see that there are currently two programs running, and the process numbers are 2173876 and 2183310 respectively.

end process

If you don't want a program to continue executing, you can end the process

kill -9 <进程号>

nohup background pip download and install

In the same way
, if the terminal directly pip installs, pip will naturally be interrupted after the interrupt is turned off.
But some of them are slow to download, such as torch, which takes several hours.

Solution: use nohup. But you can't directly nohup pip, because pip itself is a Python module.
use

nohup python -m pip install xxx&

It will hang a program until it finishes running, and its output will be in nohup.out, you can view the log.
The disadvantage is that the log will only show downloading xxx, not the progress. You don't know whether it is finished or not, you can only use the top command to find the process id to check whether the installation is complete.