background
I'm training chatGLM recently. A training often takes nearly 2 hours. However, due to unstable network, ssh is often disconnected for no reason, causing the training to have to be restarted, which is a waste of time.
solution
Let me teach you a way to execute commands in the background. Even if your ssh connection is disconnected, the shell script will be executed silently in the background!
background execution
First, to execute a shell command in the background and print the output to a file, you can use the redirection symbol ">" to redirect the output to a file. For example, assuming we have a shell script named "command.sh", we can use the following command to execute it in the background and print the output to a file named "output.txt":
nohup command.sh > output.txt 2>&1 &
This command means:
nohup: Used to execute commands in the background.command.sh: The name of the shell script to be executed.> output.txt: Redirect the output of the command to a file named "output.txt". If the file does not exist, a new file is created; if the file already exists, new output is appended to the end of the file.2>&1: Merge standard error (2) with standard output (1) so that the error output of the command will also be printed to the file.&: Put the command into the background for execution.
View log
To view the contents of the "output.txt" file, you can use tailthe command, for example:
tail -n 100 output.txt
The meaning of this command is: display 100 lines of content starting from the end of the "output.txt" file. tailThe options of the command -nindicate the number of lines to display and -fthe file name to be displayed.
Programs that are normally executed in the background will not output any information, but with the tail command, you can view the output logs in the foreground in real time.