Linux large file split processing

If the log files are not regularly split and backed up early in the project deployment, the log files will become larger and larger. Once problems occur, it is difficult to view the log files and locate the problem. Now I have learned two ways to solve it, share it.

1. Linux split command

split [--help][--version][-<行数>][-b <字节>][-C <字节>][-l <行数>][要切割的文件][输出文件名]

The split command can be divided by line or byte size.
Example:
Splitting with 20m size:
split -b 20m catalina.out part_
will generate part_aa, part_ab and other files with letter suffix.
Split by 10000 lines:
split -10000 catalina.out part_

2. logrotate tool
It provides automatic replacement, compression, deletion and mailing of log files. Each log file can be set to be processed on a daily, weekly or monthly basis. It can also be processed immediately when the file is too large.
Example:
Create a new file in the directory /etc/logrotate.d/, e.g. vi /etc/logrotate.d/tomcat
content:

/app/tomcat/logs/catalina.out {  
su root tomcat 
rotate 31  
daily  
copytruncate  
compress  
notifempty  
missingok  
dateext  
}

Then execute / usr / sbin / logrotate -f /etc/logrotate.conf
to find that the file has been backed up.

Explanation of parameters:
compress compresses the log after dumping by gzip. When
nocompress is not needed, use this parameter
copytruncate for the log file that is still open, back up the current log and truncate the
nocopytruncate backup log file but not truncate
create mode owner group transfer To store files, use the specified file mode to create a new log file
nocreate does not create a new log file when
delaycompress and compress are used together, the dumped log file will not be compressed until the next dump
nodelaycompress overrides the delaycompress option, the dump is also compressed.
errors address The error information during special storage is sent to the specified email address
ifempty, even if it is an empty file, this is the default option of logrotate.
notifempty If it is an empty file, do not dump
mail address. Send the dumped log file to the specified E-mail address.
Nomail does not send the log file when dumping.
olddir directory The dumped log file is placed in the specified directory. The log file after the current log file is
dumped in the same file system noolddir and the current log file are placed in the same directory.
Prerotate / endscript The commands that need to be executed before dumping can be put into this pair. These two keywords must be in separate lines
postrotate / endscript The commands that need to be executed after dumping can be put into this pair, these two keywords must be separated
daily specifies the dump cycle is daily,
weekly specifies the dump cycle is weekly,
monthly specifies the dump cycle is monthly
rotate count, specifies the number of dumps before the log file is deleted, 0 means no backup, 5 means 5 backups are reserved
tabootext [+] list tells logrotate not to dump files with the specified extension. The default extensions are: .rpm-orig, .rpmsave, v, and ~
size size Only dump when the log file reaches the specified size. Size can specify bytes ( (Default) and K (sizek) or M (sizem).
Dateext; add date label to the log file
Force
logrotate -vf /etc/logrotate.d/log-file

Published 137 original articles · Like 123 · Visit 250,000+

Guess you like

Origin blog.csdn.net/lz20120808/article/details/100102399