Write a tomcat log cleaning script with shell from 0

First, the purpose

The tomcat log will get bigger and bigger with the passage of time. Although we can use cronolog to cut the log output by tomcat according to the date , but as the days go on, the logs/ folder will be densely packed with logs, which are not easy to view. A lot of space is wasted, so the purpose of this article is to write a script that automatically cleans up expired logs based on the date in the filename. (The script only uses logs named with yyyy-MM-dd)

2. Technical reserve

Have at least passed the book "Linux Shell Scripting Guide (2nd Edition)" and have some development knowledge

3. Demand design

The logic of the clean-up script is to traverse the logs in the log folder, obtain the time in the file name, and determine whether it is expired, rm if it expires, and record it in the log. The script simply provides the function of "find-judgment-clear-record", and the script is executed regularly and handed over to crontab for control. Combining the above design ideas, log checking and cleaning can be performed every day.

4. Detailed design and implementation

The script mainly consists of two modules

1. The logging module
is mainly composed of one method, and its main logic includes creating the log file of the cleaning script, and can input the parameters passed from the previous level into the log file according to time, requiring the log file to be divided according to the date. .

1.1 Script

logsPath=/root/clearserver/logs #定义该清理脚本的日志文件夹地址

#定义日志记录函数
function log(){
        #根据日期构建日志路径名
        logPath=$logsPath/clear.`date +%Y-%m-%d`.log
        #判断日志文件夹是否存在，不存在便创建
        if [ ! -d $logsPath  ]; then
                mkdir $logsPath
        fi
        #判断日志文件 是否存在，不存在便创建
        if [ ! -d $logPath ];then
                touch $logPath
        fi
        #构建一个时间格式，并简单的把传递过来的参数 append到日志中
        echo [`date +%Y-%m-%d\ %H:%M:%S`] $1 >> $logPath
}

2.
Cleanup module The main logic of the cleanup module is to
traverse the log files in the log folder to be cleaned -> intercept the date in the log file name -> judge whether it is expired -> clean up, log records

2.1 Script

#设定一个过期时间，以秒为单位
let expire=60*60*24*30

#配置需要清理的日志文件夹路径
files_path=/root/clearserver/test

#读取所有文件并存入变量
files=`ls $files_path`

#使用上述函数进行日志记录
log "start to check"

#累加器，记录删除的日志文件数
count=0

#开始遍历
for file in $files;
do
         fileName=${file##*/} #获取文件名，可去掉，这是优化前的代码

         #截取文件名中的时间
         date=`echo $file|egrep -o '[0-9]*-[0-9]*-[0-9]*'`

         #转换为纪元时
         time=`date +%s -d $date`

         #当前纪元时
         now=`date +%s`

         #时间差
         differ=$[ now - time ]

         if [ $differ -gt $expire ];
         then
                #如果过期，进入文件夹，删除文件
                cd $files_path 
                rm -rf $file
                #日志记录
                if [ $? -eq 0  ];
                then
                        log "clear file : $fileName success ";#do some logic
                        let count++
                else
                        log "clear file : $fileName fail"
                fi
        fi
done
#日志记录
log "end and  cleared $count record"

3. Complete script and test

clear.sh

#!/bin/bash
logsPath=/root/clearserver/logs

function log(){
        logPath=$logsPath/clear.`date +%Y-%m-%d`.log
        if [ ! -d $logsPath  ]; then
                mkdir -p $logsPath
        fi
        if [ ! -d $logPath ];then
                touch $logPath
        fi
        echo [`date +%Y-%m-%d\ %H:%M:%S`] $1 >> $logPath
}

#init expire time
let expire=60*60*24*30

#init the logs path
files_path=/root/clearserver/test

#find all logs
files=`ls $files_path`
log "start to check"
count=0
for file in $files;
do
         fileName=${file##*/}
         date=`echo $file|egrep -o '[0-9]*-[0-9]*-[0-9]*'`
         #change to time
         time=`date +%s -d $date`
         #now time
         now=`date +%s`
         differ=$[ now - time ]
         if [ $differ -gt $expire ];
         then
                cd $files_path
                rm -rf $file
                if [ $? -eq 0  ];
                then
                        log "clear file : $fileName success ";#do some logic
                        let count++
                else
                        log "clear file : $fileName fail"
                fi
        fi
done
log "end and  cleared $count record"

3.1 Test preparation Create a clearserver folder under
the /root folder, and vim writes the script clear.sh
to create a test folder under the /clearserver folder to simulate the log folder of tomcat, enter the test folder, and use touch to create a few When the test log file is
ready , you will see the following image:

Today is November 29th. After executing the cleanup script, the logs before November will be cleaned up

bash clear.sh

Now look at the generated cleanup log content

Now that the file has been successfully cleaned up, you only need to configure the script to crontab to execute it at 2 am every day to detect and clean up useless logs every day.