disk space cleanup script

Disk space cleanup script, for reference only

This is a shell script used to clean up disk data. Its main function is to delete files in the specified directory according to the set rules. Here is a rough explanation of the script:

1. cacheRoot=/data: Set the root directory for storing data /data.

2. cleanDir=$cacheRoot/record: Set the directory to clean up as $cacheRoot/record.

3. trashBoxDir=$cacheRoot/.cache: Set the recycle bin directory to $cacheRoot/.cache.

4. cachecull=50: Set the threshold of the disk usage percentage, and start cleaning data after reaching the threshold.

5. cacherun=80: Set the maximum threshold of the disk usage percentage, and clean up the data immediately after exceeding the threshold.

6. count=0and maxCout=144: Sets the counter and maximum count value used to control how often a full cleanup operation is performed.

7. log_info(): A custom function used to print log information, including the current time, log type and content.

8. detect_disk_usage(): Detect disk usage and save the result in a variable usePecentage.

9. clean_trashbox(): Create a directory structure and empty the recycle bin directory.

10. daily_check(): The cleanup operation is performed every day, and the files older than 30 days are moved to the recycle bin directory.

11. clear_data(): Perform cleaning operations according to the set rules, keep files within a certain time range and move them to the recycle bin directory until the disk usage threshold is met.

12. while true; do ... done &: Performs cleanup operations in an infinite loop and runs in the background.

The execution flow of the script is roughly as follows:

1. Create directory structure and recycle bin directory.

2. Execute a complete cleaning operation, and move eligible files to the recycle bin directory according to the set rules.

3. Check the disk usage, if it is lower than the threshold, wait for a while to continue checking.

4. If the disk usage exceeds the threshold, the cleanup operation is performed again until the threshold is met.

5. When the counter reaches its maximum value, a full cleanup operation is performed.

Please note that the script may be written for specific needs, and the paths and rules involved need to be adjusted according to the actual situation. In addition, the script will always run as a background process, continuously monitoring and cleaning disk data, until the script is manually stopped or the server is restarted.

#!/bin/bash
# Copyright 
# author:
# file: clean_disk_data.sh
# desc: 校验分区的大小，优先删除30天之前的数据，依次删除，保留数据到1天，删除到阀值为50%

cacheRoot=/data
cleanDir=$cacheRoot/record
trashBoxDir=$cacheRoot/.cache
cachecull=50
cacherun=80
count=0
maxCout=144

function log_info()
{
	echo $(date "+%Y-%m-%d %H:%M:%S") [INFO] $1 $2
}

function detect_disk_usage()
{
	usePecentage=$(df -l | grep /data | head -n 1 | sed "s/%.*//g"|sed "s/.* //g")
	log_info "usePecentage:" $usePecentage
	if [ "$usePecentage" = "" ];then
		usePecentage=0
	fi
}

function clean_trashbox()
{
	mkdir -p $cleanDir
	mkdir -p $trashBoxDir
	mkdir -p $cacheRoot
	# mkdir -p $cacheRoot/.blank
	# rsync --delete-before -d $cacheRoot/.blank/ $trashBoxDir
	rm -rf $trashBoxDir/*
}

function daily_check()
{
	log_info "daily_check begin"
	find ${cleanDir} -maxdepth 1 ! -path ${cleanDir} -atime +30 -exec mv{} $trashBoxDir \;
	clean_trashbox
	log_info "daily_check end"
}

function clear_data()
{
	log_info "clear_data begin"
	#keep at leaset 3 days, once delete 3days
	for((i=30;i>=3;i-=3));do
		find ${cleanDir} -maxdepth 1 ! -path ${cleanDir} -atime +$i -exec mv{} $trashBoxDir \;
		clean_trashbox
		detect_disk_usage
		if [ $usePecentage -lt $cachecull ];then
			log_info "usePecentage:" $usePecentage
			return
		fi
	done
	#keep at leaset 1 days
	if [ $usePecentage -gt $cachecull ];then
		find ${cleanDir} -maxdepth 1 ! -path ${cleanDir} -atime +1 -exec mv{} $trashBoxDir \;
		return
	fi
	clean_trashbox
	log_info "daily_check end"
}

log_info "clean disk data begin"
clean_trashbox
clear_data
daily_check

while true; do
	detect_disk_usage
	if [ $usePecentage -lt $cachecull ];then
		count=$((count+1))
		sleep 600
	else
		clear_data
		sleep 20
	fi
	if [ ${count} -ge ${maxCout} ];then
		daily_check
		count=0
	fi
done &

log_info "clean disk data end"