Description of Requirement :
Today saw a system of temporary files, the size of 5.6G, this file is useless, and you want most of the files are deleted.
In this record deletion process. N rows before deleting records.
Operation process :
For the case where a large amount of data (in this case 58 million lines)
1. tail command, the end of the file data row n redirected to the new file
tail -n 30000 out.tmp > out.tmp.bak
2. Then delete the old file, modify the new file to the original name
rm out.tmp mv out.tmp.bak out.tmp
Note: this is the case file out.tmp 30000 line at the end of it preserved before it is deleted rows in my test environment, record out.tmp of about 58 million +.
For the case of relatively small data
1. Check the data file to determine the data to be deleted
The back of the line 5 are deleted, but also to redirect by -n
2.tail -n + k This function is implemented
[root@testvm01 ~]# wc -l install.log 249 install.log [testvm01 the root @ ~] # tail -n +5 the install.log> install.log.tmp # after the contents of line 5 is redirected to another temporary file [root@testvm01 ~]# rm -f install.log [root@testvm01 ~]# mv install.log.tmp install.log [root @ testvm01 ~] # head install.log previous line # 5 were gone, additional methods employed to realize the function to delete. Installing tzdata-2014g-1.el6.noarch Installing basesystem-10.0-4.el6.noarch Installing ncurses-base-5.7-3.20090208.el6.x86_64 Installing glibc-common-2.12-1.149.el6.x86_64 Installing nss-softokn-freebl-3.14.3-17.el6.x86_64 Installing glibc-2.12-1.149.el6.x86_64 Installing ncurses-libs-5.7-3.20090208.el6.x86_64 Installing bash-4.1.2-29.el6.x86_64 Installing libattr-2.4.44-7.el6.x86_64 Installing libcap-2.16-5.5.el6.x86_64 [root@testvm01 ~]# wc -l install.log 245 install.log
Summary : The same can use the head command to remove the n line at the end of the file.
In data mining, we often incremental update training logs, you need to delete outdated data before the n-th row directly sed command slower:
tail -n +3 old_file> new_file
mv new_file old_file
This deletes the first two rows, to speed faster than sed command