Linux basics, in-depth understanding of linux file system and log analysis

File system

Sector

Files are stored on the hard disk. The smallest storage unit of the hard disk is called a "sector", and each sector stores 512 bytes.

Block

Generally, eight consecutive sectors form a "block", and a block is 4K in size, which is the smallest unit of file access. When the operating system reads the hard disk, it reads multiple sectors continuously at one time, that is, read block by block.

File data

• File data includes actual data and meta-information (similar to file attributes).
• File data is stored in "blocks", and the area that stores file meta-information (such as file creator, creation date, file size, file permissions, etc.) is called inode.
• A file must occupy one inode and at least one block.

inode (index node or i node)

• The inode does not contain the file name. The file name is stored in the directory. Everything in a Linux system is a file, so a directory is also a kind of file.
• Each inode has a number, and the operating system uses the inode number to identify different files. The file name is not used internally in the Linux system, but the inode number is used to identify the file. For the system, the file name is just another name for the easy identification of the inode number. The file name and the inode number have a one-to-one correspondence, and each inode number corresponds to a file name.

Summary:
When a user tries to access a file in the Linux system, the system will first find its corresponding inode number according to the file name; get the inode information through the inode number, and according to the inode information, see if the user has the permission to access the file ; If there is, point to the corresponding data block and read the data.

inode content

inode contains the meta information of the file

• The number of bytes of the file
• User lD of the file owner
• Group lD of the file
• File read, write, and execute permissions
• File timestamp

Note: do not include the file name

Three main time attributes of Linux system files

• ctime(change time) #The last time the file or directory (attribute) was changed
• atime(access time) #The last time the file or directory was accessed
• mtime(modify time) #The time when the file or directory (content) was last modified

Inode number query

ls -i 文件名        #查普通文件
stat 文件名       #查普通文件和目录

inode size

• The node also consumes hard disk space, so when formatting, the operating system automatically divides the hard disk into two areas. One is the data area, which stores file data; the other is the inode area, which stores the information contained in the inode.
  • The size of each inode
  • -Generally 128 bytes or 256 bytes
• The total number of inodes is determined when the file system is formatted
• Use the df -i command to view the total number of inodes corresponding to each hard disk partition and the number of inodes that have been used.

Special role of inode

Due to the separation of the inode number and the file name, some Unix/Linux systems have the following phenomena:

• The file name contains special characters and may not be deleted normally. At this time, delete the inode directly, which can play the role of deleting the file;
• Move files or rename files, just change the file name without affecting the inode number;
• After opening a file, the system will use the inode number to identify the file, regardless of the file name.
• After the file data is modified and saved, a new inode number will be generated.

Delete files by inode number

find ./ -inum 52305140 -exec rm -i {} \;
find ./ -inum 50464299 -delete

Simulate inode node exhaustion fault handling

#使用fdisk创建分区/dev/sdb1，分区大小30M即可
fdisk /dev/sdb         #创建分区
mkfs.ext4 /dev/sdb1   #对于centos 7系统，节点耗尽故障处理的文件类型可以是ext3或ext4
mkdir /test
mount /dev/sdb1 /test
df -i
#模拟inode节点耗尽故障
for ((i=1; i<=7680; i++));do touch /test/file$i;done  
或者   touch {1..7680}.txt
df -i
df -hT
#删除文件恢复
rm -rf /test/*
df -i
df -hT

First add a hard disk, use fdisk to create partition /dev/sdb1, format and mount

Simulate inode node exhaustion failure

Deleted file recovery

EXT file recovery

extundelete is an open source Linux data recovery tool that supports ext3 and ext4 file systems. (Ext4 can only be restored in centos6 version)

1、使用fdisk创建分区/dev/sdb1，格式化ext3文件系统
fdisk /dev/sdb  
mkfs.ext3 /dev/sdb1
mkdir /test
mount /dev/sdb1 /test
df -hT

2、安装依赖包
yum -y install e2fsprogs-devel e2fsprogs-libs

3、编译安装 extundelete
cd /test  切换到test目录中
wget http://nchc.dl.sourceforge.net/project/extundelete/extundelete/0.2.4/extundelete-0.2.4.tar.bz2  #联网下载安装包
tar jxvf extundelete-0.2.4.tar.bz2   #解压tar包
cd extundelete-0.2.4/     #切换到解压出来的目录中
./configure --prefix=/usr/local/extundelete && make && make install  #指定安装目录，开始安装   
ln -s /usr/local/extundelete/bin/* /usr/bin/    #创建软连接，让系统识别命令

4、模拟删除并执行恢复操作
cd /test
echo a>a
echo a>b
echo a>c
echo a>d
ls
extundelete /dev/sdb1 --inode 2			#查看文件系统/dev/sdb1下存在哪些文件，i 节点是从 2 开始的，2 代表该文件系统最开始的目录。

rm -rf a b
extundelete /dev/sdb1 --inode 2	
cd ~
umount /test
extundelete /dev/sdb1 --restore-all		#恢复/dev/sdb1 文件系统下的所有内容
#在当前目录下会出现一个RECOVERED_FILES/目录，里面保存了已经恢复的文件
ls RECOVERED_FILES/

First add a hard disk, use fdisk to create partition /dev/sdb1, format and mount

Install dependent packages in yum mode

Download the installation package and install

Simulate delete and restore

Delete a, b

xfs type file backup and recovery

• CentOS 7 system uses xfs type files by default, and xfs type files can be backed up and restored using the xfsdump and xfsrestore tools.
• There are two backup levels for xfsdump: 0 means full backup; 1-9 means incremental backup. The default backup level of xfsdump is 0.

The command format of xfsdump is:

xfsdump -f 备份存放位置 要备份的路径或设备文件

Commonly used options of the xfsdump command:

Options	effect
-f	Specify the backup file directory
-L	Specify the label session label
-M	Specify the device label media label
-s	Back up a single file, the path cannot be directly followed by -s

Xfsdump usage restrictions:
1. Only the mounted file system can be backed up
2. You must use root permissions to operate
3. Only the XFS file system can be backed up
4. The data after the backup can only be parsed by xfsrestore
5. You cannot back up two with File systems with the same UUID (can be viewed with the blkid command)

operating:

1、使用fdisk创建分区/dev/sdb1，格式化xfs文件系统
fdisk /dev/sdb
partprobe /dev/sdb   #重读分区表，磁盘查询不到时使用
mkfs.xfs [-f] /dev/sdb1
mkdir /data
mount /dev/sdb1 /data/
cd /data
cp /etc/passwd ./
mkdir test
touch test/a

2、使用 xfsdump 命令备份整个分区
rpm -qa | grep xfsdump
yum install -y xfsdump
xfsdump -f /opt/dump_sdb1 /dev/sdb1 [-L dump_sdb1 -M sdb1]

3、模拟数据丢失并使用 xfsrestore 命令恢复文件
cd /data/
rm -rf ./*
ls

xfsrestore -f /opt/dump_sdb1 /data/

Use fdisk to create partition /dev/sdb2, format and mount

Use the xfsdump command to back up the entire partition

Simulate data loss and use the xfsrestore command to restore files

Log file

Log function

• Used to record various events that occur during the operation of the system and programs
• By reading the log, it is helpful to diagnose and solve system failures

Classification of log files

• Kernel and system logs
  • Unified management by the system service rsyslog, the log format is basically similar
  • Main configuration file /etc/rsyslog.conf
• User log
  • Record system user login and logout related information
  • Main configuration file /var/log/secure
• Program log
  • Log files independently managed by various applications, the record format is not uniform

Log save location

Placed in the directory /var/log/ by default

Common log files:

#Kernel and public message log:

• /var/log/messages: Record Linux kernel messages and public log information of various applications, including startup, ro errors, network errors, program failures, etc. For applications or services that do not use a separate log file, you can generally obtain related event record information from the log file.

#Plan task log:

• /var/log/cron: Record event information generated by crond scheduled tasks.

#System boot log:

• /var/log/maillog: Record the e-mail activity entering or sending out the system.

#User login log:

• /var/log /secure: Record security event information related to user authentication.
• /var/log/lastlog: Record the latest login events of each user. Binary format
• /var/log/wtmp: Record each user login, logout and system startup and shutdown events. Binary format
• /var/run/btmp: Record failed, wrong login attempts and verification events. Binary format

Kernel and system logs

• Unified management by the system service rsyslog

  • Software package: rsyslog-7.4.7-16.el7.x86_64,
  • Main program: /sbin/rsyslogd
  • Configuration file: /etc/rsyslog.cont

View the rsyslog.conf configuration file

vim /etc/rsyslog.conf		#查看rsyslog.conf配置文件
*.info;mail.none;authpriv.none;cron.none         /var/log/messages

*.info		#表示info等级及以上的所有等级的信息都写到对应的日志文件里
mail.none	#表示某事件的信息不写到日志文件里（这里比如是邮件）

Log message priority

Priority level of Linux system kernel log messages (the smaller the number level, the higher the priority, the more important the message):

Grade number	Priority level	Description
0	EMERG (emergency)	Will cause the host system to be unavailable.
1	ALERT (alert)	Problems that must be resolved immediately.
2	CRIT (serious)	More serious situation.
3	ERR (error)	An error occurred during operation.
4	WARNING	Important events that may affect system functions and need to remind users.
5	NOTICE	Will not affect normal functions, but events that require attention.
6	INFO (information)	General information.
7	DEBUG (debugging)	Program or system debugging information, etc.

General format of the log

• #Public log/var/log/messages file record format
• Time stamp: the date and time when the message was sent.
• Host name: The name of the computer that generated the message.
• Subsystem name: The name of the application that issued the message.
• Message: The specific content of the message.

User log

• Save the relevant information of the user login and logout of the system
  • /var/log/secure: Records security event information related to user authentication.
  • /var/log/lastlog: Record the latest login events of each user. Binary format
  • /var/log/wtmp: Record each user login, logout and system startup and shutdown events. Binary format
  • /var/run/btmp: Record failed, wrong login attempts and verification events. Binary format
• analyzing tool
  • users、 who、w 、last、lastb
  • The last command is used to query the user records that have successfully logged in to the system
  • The lastb command is used to query user records that failed to log in

Program log

• Log files independently managed by various applications, the record format is not uniform

  • Web service: Nar/log/httpd/
    • access_log //Record customer access events
    • error_log //Record error events
  • Proxy service: /var/log/squid/
    • access.log、cache.log

• analyzing tool

  • Text view, grep filter search, view in Webmin management suite
  • Text filtering, formatting and editing tools such as awk and sed
  • Webalizer, Awstats and other dedicated log analysis tools

Log management

• Make timely backups and archives
• Extend the log retention period
• Control log access
  • Logs may contain various sensitive information, such as accounts, passwords, etc.
• Centralized management of logs
  • Send the server's log file to the unified log file server
  • Facilitate the unified collection, sorting and analysis of log information
  • Prevent accidental loss, malicious tampering or deletion of log information