Linux file system and log files

Linux file system and log files

Overview of inodes and blocks

File data includes meta-information and actual data. The
file is stored on the hard disk. The smallest storage unit of the hard disk is "sector". Each sector stores a 512-byte
block (block)
. Eight consecutive sectors form a block, which
is the smallest file storage. Unit
inode (index node)
Chinese translation is "index node", also called i node,
used to store file meta information

The inode contains the meta information of the
file. The number of bytes of the
file. The User ID of the file owner (not including the file name)
. The Group ID of the
file. The read and write execution permissions of the
file. The time stamp of the file
...
You can use the stat command to view the inode information of a file

[root@localhost opt]# touch aa.txt
[root@localhost opt]# stat aa.txt
  文件："aa.txt"
  大小：0         	块：0          IO 块：4096   普通空文件
设备：fd00h/64768d	Inode：33567187    硬链接：1
权限：(0644/-rw-r--r--)  Uid：(    0/    root)   Gid：(    0/    root)
环境：unconfined_u:object_r:usr_t:s0
最近访问：2020-07-02 03:01:28.719405830 +0800
最近更改：2020-07-02 03:01:28.719405830 +0800
最近改动：2020-07-02 03:01:28.719405830 +0800
创建时间：-

##The three main time attributes of Linux system files
ctime (change time) the
last time a file or directory (attribute) was changed
atime (access time) the
last time a file or directory was accessed
mtime (modify time) the
last time a file or Catalog (content) time

The structure of the catalog file

A directory is also a kind of file
. The structure of a directory file.

Each inode has a number. The operating system uses the inode number to identify different files. The
Linux system does not use file names, but uses the inode number to identify files.
For users, the file name is just the inode number. Nickname for easy identification

Inode number When a
user opens a file through a file name, the internal process of the
system 1. The system finds the inode number corresponding to the file name
2. Gets the inode information through the inode number
3. Finds the block where the file data is located according to the inode information, and reads it out data

How to check the inode number

ls -i command: View the inode number corresponding to the file name

[root@localhost opt]# ls -i aa.txt
33567187 aa.txt
[root@localhost opt]# stat aa.txt
  文件："aa.txt"
  大小：0         	块：0          IO 块：4096   普通空文件
设备：fd00h/64768d	Inode：33567187    硬链接：1
权限：(0644/-rw-r--r--)  Uid：(    0/    root)   Gid：(    0/    root)
环境：unconfined_u:object_r:usr_t:s0
最近访问：2020-07-02 03:01:28.719405830 +0800
最近更改：2020-07-02 03:01:28.719405830 +0800
最近改动：2020-07-02 03:01:28.719405830 +0800
创建时间：-

Structure after hard disk partition

inode size

Inodes also consume hard disk space
. The size of each inode is
generally 128 bytes or 256 bytes.

Determine the total number of inodes when formatting the file system.
Use the df -i command to view the total number of inodes and the number of

inodes used in each hard disk partition. The special function
of inodes . Because the inode number is separated from the file name, some Unix/Linux systems have the following phenomena
When the file name contains special characters, the file may not be deleted normally. You can delete the inode directly, or you can delete the file. When you
move or rename a file, change the file name without affecting the inode number. After
opening a file, the system uses the inode number to identify the file , No longer consider the file name

Link file

Create link files for files or directories
Link file categories

	Soft link (also known as symbolic link)	Hard link
After deleting the original file	Failure	Still available
Use range	Suitable for files or directories	Can only be used for files
Save location	Can be located in a different file system from the original file	Must be in the same file system (such as a Linux partition) with the original file

Hard link

Generally, there is a one-to-one correspondence between file name and inode number, and each inode number corresponds to a file name.
However, the Linux system allows multiple file names to point to the same inode number. This means that you can use different file names to access the same content. The
ln command can create a hard link. The basic format of the command is
ln source file target.
You cannot hard link a directory.
After running this command, the source file and target file have the same inode number, and both point to The same inode.
The "number of links" in the inode information will increase by 1.
When a file has multiple hard links, modifying the file content will affect all file names, but deleting one file name will not affect the other file name. Access and delete a file name will reduce the "number of links" in the inode information by 1

Soft link

A soft link is to create a separate file again, and this file will let the data read point to the file name of the file it is connected to.
The basic format of the soft link creation command is:
ln [-s] source file or directory... link file Or target location

Recover accidentally deleted files

Recover XFS type files

xfsdump command format
xfsdump -f backup storage location path or device file to be backed up

xfsdump backup level (default is 0)
0: full backup
1-9: incremental backup

xfsdump commonly used options: -f, -L, -M, -s

xfsrestore -f The location of the restored file to store the location of the restored file

xfsdump usage restrictions.
Only the mounted file system can be backed up. You
must use root privileges to operate.
Only the XFS file system can be backed
up. The data after the backup can only be analyzed by xfsrestore.
Two file systems with the same UUID cannot be backed up.

Recover EXT type files

Install the extundelete package

[root@localhost ~]# yum -y install e2fsprogs*
[root@localhost ~]# yum -y install wget
[root@localhost ~]# cd /opt
[root@localhost opt]# wget http://nchc.dl.sourceforge.net/project/extundel/extundelete-0.2.4.tar.bz2
[root@localhost opt]# ls
data  extundelete-0.2.4.tar.bz2

Format a disk into ext4 format

[root@localhost ~]# mkfs.ext3  /dev/sdb1 
mke2fs 1.42.9 (28-Dec-2013)
文件系统标签=
OS type: Linux
块大小=4096 (log=2)
分块大小=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1310720 inodes, 5242624 blocks
262131 blocks (5.00%) reserved for the super user
第一个数据块=0
Maximum filesystem blocks=4294967296
160 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000

Allocating group tables: 完成                            
正在写入inode表: 完成                            
Creating journal (32768 blocks): 完成
Writing superblocks and filesystem accounting information: 完成   

[root@localhost opt]# mkdir data   
[root@localhost opt]# mount /dev/sdb1 /opt/data

[root@localhost opt]# cd data/
[root@localhost data]# ls
lost+found
[root@localhost data]# touch aa bb cc dd 
[root@localhost data]# ls
aa  bb  cc  dd  lost+found

[root@localhost data]# rm -rf aa bb
[root@localhost data]# ls
cc  dd  lost+found

[root@localhost data]# cd ~
[root@localhost ~]# umount /opt/data/

[root@localhost ~]# extundelete /dev/sdb1 --restore-all	
NOTICE: Extended attributes are not restored.
Loading filesystem metadata ... 160 groups loaded.
Loading journal descriptors ... 28 descriptors loaded.
Searching for recoverable inodes in directory / ... 
0 recoverable inodes found.		
Looking through the directory structure for deleted files ... 
0 recoverable inodes still lost.
No files were undeleted.
[root@localhost opt]# mount /dev/sdb1 /opt/data
[root@localhost ~]# cd RECOVERED_FILES/	
[root@localhost RECOVERED_FILES]# ls		
aa  cc
[root@localhost RECOVERED_FILES]# cp a /opt 		
[root@localhost RECOVERED_FILES]# ls /opt
aa  bb  cc  dd  lost+found

Log file

Log function

Used to record various events that occur in the system and program operation.
By reading the log, it is helpful to diagnose and solve system failures
. The classification of log files The classification of
kernel and system

• Kernel and system logs are
  managed uniformly by the system service rsyslog, and the log format is basically similar
• User log
  Record system user login and logout related information
• Program log
  A log file independently managed by various applications, the record format is not uniform

Log file save location The
default location: /var/log directory

Introduction to main log files

Kernel and public message log	/ var / log / messages
Scheduled task log	/ var / log / cron
System boot log	/ var / log / dmesg
Mail system log	/ var / log / maillog
User login log	/ var / log / lastlog 、 / var / log / secure 、 / var / log / wtmp 、 / var / log / btmp

Kernel and system logs are
managed uniformly by the system service rsyslogd
Package: rsyslog-7.4.7-16.el7.x86_64
Main program: /sbin/rsyslogd
Configuration file: /etc/rsyslog.conf

The level of the log message

The log files managed by the rsyslogd service are the most important log files in the Linux system. They record the most basic system messages
in the Linux system such as the kernel, user authentication, mail, and scheduled tasks. In the Linux kernel, according to the importance of the log messages Different, divide it into different priorities (the smaller the number, the higher the priority, the more important the message)

level	Explanation
0 EMERG (emergency)	Will cause the host system to be unavailable
1 ALERT (warning)	Problems that must be resolved immediately
2 CRIT (serious)	More serious situation (some functions are not available)
3 ERR (error)	Run error
4 WARNING	Events that may affect system functions
5 NOTICE (note)	Will not affect the system but it is worth noting
6 INFO (information)	General information
7 DEBUG (debugging)	Program or system debugging information, etc. (may be used during maintenance)

The kernel and most system messages are recorded in the public log file /var/log/messages, while some other program messages are recorded in their own independent log files.
Log messages can also be recorded to a specific storage device or sent directly To designated users

User log

Save directory
saves user login, logout and other related information

/var/log/lastlog：最近的用户登录事件

/var/log/wtmp：用户登录，注销及系统开，关机事件

/var/log/utmp：当前登录的每个用户的详细信息

/var/log/secure：与用户验证相关的安全性事件

Program log analysis

Manage
web services independently by the corresponding program : /var/log/httpd/

• Access_log, error_log
  proxy service: /var/log/squid/
• access.log, cache.log
  FTP service: /var/log/xferlog

analyzing tool

• Text view, grep filter search, view in webmin management suite
• awk, sed and other text filtering, formatting editing tools
• Special log analysis tools such as webalizer and awstats

Log management strategy

Make timely backups and archives
Extend log retention period
Control log access rights

• Logs may contain various sensitive information, such as accounts, passwords, etc.
  Centralized management of logs
• Send the server's log file to the unified log file server
• Facilitate the unified collection, sorting and analysis of log information
• Prevent accidental loss, malicious tampering or deletion of log information