Liunx-- summary of
disk storage and file systems
Magnetic disk storage
mechanical hard disk: the disk, head, and control of the disk spindle motor, the head controller, data converters, interfaces, components, and several cache
SSD: solid state electronic storage hard disk array chip made by the control unit and a storage unit (FLASH chips, DRAM chips) composition
First, the hard disk storage term
head: the head
track: Track
cylinder: cylinder - the same track called column
sector: the sector, 512bytes
fdisk -l / dev / sd # - Observation hard disk indicator
CHS-bit addressing with 24bit
wherein the front cylinder 10 denotes a cylindrical intermediate 8 represents the head, behind the six sector represents the sector
early chs manner by addressing the maximum addressable space 8GB 512 2 ^ 62 ^ 8 * 2 ^ 10
The LBA (Logical Block Addressing)
LBA is an integer, by converting into a specific address in CHS format the disk to complete
the definition of 28-bit addressing mode ATA-1 specification, the sector 512 per group is calculated, ATA-1 Suo LBA 28 reaches the upper limit defined 128 GiB. In 2002 ATA-6 48 specification uses LBA, maximum capacity per sector is calculated in the same group of up to 512 128 Petabytes
Second, the two kinds of disk partitioning: MBR, GPT
MBR partition a hard drive up to four primary partitions, a main partition 3 may be extended + 1 (N logical partitions)
MBR partition structure
of the MBR The MBR consists of four parts (sector 0 track 0 512Byte)
1. Main boot program (offset address 0000H - 0088H), which is responsible for loading the active partition, and a system boot program run
error information data area, the offset address 0089H - 00E1H as an error message, 00E2H - 01BDH bytes are all 0
2. partition table (DPT, Disk | partition | table ) having four partition entries, the offset address 01BEH - 01FDH, each partition entry is 16 bytes long, 64 bytes total partition entries 1, a partition entry 2 partition item 3, item 4 partition
3. the end flag word offset address 01FE - 01FF is 2 bytes end flag 55AA
GPT partition: GPT: GUID (Globals Unique Identifiers) partition | table (partition table) partition 128, 64-bit, support 8Z (512Byte / block) 64Z (4096Byte / block)
Use 128 UUID (Universally Unique Identifier) represents a disk and partition, GPT partition table in the automatic backup two head and tail, and the CRC check bits
hexdump -C -n 512 / dev / sd # - front view the hard disk 512 bytes
dd if = / dev / sda of = / data / mbr bs = 1 count = 64 - dd backup partition table (partition table on it Office)
. . . Partitioning tool
operations are parted command with immediate effect, be careful with
the use of: parted [option] ... [equipment [command [argument] ...] ...]
parted / dev / sdb mklabel gpt | msdos-- fight then mark the partition gpt | msdos
parted the -l View partition Table: gpt partition information listed
parted / dev / sdb print print the partition information
parted / dev / sdb mkpart primary 1 200 ( default M) - partition successive partitions do not jumping points
parted / dev / sdb rm 1 delete partition
fdisk mbr and gdisk gpt-- substantially the same command format is not repeated
fdisk / dev / sd # interactive partition management
p partition list
t to change the partition type change ID
n-create a new partition -p master (1-4) e 1 starts to extend the default own 2M + \ G \ K
- partition should be noted that continuous partition, do jumping points
d delete partition partition numbers correspond to numbers easy to change the zoning
v verify the partition
u conversion unit
w to save and exit out immediately executed will change
q does not save and exit
. . Divided over the memory area may appear not to load a new partition information
lsblk
LL / dev / sdb *
CAT / proc / partitions-- to see that all the hard disk information memory
fdisk -l / dev / sd # - see a hard disk partition status of all
partprobe 5,7 synchronizing disk information entOS
. . . .
When centos6 partx -a / dev / DEVICE create a synchronization
partx -d when --nr 6-7 / dev / DEVICE delete synchronization
File system
file system is a method for operating a system for specific files on a storage device or partition, and a data structure; i.e., a method for organizing files on a storage device. Operating system responsible for managing and storing software file structure information is called a file management system, referred to as the file system
View system Supported file systems: / lib / modules / uname -r / kernel / fs
various file systems:
https://en.wikipedia.org/wiki/Comparison_of_file_systems
ext4: ext is the latest version of the file system. Offers many new features, including a nanosecond timestamp, create and use a giant file (16TB), the largest 1EB file system, as well as to enhance the speed of
xfs: SGI, support up to 8EB file system
btrfs (Oracle), reiserfs, jfs (AIX), swap
CD: iso9660
Windows: FAT32, exFAT, NTFS
Unix: FFS (the FAST), UFS (UNIX), JFS2
network file system: NFS, CIFS
cluster file systems: GFS2, OCFS2 (oracle)
distributed file system : FastDFS, Ceph, MooseFS, MogileFS, GlusterFS, Luster
the RAW: untreated or unformatted generated file system
lsblk -f view the file system
a file system creation command (two hundred twenty-one following command set, each of the substantially common option)
the mkfs command
(. 1) mkfs.FS_TYPE / dev / the DEVICE
ext4-XFS-Btrfs-vfat
(2) the mkfs fs_type -t / dev / the DEVICE
-L 'the lABEL' set volume label
-f forces already exists xfs heavy
mkfs -b size arranged at a fast speed in units
mke2fs: ext series dedicated file system management tool
-t {ext2 | ext3 | ext4} specifies the file system type
-b {1024 | 2048 | 4096} specify the block size
-L 'lABEL' set volume label
-j corresponds ext3 -t
the mkfs.ext3 the mkfs -t ext3 = = = the mke2fs the mke2fs -j -t ext3
-i # create space for the data for each byte inode number; it should not be less than block size
-N # specify how many partitions to create inode
-I inode a record amount of disk space, 128 --- 4096
-m # default 5%, of the total space reserved space for executives percentage
-O FEATURE [, ...] enable the specified characteristics
-O ^ FEATURE Closes the specified characteristics
ext text setting system tune2fs series tunable parameters values
-l See specified file system superblock information; Super Block
-L 'the LABEL' modified label
-m # repair percentage of space reserved to the administrator
-j be upgraded ext2 ext3
-O enable or disable the file system properties, -O ^ has_journal
-o adjust the default file system mount options, -o acl ^
-U UUID modify UUID number
dumpe2fs: the disk block group management
-h: View superblock information, do not display group information
File system check and repair: usually occurs in a non-normal shutdown after crash or
mounted as a file system is marked as "no clean"
Note: Be sure not to repair in the mounted state
fsck: File System Check - fsck / dev / sdb3 -y (auto-answer) and then fix any data loss
fsck.FS_TYPE
fsck -t fS_TYPE
-p automatically fix errors
-r interactive fix the error
fS_TYPE must have the same file on the partition type
e2fsck: ext series of documents dedicated check and repair tool
-y automatically the answer is yes
-f Force fixing
. . .
centos6 etx4 subsequent addition of a hard disk partition is no acl rights
fdisk ne + 100M
mkfs.etx4 / dev / sda6
tune2fs the -l / de / sda6
mount into the disk set acl permissions
tune2fs -o acl / dev / sdb6 add acl rights
tune2fs -o ^ acl / dev / sdb6 go
File System Tag: Another way of pointing device - device independent
blkid [OPTION] [DEVICE] - See block device -U UUID attribute information to find the corresponding apparatus according to the specified the UUID
-L LABEL The LABEL specified to look corresponding device
e2label: file system management ext series the e2label the dEVICE the lABEL [the lABEL]
findfs: find the partition
findfs [options] lABEL = <label > findfs [options] UUID = <uuid>
Two, mount mount
The existing directory of a file system with additional root file system to establish relationships, thus making this directory as the file access behavior of other entry
mount by looking at etc / mtab file shows / all equipment that is currently mounted
Mount [-fnrsvw] [- T vfstype] [- O Options] the dir Device
Device: specified device will be mounted;
(1) the device file: e.g. / dev / sda5
(2) label: -L 'LABEL', e.g. -L 'MYDATA'
(. 3) the UUID, -U 'the UUID': e.g. -U '0c50523c-43f1-45e7-85c0-a126711d406e'
(. 4) pseudo file system name: proc, the sysfs, devtmpfs, CONFIGFS
the dir: mount point
pre-existing, empty directory is recommended
process is using the device can not be uninstalled
mount common command options - View to mount a device mount command
-t vsftype specified file system to be mounted on the device type
-r readonly, read-only hang load
-w read and write, read mount
-n Do not update / etc / mtab, 6mount invisible CAT / etc / mtab
CAT / proc / mounts viewing the kernel to track all the devices have been mounted
centos6 / etc / mtab / proc / mounts
-a automatically mounted automatically mount all supported devices (defined in the / etc / fstab file, and the mount option functions are auto)
-L 'the lABEL' to specify the label mount device
-U ' UUID 'to specify the device to be mounted UUID
-B,--bind binding directory to another directory (mount directory)
- When two files inode may be the same as when the directory is mounted
-mount -B / boot / grub2 / mnt / grub2
Uninstall command
findmnt MOUNT_POINT | device-- to see whether the file to mount the $? Value judgment
view is accessing the specified file system processes
lsof MOUNT_POINT-- to see who is accessing
fuser -v MOUNT_POINT-- see who has access to
kill all processes that are accessing the file system specified by the
fuser -km MOUNT_POINT
unloading umount DEVICE umount MOUNT_POINT
mount / dev / sda / mnt / sda - uuid or label can be linked to a unique well of the mount uuid
File system space occupancy and other tools to view information
df [OPTION] ... [FILE] ...
-H with 10 units -T file system type Human-readable -i inodes -h INSTEAD of Blocks
-P to Posix-compliant output format
view a directory of overall space occupied state
du [OPTION] ... DIR -h human -readable sum of the Summary -s
--max-depth = # specify the maximum directory hierarchy
. . .
Empty or sparse file
dd IF = / dev / ZERO of = / Data / BIGFILE COUNT = 1023 BS = 1M Seek = 2048
LS -H BIGFILE 3G - see logical size
du -sh bigfile 1G - occupy only 1G df / the actual size of the data is seen
loop device associated file
ll / dev / loop c6 loop device default 0-78 a view
dd IF = / dev / ZERO of = / data / BIGFILE 1M COUNT = 1023 BS = 2048 = Seek
the mkfs. ext4 / Data / BIGFILE
the losetup / dev / LOOP6 / Data / device associated bigfile--
the losetup -a
the losetup -d
. . . .
mknod creates a special file basically anything can be created
mknod / dev / loop100 b 7 8 - Create loop device
RM -rf loop100
/boot/grub/grub.conf below may be restarted max_loop = 100
ll / devloop
mount -o ptions: Use commas to separate multiple options
async asynchronous mode sync sync mode, memory changes, at the same time writing to disk..
atime / the noatime contain directories and files (stored for more than one day to be updated or mtime> atime time
diratime / nodiratime directory access timestamp
auto / noauto support auto-mount support - a option
exec / noexec support to run the application on file system
dev / nodev whether to support the use of device files on the file system
suid / nosuid support suid and sgid permissions
remount remount
ro read only rw read and write
user / nouser whether to allow ordinary users to mount this device, / etc / fstab use
acl acl-enabled features on this file system
loop using the loop device
Mount profile file
/ etc / fstab per line to define a mounted file system
1, the device to be mounted or pseudo file system
device files LABEL: LABEL = "" UUID: UUID = ""
pseudo file system name: proc , the sysfs
2, the mount point
3, the file system type: ext4, XFS, ISO9660, NFS, none
. 4, mount options: defaults, acl, bind -o option
5, dump frequency: 0: 1 backup is not: dump day 2: Transfer every other day
6, fsck order to check the file system: the allowed number 1 2 0
0: no self-test
1: first, the self-test; rootfs generally only used only
2: non rootfs use
. . .
/ dev / sdr6 / mnt / sda6 ext4 defaults 0 3
hole used to mount the file writing uuid in fstab
/ Boot / mnt / Boot none 0 0 file linked to the bind file configuration
mount-a automatically mount all supported devices automatically mount and mount options have auto function
Three, swap
swap supplement system RAM, Swap partitions support virtual memory. When the data is not enough RAM to save the system processed data is written to a swap partition
when the system lacks swap space, the kernel memory exhaustion due to RAM and terminate the process. Configuring too much swap space will cause the storage device is assigned but idle state, wasteful, excessive swap space will cover up the memory leak
Mount swap partition - basic settings include: • Create a swap partition or file
• Use mkswap write special signature • / fstab file, add the appropriate entry in the / etc • Use swapon -a activate swap space
Enable: swapon
swapon [OPTION ] ... [the DEVICE]
-a: activating all swap
-p pRIORITY: Specifies the priority
/ etc / fstab in the fourth column: pri = value
disabled: the swapoff [the OPTION] ... [the DEVICE]
the sWAP priority level
can be specified swap area 0-32767 priority, the greater the higher the priority value
if the user is not specified, then the kernel will automatically swap to specify a priority, the priority start from -1, each addition of a new user is not specify the priority of swap, this will give priority minus one
to add a swap of default is relatively high priority, unless the user has their own designated a priority, and user-specified priority (is positive) is always higher than the default specified core priority (negative)
to optimize performance: distributed storage, high-performance disk storage
. . .
See sub-swap area free -h memory usage
fdisk / dev / sdc pmt L 19 W modified partition disk ID (GPT)
the mkswap swap file system creates = mkswap / dev / sdc1
written free -h fstab file did not see
swapon -a -h as Free open swap
CAT / proc / swaps to view the swap partition specific number = swapon -s
swapoff / dev / sdc1 disabled
Supplementary
dd command: Convert A File Copy and
dd = IF / the PATH / the FROM / of the SRC = / the PATH / the TO / BS DEST = # = # COUNT
IF = File read from the named file rather than from the standard input
of = file write to the named file rather than to standard output
ibs = size a read of size byte
OBS = size a write of size byte
BS = size block size, specify block sizes (both are ibs also OBS)
CBS = size in a conversion of size byte
skip = blocks from the start block blocks a negligible size ibs
seek = blocks from the start block blocks a negligible size obs
count = n Copies n BS
CONV conversion = [, ... conversion] transformation parameters transform file with the specified parameters:
dd IF = / Data / file of = fstab CONV = ucase
ASCII converted EBCDIC to ASCII
converter EBCDIC ASCII to EBCDIC
LCASE to uppercase to lowercase
ucase converts lowercase to uppercase
nocreat does not create an output file
does not stop noerror error
notrunc not truncate the output file
sync to fill each input block to ibs bytes, and the gap filled with air (the NUL) character
Fdatasync completed before the write, physical write output file, read and write speed is determined
Backup MBR
dd IF = / dev / SDA of = / tmp / mbr.bak BS = 512. 1 COUNT =
destroy the MBR the bootloader
dd IF = / dev / ZERO of = / dev / = 64 COUNT = SDA BS = 446. 1 Seek
there is a large binary file fileA and of 2K. Now want to read from the 64th byte position, the size needs to be read is 128Byts. FileB have, trying to read the above 128Bytes written to the position of the start 32 bytes, substitutions 128Bytes, to achieve the following
dd if = fileA of = fileB bs = 1 count = 128 skip = 63 seek = 31 conv = notrunc
1, the backup:
dd IF = / dev / sdx of = / dev / SDY
backup local / dev / sdx entire disk to / dev / SDY
dd IF = / dev / sdx of = / path / to / Image
and / dev backup / sdx overall image data file to the specified path
dd if = / dev / sdx | gzip> /path/to/image.gz
backup / dev / sdx overall data, using gzip compression, to the specified path
recovery:
dd IF = / path / to / image of = / dev / sdx
to restore the backup file to the specified disk
gzip -dc /path/to/image.gz | dd of = / dev sdx /
compressed backup file into the specified tray
2, copy the data to the hard disk memory
dd if = / dev / mem of = / root / mem.bin bs = 1024
will be copied to the memory of the data file in the root directory mem.bin
3, copied from the CD iso image
dd if = / dev / cdrom of = / root / cd.iso
copy the data to the optical disc under the root folder, and the file is saved as cd.iso
4, disk data destruction
dd if = / dev / urandom of = / dev / sda1
random data to fill a hard disk, is necessary in some situations can destroy data after performing this operation, / dev / sda1 will not mount create and copy operation can not be performed
5, to obtain the most appropriate size Block
dd IF = / dev / ZERO of = / the root / 1Gb.file COUNT = 1024 BS = 1000000
dd IF = / dev / ZERO of = / the root / BS 1Gb.file COUNT = 2048 = 500000
dd if = / dev / zero of = / root / 1Gb.file bs = 4096 count = 250000
by comparing dd instruction execution time of the output command, the block size to determine the best size system
6, test disk write speed
dd if = / dev / zero of = / root / 1Gb.file bs = 1024 count = 1000000 conv = fdatasync
test disk read speed
dd if = / root / 1Gb.file bs = 64k | dd of = / dev / null conv = fdatasync