Linux online storage management (a) FC disk device management

 

Please reprint the first to keep the original source in the text: EMC Chinese Support Forum - https://community.emc.com/go/chinese

Introduction

 

This paper describes the FC disk management under Linux environment is only relevant, introduced the main contents include FC drive, Linux SCSI disk permanent name, add and delete FC disk, FC disk path to add online scan and delete storage and FC link.

More information

 

Fibre Channel drive

 

Fibre Channel Linux driving system space user interface is located / sys / class folder. The following entries, H denotes a host, B on behalf of bus number, T representative of target, LUN ID behalf L, R & lt representative port number.

note:

If the host before using a multi-pathing software, modify the configuration parameters, recommendations of the Advisory multipathing software vendors

/ Sys / class directory under three main folders file folder associated with fibre channel fc_transport, fc_remote_ports, fc_hosts. Wherein fc_transport information representative of the allocated storage; fc_remote_ports agent host to the memory port link information (fc_remote_ports folder contains link information store is not allocated to the host); fc_host agent host HBA card information.

 

Transport： /sys/class/fc_transport/targetH:B:T/

• port_id - 24 bit port switch port ID storage
• node_name - 64 bit memory port node name
• port_name - 64 bit port name storage port

 

Remote Port：/sys/class/fc_remote_ports/rport-H:B-R/

• port_id - 24 bit port switch port ID storage
• node_name - 64 bit memory port node name
• port_name - 64 bit port name storage port
• dev_loss_tmo - link failure latency. Faulty link no longer process any new IO. The default value depending on the specific dev_loss_tmo HBA on card, Qlogic default is 35 seconds, Emulex default is 30 seconds. HBA card comes with driver can override the value of this parameter. dev_loss_tmo maximum 600 seconds, if the value is less than 0 or greater than dev_loss_tmo 600, HBA timeout value comes into force.
• fast_io_fail_tmo - IO fault latency. Link fluctuations, IO try long.

 

Host：/sys/class/fc_host/hostH

• port_id - 24 bit switch port ID HBA port.
• issues_lip - Reset HBA port, try again found storage ports.

 

Persistent Naming

 

The operating system sends IO to the storage path through, Linux SCSI disk path system has the following components:

• HBA PCI card identifier
• HBA card of pipe number
• SCSI target address storage end
• LUN(Logical Unit Number) 号

SCSI disk path has on Linux 3 in expressions: a, / dev / sd catalog; II. Major: minor number; three, / dev / disk / by-path, the directory is flexible connection / dev / sd device . Fiber Channel disk path model as follows:

 

The above three SCSI disk path is not a permanent change, add or delete when the new PCI device when the server, the path will change, sometimes even restart the server may also lead to changes in the path becomes.

To ensure the disk path that the application can be a permanent change, in the following ways:

1. WWID

According to the SCSI standard, SCSI disks each have a WWID. Similar card MAC address is unique requirements. Marked by WWID SCSI disk disk path can ensure permanent change, on Linux / dev / disk / by-id for each SCSI disk WWID directory contains the access path. Example:

scsi-3600508b400105e210000900000490000 -> ../../sda

 

Tip: Linux comes with the device-mapper-multipath tool is through WWID to detect SCSI disk path, you can merge multiple paths to the same device, and creates a new path in the device / dev / mapper / below. You can see the disk WWID path through the -l multipath, Host: Channel: Target: Lun and / dev / sd and major: minor correspondence.

 

1. UUID

UUID is generated in the file system creation time, used to mark the file system, similar to the same WWID is unique. So use UUID to mark SCSI disk, but also to ensure the path is permanent. On Linux / dev / disk / by-uuid you can see key mapping between the file system and disk devices and / dev / sd each has been created.

Note: Linux comes with md and LVM tools are also written in the SCSI disk UUID information above.

1. UDEV

UDEV is Linux provides a mechanism to allow users to customize the device named. It can WWID / UUID information with disk path mapped up by UDEV, so you can ensure that equipment path permanent.

 

 

Disk Delete

 

 

Before removing disks is recommended that you back up the data, the memory dirty data written to disk, and then delete all associated disk path. For disk multipathing software, you need to delete multiple paths and each path disk device. Recommended Action Delete disk when the system is idle, dirty data is written to disk memory will increase the load on the system, it can vmstat 1 100 observation system load. If you meet one of the conditions it is not recommended to delete operations:

• vmstat 100 times more than 10 times the output of the memory is less than 5% free system memory.
• si vmstat results and so the column is not empty, the system represents the ongoing swaping, the memory data is written to disk.

Delete disk, follow these steps:

1. Close process using disk backup disk data. You can view process is accessing a disk through the fuser command.
2. umount unmount the file system on the disk to be deleted
3. md and LVM remove the disk. LVM can be used to change vgreduce remove the disk from the volume group, and then use the pvremove delete LVM metadata from the disk.
4. If the disk multipathing software, view the disk all paths through mulitpath -l, and then remove the disk by multipath -f. (If you use powerpath multipathing software, please refer powerpath operating manual)
5. blockdev -flushfs dirty data is written to disk. This step is especially important for things bare equipment case, because the device can not bare the dirty data is written to disk by umount or vgreduce.
6. Disk path delete the application or script references.
7. Use command echo. 1> / SYS / Block / Device-name / Device / Delete remove the disk, Device-name beginning sd, such sda, sdb; or use the command echo. 1> / SYS / class / the scsi_device / H: C: T : L / Device / delete the delete disk, H represents the HBA card number, c represents the HBA card Channel, t represents the target ID SCSI, i represents the LUN ID. h: c: t: l This information can be viewed by lsscsi, scsi_id, multipath -l, ls -l / dev / disk / by- * mode.
8. Delete disk files.

 

Delete disk path

 

When the system software when using a multi-path, a path can be deleted without affecting the online business use. Steps are as follows:

1. Delete disk path to the application in an application or script.
2. Use the command echo offline> / sys / block / sda / device / state disk path offline. Multi-path software will use the remaining path processing IO.
3. 1 Use the command echo> / SYS / Block / Device-name / the Delete command to remove the disk path from the SCSI subsystem, Device-name usually starts with sd, such as sda, sdb.

 

Add a new disk or disk path

 

Add a new disk or disk path, the system may automatically assign a name to use the old disk to the new disk, such as / dev / sd, major: minoe and / dev / disk / by-path. Therefore, before adding, make sure that applications and scripts have been deleted a reference to the old disk.

 

Add a new disk or disk path as follows:

1. Complete switch and storage configuration, record number storage port WWPN.
2. Use the following command to re-scan the disk devices on the system.

echo “c t l” > /sys/class/scsi_host/hosth/scan

h representative of HBA number, C representative of the HBA Channel, T Representative target ID the SCSI, L Representative LUN ID.

Using a command echo "scsi add-single-device 0 0 0 0"> / proc / scsi / scsi complete the scan.

1. Some HBA card in the memory allocation, we need to discover newly added devices issue_lip to the specific reference to "store link scanning." (If you need issue_lip, recommend stopping IO operations)
2. Newly assigned LUN is not shown in the operating system, the array may be re-acquired by sg_luns LUN list command (from sg3_utils package).
3. c: t: l information can use the command grep "store SP port WWPN" / sys / class / fc_transport / * / [node_name | port_name] Gets; can also lsscsi, scsi_id, multipath -l or ls -l / dev / disk / by- * method to get.
4. Use multipath software multipath (or other multi-path software) to confirm the disk path is added to normal.

Storage link scanning

 

Linux operating system provides a variety of storage link reset operation. Memory reset link is typically used to add or remove multipath device, which is a destructive operation, IO operations will result in a timeout. Please use caution this type of operation, please note the following additional operation:

1. Make sure the link is not operating before the new IO, dirty data has been written back to memory.
2. Under the system memory resource usage tensions, not recommended link deletion. The system memory usage can be evaluated by vmstat 1 100 command.

The following command can be used to store link rescan:

• echo "1" > /sys/class/fc_host/host/issue_lip

issue_lip reset HBA link, and the link re-scan the entire configuration SCSI target. This type of operation is an asynchronous operation, specifically the time required to complete the reference / vag / log / message log. Linux operating system comes with driver support issue_lip lpfc and qla2xxx command.

• echo “- - - ” > /sys/class/scsi_host/hostH/scan
• rmmod drive / modprobe drive delete / reload the HBA driver.

reference

 

• Redhat Linux – Storage Administration Guide