一,概述
SCSI(Internet SCSI) 支持从客户端 ( 发起端 ) 通过 IP 向远程服务器上的 iSCSI 存储设备 ( 目标 ) 发送 SCSI 命令。
iSCSI限定名称用于确定发起端和目标 , 并采用 iqn.yyyy-mm.{reverse domain}:label 的格式
默认情况下 , 网络通信是至 iSCSI 目标上的端口 3260/tcp 的明文端口ISCSI 名称设定
iSCSI 发起端 : 需要访问原始 SAN 存储的客户端
iSCSI 目标 : 从 iSCSI 服务器提供的远程硬盘磁盘 , 或“目标门户”
iSCSI 目标门户 : 通过网络向发起端提供目标的服务器。
SCSI(Internet SCSI) 支持从客户端 ( 发起端 ) 通过 IP 向远程服务器上的 iSCSI 存储设备 ( 目标 ) 发送 SCSI 命令。
iSCSI限定名称用于确定发起端和目标 , 并采用 iqn.yyyy-mm.{reverse domain}:label 的格式
默认情况下 , 网络通信是至 iSCSI 目标上的端口 3260/tcp 的明文端口ISCSI 名称设定
iSCSI 发起端 : 需要访问原始 SAN 存储的客户端
iSCSI 目标 : 从 iSCSI 服务器提供的远程硬盘磁盘 , 或“目标门户”
iSCSI 目标门户 : 通过网络向发起端提供目标的服务器。
IQN:“iSCSI 限定名称”。每个发起端和目标需要唯一名称进行标识最好的做法是使用一个在 Internet 上可能独一无二的名称
二,安装配置iscsi
服务端
[root@server ~]# fdisk -l >>>>>查看磁盘 Disk /dev/vda: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x00013f3e Device Boot Start End Blocks Id System /dev/vda1 * 2048 20970332 10484142+ 83 Linux Disk /dev/vdb: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes [root@server ~]# fdisk /dev/vdb >>>>>>建立一个磁盘分区用来做共享分区 Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Device does not contain a recognized partition table Building a new DOS disklabel with disk identifier 0xce7e4760. Command (m for help): p Disk /dev/vdb: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0xce7e4760 Device Boot Start End Blocks Id System Command (m for help): n Partition type: p primary (0 primary, 0 extended, 4 free) e extended Select (default p): Using default response p Partition number (1-4, default 1): First sector (2048-20971519, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-20971519, default 20971519): +2G Partition 1 of type Linux and of size 2 GiB is set Command (m for help): p >>>>>打印分区表 Disk /dev/vdb: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0xce7e4760 Device Boot Start End Blocks Id System /dev/vdb1 2048 4196351 2097152 83 Linux Command (m for help): wq The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@server ~]# partprobe >>>>>>同步分区表 Warning: Unable to open /dev/sr0 read-write (Read-only file system). /dev/sr0 has been opened read-only. Warning: Unable to open /dev/sr0 read-write (Read-only file system). /dev/sr0 has been opened read-only. Warning: Unable to open /dev/sr0 read-write (Read-only file system). /dev/sr0 has been opened read-only. [root@server ~]# yum install targetcli -y >>>>>>安装iscsi软件管理包 [root@server ~]# systemctl status target target.service - Restore LIO kernel target configuration Loaded: loaded (/usr/lib/systemd/system/target.service; disabled) Active: inactive (dead) [root@server ~]# systemctl start target >>>>>>启动 [root@server ~]# systemctl enable target ln -s '/usr/lib/systemd/system/target.service' '/etc/systemd/system/multi-user.target.wants/target.service' [root@server ~]# targetcli Warning: Could not load preferences file /root/.targetcli/prefs.bin. targetcli shell version 2.1.fb34 Copyright 2011-2013 by Datera, Inc and others. For help on commands, type 'help'. /> ls >>>>>查看 o- / ......................................................................... [...] o- backstores .............................................................. [...] | o- block .................................................. [Storage Objects: 0] | o- fileio ................................................. [Storage Objects: 0] | o- pscsi .................................................. [Storage Objects: 0] | o- ramdisk ................................................ [Storage Objects: 0] o- iscsi ............................................................ [Targets: 0] o- loopback ......................................................... [Targets: 0] /> /backstores/block create westos:storage1 /dev/vdb1 >>>>>把/dev/vdb1作为磁盘分区,起名为westos:storage1 Created block storage object westos:storage1 using /dev/vdb1. /> /iscsi create iqn.2018-06.com.example:storage1 >>>>>创建iscsi限定名称 格式为iqn.年-月.域名反写:label标示 Created target iqn.2018-06.com.example:storage1. Created TPG 1. /> ls >>>>>再次查看 o- / ......................................................................... [...] o- backstores .............................................................. [...] | o- block .................................................. [Storage Objects: 1] | | o- westos:storage1 ............... [/dev/vdb1 (2.0GiB) write-thru deactivated] | o- fileio ................................................. [Storage Objects: 0] | o- pscsi .................................................. [Storage Objects: 0] | o- ramdisk ................................................ [Storage Objects: 0] o- iscsi ............................................................ [Targets: 1] | o- iqn.2018-06.com.example:storage1 .................................. [TPGs: 1] | o- tpg1 ............................................... [no-gen-acls, no-auth] | o- acls .......................................................... [ACLs: 0] | o- luns .......................................................... [LUNs: 0] | o- portals .................................................... [Portals: 0] o- loopback ......................................................... [Targets: 0] ######/iscsi/iqn.2018-06.com.example:storage1/tpg1/下面的三个都要设置 /> /iscsi/iqn.2018-06.com.example:storage1/tpg1/acls create iqn.2018-06.com.example:westoskey >>>>>创建acl密钥 Created Node ACL for iqn.2018-06.com.example:westoskey /> /iscsi/iqn.2018-06.com.example:storage1/tpg1/luns create /backstores/block/westos:storage1 >>>>>密钥和设备关联 Created LUN 0. Created LUN 0->0 mapping in node ACL iqn.2018-06.com.example:westoskey /> /iscsi/iqn.2018-06.com.example:storage1/tpg1/portals create 172.25.254.200 >>>>开放端口 Using default IP port 3260 Created network portal 172.25.254.200:3260. /> exit Global pref auto_save_on_exit=true Last 10 configs saved in /etc/target/backup. Configuration saved to /etc/target/saveconfig.json >>>>>提示配置信息的保存位置 注意:这个里面的命令可以tab显示出来,help查看命令格式
客户端
[root@client ~]# fdisk -l >>>>>>查看是否有共享磁盘 Disk /dev/vda: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x00013f3e Device Boot Start End Blocks Id System /dev/vda1 * 2048 20970332 10484142+ 83 Linux Disk /dev/vdb: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes [root@client ~]# yum search iscsi Loaded plugins: langpacks ================================= N/S matched: iscsi ================================== iscsi-initiator-utils.i686 : iSCSI daemon and utility programs iscsi-initiator-utils.x86_64 : iSCSI daemon and utility programs iscsi-initiator-utils-iscsiuio.x86_64 : Userspace configuration daemon required for : some iSCSI hardware libiscsi.i686 : iSCSI client library libiscsi.x86_64 : iSCSI client library Name and summary matches only, use "search all" for everything. [root@client ~]# yum install iscsi-initiator-utils.x86_64 -y >>>>>>安装客户端软件 [root@server ~]# systemctl status firewalld >>>>>>>服务端关闭防火墙 firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled) Active: inactive (dead) [root@client ~]# iscsiadm -m discovery -t st -p 172.25.254.200 >>>>>>>客户端发现设备 ####-t 类型 -p IP端口 [root@client ~]# iscsiadm -m node -T iqn.2018-06.com.example:storage1 -p 172.25.254.200 -l >>>>>>>客户端登陆 ####-T iqn共享设备名称 -p IP端口 -l 登陆 Logging in to [iface: default, target: iqn.2018-06.com.example:storage1, portal: 172.25.254.200,3260] (multiple) iscsiadm: Could not login to [iface: default, target: iqn.2018-06.com.example:storage1, portal: 172.25.254.200,3260]. iscsiadm: initiator reported error (24 - iSCSI login failed due to authorization failure) iscsiadm: Could not log into all portals >>>>>>>登陆失败,要先修改配置文件,添加密钥 [root@client ~]# vim /etc/iscsi/initiatorname.iscsi [root@client ~]# cat /etc/iscsi/initiatorname.iscsi >>>>>>>直接修改 InitiatorName=iqn.2018-06.com.example:westoskey [root@client ~]# systemctl restart iscsid.service >>>>>>>重启 [root@client ~]# iscsiadm -m discovery -t st -p 172.25.254.200 >>>>>>>>发现 [root@client ~]# iscsiadm -m node -T iqn.2018-06.com.example:storage1 -p 172.25.254.200 -l >>>>>>>>登陆成功 Logging in to [iface: default, target: iqn.2018-06.com.example:storage1, portal: 172.25.254.200,3260] (multiple) Login to [iface: default, target: iqn.2018-06.com.example:storage1, portal: 172.25.254.200,3260] successful. [root@client ~]# fdisk -l >>>>>>>查看设备 Disk /dev/vda: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x00013f3e Device Boot Start End Blocks Id System /dev/vda1 * 2048 20970332 10484142+ 83 Linux Disk /dev/vdb: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/sda: 2147 MB, 2147483648 bytes, 4194304 sectors >>>>>>出现共享设备 Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 4194304 bytes
三,客户端使用共享磁盘
[root@client ~]# fdisk /dev/sda >>>>>>对共享磁盘进行分区 Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Device does not contain a recognized partition table Building a new DOS disklabel with disk identifier 0x759cd7b5. Command (m for help): n Partition type: p primary (0 primary, 0 extended, 4 free) e extended Select (default p): Using default response p Partition number (1-4, default 1): First sector (8192-4194303, default 8192): Using default value 8192 Last sector, +sectors or +size{K,M,G} (8192-4194303, default 4194303): +500M Partition 1 of type Linux and of size 500 MiB is set Command (m for help): p Disk /dev/sda: 2147 MB, 2147483648 bytes, 4194304 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 4194304 bytes Disk label type: dos Disk identifier: 0x759cd7b5 Device Boot Start End Blocks Id System /dev/sda1 8192 1032191 512000 83 Linux Command (m for help): wq The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@client ~]# partprobe >>>>>>同步数据表 Warning: Unable to open /dev/sr0 read-write (Read-only file system). /dev/sr0 has been opened read-only. Warning: Unable to open /dev/sr0 read-write (Read-only file system). /dev/sr0 has been opened read-only. Warning: Unable to open /dev/sr0 read-write (Read-only file system). /dev/sr0 has been opened read-only. [root@client ~]# mkfs.xfs /dev/sda1 >>>>>>>格式化磁盘分区 meta-data=/dev/sda1 isize=256 agcount=8, agsize=16000 blks = sectsz=512 attr=2, projid32bit=1 = crc=0 data = bsize=4096 blocks=128000, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=0 log =internal log bsize=4096 blocks=853, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [root@client ~]# mount /dev/sda1 /mnt/ >>>>>>>手动挂载 [root@client ~]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/vda1 10473900 3159092 7314808 31% / devtmpfs 927072 0 927072 0% /dev tmpfs 942660 80 942580 1% /dev/shm tmpfs 942660 17028 925632 2% /run tmpfs 942660 0 942660 0% /sys/fs/cgroup /dev/sda1 508588 25888 482700 6% /mnt
四,客户端自动挂载
[root@client ~]# vim /etc/fstab /dev/sda1 /mnt xfs defaults,_netdev 0 0 >>>>>>>一定要写上_netdev
系统启动时默认先加载/etc/fstab后启动网络服务,但是如果不先启动网络,那么客户端和服务端就无法通信,这个文件的内容也就无法完成,系统就会一直卡住。
[root@client ~]# reboot [root@156 ~]# rht-vmctl poweroff desktop >>>>>>>第一次需要手动关机,再开机,否则会卡住关不了 Powering off desktop.. [root@156 ~]# rht-vmctl start desktop
#####如果没有写_netdev
#####进入单用户界面,改变系统根
####进入/etc/fstab然后添加,exit两次退出
五,客户端卸载
[root@client ~]# yum install tree [root@client ~]# tree /var/lib/iscsi/ >>>>>>查看树形结构 /var/lib/iscsi/ |-- ifaces |-- isns |-- nodes | `-- iqn.2018-06.com.example:storage1 | `-- 172.25.254.200,3260,1 | `-- default |-- send_targets | `-- 172.25.254.200,3260 | |-- iqn.2018-06.com.example:storage1,172.25.254.200,3260,1,default -> /var/lib/iscsi/nodes/iqn.2018-06.com.example:storage1/172.25.254.200,3260,1 | `-- st_config |-- slp `-- static [root@client ~]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/vda1 10473900 3159388 7314512 31% / devtmpfs 927072 0 927072 0% /dev tmpfs 942660 80 942580 1% /dev/shm tmpfs 942660 17032 925628 2% /run tmpfs 942660 0 942660 0% /sys/fs/cgroup /dev/sda1 508588 25888 482700 6% /mnt [root@client ~]# umount /mnt/ >>>>>>取消挂载 [root@client ~]# vim /etc/fstab >>>>>>删除开机自动挂载的命令 [root@client ~]# fdisk -l >>>>>>查看共享磁盘 Disk /dev/vda: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x00013f3e Device Boot Start End Blocks Id System /dev/vda1 * 2048 20970332 10484142+ 83 Linux Disk /dev/vdb: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/sda: 2147 MB, 2147483648 bytes, 4194304 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 4194304 bytes Disk label type: dos Disk identifier: 0x759cd7b5 Device Boot Start End Blocks Id System /dev/sda1 8192 1032191 512000 83 Linux >>>>>>设备仍然存在 [root@client ~]# iscsiadm -m node -T iqn.2018-06.com.example:storage1 -p 172.25.254.200 -u >>>>>> -u 表示unlogin Logging out of session [sid: 1, target: iqn.2018-06.com.example:storage1, portal: 172.25.254.200,3260] Logout of [sid: 1, target: iqn.2018-06.com.example:storage1, portal: 172.25.254.200,3260] successful. [root@client ~]# fdisk -l Disk /dev/vda: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x00013f3e Device Boot Start End Blocks Id System /dev/vda1 * 2048 20970332 10484142+ 83 Linux Disk /dev/vdb: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes >>>>>共享设备消失,只是暂时的 [root@client ~]# systemctl restart iscsi >>>>>>重启服务之后又会出现 [root@client ~]# fdisk -l Disk /dev/vda: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x00013f3e Device Boot Start End Blocks Id System /dev/vda1 * 2048 20970332 10484142+ 83 Linux Disk /dev/vdb: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/sda: 2147 MB, 2147483648 bytes, 4194304 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 4194304 bytes Disk label type: dos Disk identifier: 0x759cd7b5 Device Boot Start End Blocks Id System /dev/sda1 8192 1032191 512000 83 Linux >>>>>>共享设备仍然出现 [root@client ~]# iscsiadm -m node -T iqn.2018-06.com.example:storage1 -p 172.25.254.200 -u >>>>>>再次登出 Logging out of session [sid: 2, target: iqn.2018-06.com.example:storage1, portal: 172.25.254.200,3260] Logout of [sid: 2, target: iqn.2018-06.com.example:storage1, portal: 172.25.254.200,3260] successful. [root@client ~]# iscsiadm -m node -T iqn.2018-06.com.example:storage1 -p 172.25.254.200 -o delete >>>>> -o operation 删除操作 [root@client ~]# tree /var/lib/iscsi//var/lib/iscsi/ >>>>> 查看树形结构 |-- ifaces |-- isns |-- nodes |-- send_targets | `-- 172.25.254.200,3260 | `-- st_config |-- slp `-- static 7 directories, 1 file [root@client ~]# systemctl restart iscsi >>>>>重启服务 [root@client ~]# fdisk -l >>>>>查看设备 Disk /dev/vda: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x00013f3e Device Boot Start End Blocks Id System /dev/vda1 * 2048 20970332 10484142+ 83 Linux >>>>>>共享设备消失 Disk /dev/vdb: 10.7 GB, 10737418240 bytes, 20971520 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes