记一次linux pv missing故障

现象描述:

linux一个逻辑卷lv_home丢失,无法正常挂载,之前挂载在/home目录下:

[root@VM001 ~]# cat /etc/fstab 
.......
/dev/VolGroup/lv_home   /home                   ext4    defaults        0 0

查看文件系统挂载情况,发现lv_home并没有挂载:

[root@VM001 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
                       20G  7.3G   12G  40% /
tmpfs                 3.9G  224K  3.9G   1% /dev/shm
/dev/xvda1            477M  124M  328M  28% /boot

尝试手动挂载发现找不到lv_home这个逻辑卷:

[root@VM0001 ~]# mount /dev/VolGroup/lv_
lv_root  lv_swap  
[root@VM001 ~]# cd /dev/VolGroup/
[root@VM000000875 VolGroup]# ls
lv_root  lv_swap
[root@VM001 VolGroup]# ll
total 0
lrwxrwxrwx. 1 root root 7 Jul 31 11:06 lv_root -> ../dm-0
lrwxrwxrwx. 1 root root 7 Jul 31 11:06 lv_swap -> ../dm-1

查看逻辑卷状态,发现处于异常状态(NOT available):

[root@VM001 VolGroup]# lvdisplay  
.......
  --- Logical volume ---
  LV Path                /dev/VolGroup/lv_home
  LV Name                lv_home
  VG Name                VolGroup
  LV UUID                4rdajn-62q9-RX5T-QorD-vLsh-hMEv-zxT1Qj
  LV Write Access        read/write
  LV Creation host, time localhost.localdomain, 2016-03-17 09:19:33 +0800
  LV Status              NOT available
  LV Size                1.17 TiB
  Current LE             306815
  Segments               3
  Allocation             inherit
  Read ahead sectors     auto
[root@VM001 VolGroup]# lvscan 
  ACTIVE            '/dev/VolGroup/lv_root' [20.00 GiB] inherit
  ACTIVE            '/dev/VolGroup/lv_swap' [5.00 GiB] inherit
  inactive          '/dev/VolGroup/lv_home' [1.17 TiB] inherit

问题排查:

执行vgdisplay -v命令有2个pv missing信息,此处为关键报错信息:

[root@VM001 ~]# vgdisplay -v
    Using volume group(s) on command line.
    Finding all volume groups.
    Wiping cache of LVM-capable devices
    There are 2 physical volumes missing.
    There are 2 physical volumes missing.
  --- Volume group ---
  VG Name               VolGroup
  System ID             
  Format                lvm2
  Metadata Areas        4
...........

查看pv状态:

[root@VM001 ~]# pvs
  PV         VG       Fmt  Attr PSize    PFree   
  /dev/xvda2 VolGroup lvm2 a-m    49.51g       0 
  /dev/xvda3 VolGroup lvm2 a-m   973.99g       0 
  /dev/xvde  VolGroup lvm2 a--   200.00g       0 
  /dev/xvdf1 VolGroup lvm2 a--  1023.99g 1023.99g

查看pvs参数说明:

# man pvs
.......
NOTES
       The pv_attr bits are:

       1  (d)uplicate, (a)llocatable, (u)sed

       2  e(x)ported

       3  (m)issing

可以确认/dev/xvda2和/dev/xvda3两个物理卷状态为missing(a-m).

问题解决:

用vgextend - -restoremissing命令成功恢复pv到正常状态:

man vgextend有如下说明:

# man vgextend
.......
DESCRIPTION
       vgextend adds one or more PVs to a VG. This increases the space available for LVs in the VG.

       Also,  PVs  that have gone missing and then returned, e.g. due to a transient device failure, can be added back to the VG without re-initializing them
       (see --restoremissing).
.......
- -restoremissing
Add a PV back into a VG after the PV was missing and then returned, e.g. due to a transient failure. The PV is not reinitialized
# 在PV丢失后再将PV添加到VG中,然后返回,例如由于瞬时故障。没有重新初始化PV。

命令格式:

# vgextend VG PV

执行命令:

# vgextend --restoremissing VolGroup /dev/xvda2
# vgextend --restoremissing VolGroup /dev/xvda2

执行完成以后查看pv状态,已经恢复正常:

[root@VM001 ~]# pvs
  PV         VG       Fmt  Attr PSize    PFree   
  /dev/xvda2 VolGroup lvm2 a--    49.51g       0 
  /dev/xvda3 VolGroup lvm2 a--   973.99g       0 
  /dev/xvde  VolGroup lvm2 a--   200.00g       0 
  /dev/xvdf1 VolGroup lvm2 a--  1023.99g 1023.99g

再查看lv状态,也恢复正常,能够正常挂载并进行扩容:

[root@VM001 ~]# vgdisplay -v
    Using volume group(s) on command line.
    Finding all volume groups.
  --- Volume group ---
  VG Name               VolGroup
  System ID 
............
  --- Logical volume ---
  LV Path                /dev/VolGroup/lv_home
  LV Name                lv_home
  VG Name                VolGroup
  LV UUID                4rdajn-62q9-RX5T-QorD-vLsh-hMEv-zxT1Qj
  LV Write Access        read/write
  LV Creation host, time localhost.localdomain, 2016-03-17 09:19:33 +0800
  LV Status              available
  # open                 1
  LV Size                2.15 TiB
  Current LE             562815
  Segments               4
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:2        

手动执行挂载,此时lv_home逻辑卷挂载恢复正常:

[root@VM001 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
                       20G  7.3G   12G  40% /
tmpfs                 3.9G  224K  3.9G   1% /dev/shm
/dev/xvda1            477M  124M  328M  28% /boot
/dev/mapper/VolGroup-lv_home
                      2.2T  1.2T  876G  58% /home

网上看到的其他类似问题:

案例1:
执行lvextend -l +100%FREE /dev/VolGroup00/LogVol00的时候,出现了如下错误:

Device still marked missing because of allocated data on it, remove volumes and consider bgreduce --removemissing.

在查阅许多资料之后,发现可以使用restoremissing 参数来恢复

[root@Caron ~]# vgextend --restoremissing VolGroup00 /dev/sdh1

再次执行lvextend命令的时候,提示成功。重新resize一下lvm大小:

[root@Caron ~]# resize2fs /dev/VolGroup00/LogVol00
[root@Caron ~]# df -h | more
Filesystem                       Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00  936G  612G  283G  69% /
devtmpfs                         126G     0  126G   0% /dev
tmpfs                            126G     0  126G   0% /dev/shm
tmpfs                            126G  4.1G  122G   4% /run
tmpfs                            126G     0  126G   0% /sys/fs/cgroup
/dev/sda1                        283M  100M  164M  38% /boot
tmpfs                            126G  4.1G  122G   4% /run/netns
tmpfs                             26G     0   26G   0% /run/user/0

案例2:

1.PV丢失的故障处理,手工创建逻辑卷失败,报错显示有PV丢失:

[root@node29archive]# lvcreate -L 1G -n testlv vgcloud
Cannotchange VG vgcloud while PVs are missing.
Considervgreduce --removemissing.

2.进一步查看:

[root@node29archive]# pvs
  PV                   VG      Fmt Attr PSize PFree 
  /dev/mapper/mpathbp1 vgcloud lvm2 a--  1.82t 89.40g
  /dev/mapper/mpathbp2 vgcloud lvm2 a--  1.82t 91.14g
  /dev/mapper/mpathbp3 vgcloud lvm2 a--  1.82t 30.14g
  /dev/mapper/mpathbp4 vgcloud lvm2 a--  1.64t 30.00g
  /dev/mapper/mpathcp1 vgcloud lvm2 a-m  1.82t 84.14g
  /dev/mapper/mpathcp2 vgcloud lvm2 a-m  1.82t 117.14g
  /dev/mapper/mpathcp3 vgcloud lvm2 a-m  1.82t  1.33t
  /dev/mapper/mpathcp4 vgcloud lvm2 a--  1.81t  1.81t
[root@node29archive]# vgs
  VG     #PV #LV #SN Attr   VSize  VFree
  vgcloud  8 126   0 wz-pn- 14.36t 3.58t

其中发现有三个逻辑卷处于missing的状态

3.经过查找资料得知可以使用vgextend –restoremissing操作解决

[root@node26tmp]# vgextend --restoremissing vgcloud /dev/mapper/mpathcp1

  Volume group "vgcloud" successfullyextended

[root@node26tmp]# vgextend --restoremissing vgcloud /dev/mapper/mpathcp2

  Volume group "vgcloud" successfullyextended

[root@node26tmp]# vgextend --restoremissing vgcloud /dev/mapper/mpathcp3

  Volume group "vgcloud" successfullyextended

4.再看已经没有missing状态标示了,手工创建逻辑卷

[root@node29archive]# lvcreate -L 1G -n testlv vgcloud

  Logical volume "testlv" created

猜你喜欢

转载自blog.csdn.net/networken/article/details/81325066