环境：虚拟机
[root@rac1 ~]# lsb_release -a
LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch
rinting-4.0-amd64rinting-4.0-noarch
Distributor ID: CentOS
Description: CentOS release 6.10 (Final)
Release: 6.10
Codename: Final
[root@rac1 ~]# uname -r
2.6.32-754.el6.x86_64
SQL*Plus: Release 11.2.0.4.0 Production on Fri Feb 22 09:06:15 2019
在安装过程中，grid，asm，db软件安装都还比较顺利，但是在dbca的过程中，13%
[root@rac1 trace]# tail -n 200 alert_bol1.log
ALTER DATABASE DEFAULT TABLESPACE "USERS"
Completed: ALTER DATABASE DEFAULT TABLESPACE "USERS"
Fri Feb 22 12:42:13 2019
Thread 1 advanced to log sequence 2 (LGWR switch)
Current log# 2 seq# 2 mem# 0: +DATA/bol/onlinelog/group_2.267.1000903313
Current log# 2 seq# 2 mem# 1: +FRA/bol/onlinelog/group_2.261.1000903313
Fri Feb 22 12:42:31 2019
Thread 1 cannot allocate new log, sequence 3
Checkpoint not complete
Current log# 2 seq# 2 mem# 0: +DATA/bol/onlinelog/group_2.267.1000903313
Current log# 2 seq# 2 mem# 1: +FRA/bol/onlinelog/group_2.261.1000903313
Thread 1 advanced to log sequence 3 (LGWR switch)
Current log# 1 seq# 3 mem# 0: +DATA/bol/onlinelog/group_1.266.1000903311
Current log# 1 seq# 3 mem# 1: +FRA/bol/onlinelog/group_1.260.1000903311
Fri Feb 22 12:43:04 2019
Thread 1 cannot allocate new log, sequence 4
Checkpoint not complete
Current log# 1 seq# 3 mem# 0: +DATA/bol/onlinelog/group_1.266.1000903311
Current log# 1 seq# 3 mem# 1: +FRA/bol/onlinelog/group_1.260.1000903311
Thread 1 advanced to log sequence 4 (LGWR switch)
Current log# 2 seq# 4 mem# 0: +DATA/bol/onlinelog/group_2.267.1000903313
Current log# 2 seq# 4 mem# 1: +FRA/bol/onlinelog/group_2.261.1000903313
Fri Feb 22 12:43:15 2019
WARNING: Write Failed. group:3 disk:0 AU:315 offset:210432 size:1024
WARNING: Write Failed. group:2 disk:1 AU:1026 offset:210432 size:1024
Errors in file /u01/app/oracle/diag/rdbms/bol/bol1/trace/bol1_lgwr_13834.trc:
ORA-15080: synchronous I/O operation to a disk failed
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 5: Input/output error
Additional information: -1
Additional information: 1024
WARNING: failed to write mirror side 1 of virtual extent 12 logical extent 0 of file 267 in group 2 on disk 1 allocation unit 1026
Errors in file /u01/app/oracle/diag/rdbms/bol/bol1/trace/bol1_lgwr_13834.trc:
ORA-15080: synchronous I/O operation to a disk failed
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 5: Input/output error
Additional information: -1
Additional information: 1024
WARNING: failed to write mirror side 1 of virtual extent 12 logical extent 0 of file 261 in group 3 on disk 0 allocation unit 315
Errors in file /u01/app/oracle/diag/rdbms/bol/bol1/trace/bol1_lgwr_13834.trc:
ORA-00345: redo log write error block 24987 count 2
ORA-00312: online log 2 thread 1: '+DATA/bol/onlinelog/group_2.267.1000903313'
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-00345: redo log write error block 24987 count 2
ORA-00312: online log 2 thread 1: '+FRA/bol/onlinelog/group_2.261.1000903313'
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
Errors in file /u01/app/oracle/diag/rdbms/bol/bol1/trace/bol1_lgwr_13834.trc:
ORA-00340: IO error processing online log 2 of thread 1
ORA-00345: redo log write error block 24987 count 2
ORA-00312: online log 2 thread 1: '+DATA/bol/onlinelog/group_2.267.1000903313'
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-00345: redo log write error block 24987 count 2
ORA-00312: online log 2 thread 1: '+FRA/bol/onlinelog/group_2.261.1000903313'
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
LGWR (ospid: 13834): terminating the instance due to error 340
Fri Feb 22 12:43:16 2019
System state dump requested by (instance=1, osid=13834 (LGWR)), summary=[abnormal instance termination].
System State dumped to trace file /u01/app/oracle/diag/rdbms/bol/bol1/trace/bol1_diag_13812_20190222124316.trc
Dumping diagnostic data in directory=[cdmp_20190222124316], requested by (instance=1, osid=13834 (LGWR)), summary=[abnormal instance termination].
Instance terminated by LGWR, pid = 13834
遇到此问题，一开始以为是缺少什么安装包或者共享磁盘的配置问题
在换一套安装依然如此
第一套使用udev，第二套使用asmlib
asm相关状态
[grid@rac2 ~]$ crs_stat -t -v Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------- ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1 ora.FRA.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1 ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE rac1 ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE rac1 ora.OCR.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1 ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE rac1 ora.cvu ora.cvu.type 0/5 0/0 ONLINE ONLINE rac1 ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE ora....network ora....rk.type 0/5 0/ ONLINE ONLINE rac1 ora.oc4j ora.oc4j.type 0/1 0/2 ONLINE ONLINE rac1 ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE rac1 ora....SM1.asm application 0/5 0/0 ONLINE ONLINE rac1 ora....C1.lsnr application 0/5 0/0 ONLINE ONLINE rac1 ora.rac1.gsd application 0/5 0/0 OFFLINE OFFLINE ora.rac1.ons application 0/3 0/0 ONLINE ONLINE rac1 ora.rac1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac1 ora....SM2.asm application 0/5 0/0 ONLINE ONLINE rac2 ora....C2.lsnr application 0/5 0/0 ONLINE ONLINE rac2 ora.rac2.gsd application 0/5 0/0 OFFLINE OFFLINE ora.rac2.ons application 0/3 0/0 ONLINE ONLINE rac2 ora.rac2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac2 ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE rac1
[grid@rac1 .ssh]$ olsnodes -n rac1 1 rac2 2 [grid@rac1 .ssh]$ olsnodes -n -i -s -t rac1 1 rac1vip Active Unpinned rac2 2 rac2vip Active Unpinned [grid@rac1 .ssh]$ srvctl status asm -a ASM is running on rac2,rac1 ASM is enabled.
[grid@rac2 ~]$ crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 4c22914de7394f7abfd6c9635cb1c1fd (ORCL:OCR_VOTE1) [OCR] 2. ONLINE 0f6aab9aaa744faebfc22d0622804518 (ORCL:OCR_VOTE2) [OCR] 3. ONLINE 1d1fd2ecffcf4f76bf0ea356730a9c52 (ORCL:OCR_VOTE3) [OCR] Located 3 voting disk(s).
[grid@rac1 trace]$ ps -ef|grep lsnr|grep -v 'grep'|grep -v 'ocfs'|awk '{print$9}' LISTENER LISTENER_SCAN1
[grid@rac1 ~]$ asmcmd ASMCMD> lsdg State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name MOUNTED EXTERN N 512 4096 1048576 163840 161875 0 161875 0 N DATA/ MOUNTED EXTERN N 512 4096 1048576 81920 81692 0 81692 0 N FRA/ MOUNTED EXTERN N 512 4096 1048576 6144 5744 0 5744 0 Y OCR/
在查找过程中，找到相似的文档和案例
https://community.oracle.com/thread/2477850?start=0&tstart=0
https://www.itcodemonkey.com/article/8825.html
听说这次用的存储是自定义？ 第一次使用的。。。目前还没有搞定此问题