OCR和votedisk都放在了DATA盘中，现需要创建DG_OCR磁盘组和DG_VOTE磁盘组分别放置OCR和votedisk信息
--先查看磁盘组版本
SQL> set lines 300
SQL> col name for a15
SQL> col DISK_REPAIR_TIME for a15
SQL> col COMPATIBILITY for a15
col DATABASE_COMPATIBILITY for a15
select a.group_number,
a.name,
a.type,
a.state,
a.total_mb,
a.free_mb,
a.COMPATIBILITY,
a.DATABASE_COMPATIBILITY,
a.OFFLINE_DISKS,
a.VOTING_FILES,
b.value as disk_repair_time
SQL> SQL> 2 3 4 5 6 7 8 9 10 11 12 from v$asm_diskgroup a, v$asm_attribute b
where a.group_number = b.group_number
and b.name = 'disk_repair_time'; 13 14
GROUP_NUMBER NAME TYPE STATE TOTAL_MB FREE_MB COMPATIBILITY DATABASE_COMPAT OFFLINE_DISKS V DISK_REPAIR_TIM
------------ --------------- -------- ---------- ---------- ---------- --------------- --------------- ------------- - ---------------
1 DATA EXTERN MOUNTED 409600 409190 11.2.0.0.0 10.1.0.0.0 0 Y 3.6h
--先设置为11.2.0.4
alter diskgroup DATA set attribute 'compatible.asm'='11.2.0.4.0';
alter diskgroup DATA set attribute 'compatible.rdbms'='11.2.0.4.0';
GROUP_NUMBER NAME TYPE STATE TOTAL_MB FREE_MB COMPATIBILITY DATABASE_COMPAT OFFLINE_DISKS V DISK_REPAIR_TIM
------------ --------------- -------- ---------- ---------- ---------- --------------- --------------- ------------- - ---------------
1 DATA EXTERN MOUNTED 409600 409190 11.2.0.4.0 11.2.0.4.0 0 Y 3.6h
--直接创建磁盘组
create diskgroup DG_VOTE normal redundancy DISK '/dev/rhdisk11','/dev/rhdisk12','/dev/rhdisk13' ATTRIBUTE 'compatible.asm'='11.2.0.4.0','compatible.rdbms'='11.2.0.4.0';
create diskgroup DG_OCR normal redundancy DISK '/dev/rhdisk14','/dev/rhdisk15' ATTRIBUTE 'compatible.asm'='11.2.0.4.0','compatible.rdbms'='11.2.0.4.0';
GROUP_NUMBER NAME TYPE STATE TOTAL_MB FREE_MB COMPATIBILITY DATABASE_COMPAT OFFLINE_DISKS V DISK_REPAIR_TIM
------------ --------------- -------- ---------- ---------- ---------- --------------- --------------- ------------- - ---------------
1 DATA EXTERN MOUNTED 409600 409190 11.2.0.4.0 11.2.0.4.0 0 Y 3.6h
2 DG_VOTE NORMAL MOUNTED 30720 30561 11.2.0.4.0 11.2.0.4.0 0 N 3.6h
3 DG_OCR NORMAL MOUNTED 20480 20374 11.2.0.4.0 11.2.0.4.0 0 N 3.6h
#备份ocr
ntracd01[/]#/grid/app/11.2.0/grid/bin/ocrconfig -showbackup
PROT-24: Auto backups for the Oracle Cluster Registry are not available
PROT-25: Manual backups for the Oracle Cluster Registry are not available
ntracd01[/]#/grid/app/11.2.0/grid/bin/ocrconfig -manualbackup
ntracd01 2018/09/25 15:53:23 /grid/app/11.2.0/grid/cdata/ntracd/backup_20180925_155323.ocr
ntracd01[/]#/grid/app/11.2.0/grid/bin/ocrconfig -showbackup
PROT-24: Auto backups for the Oracle Cluster Registry are not available
ntracd01 2018/09/25 15:53:23 /grid/app/11.2.0/grid/cdata/ntracd/backup_20180925_155323.ocr
#votedisk
#votedisk在+DATA中
ntracd01[/]#/grid/app/11.2.0/grid/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 389a7614125d4f93bf2802c6c61aed58 (/dev/rhdisk3) [DATA]
Located 1 voting disk(s).
#直接replace votedisk到+DG_VOTE
ntracd01[/]#/grid/app/11.2.0/grid/bin/crsctl replace votedisk +DG_VOTE
Successful addition of voting disk 9dfe49f635724f6dbf6ad995e5c52ea9.
Successful addition of voting disk 2bcf065a1e0d4fcdbf9170434c90e42c.
Successful addition of voting disk a19d1626d8494f96bf982013146631e8.
Successful deletion of voting disk 389a7614125d4f93bf2802c6c61aed58.
Successfully replaced voting disk group with +DG_VOTE.
CRS-4266: Voting file(s) successfully replaced
ntracd01[/]#/grid/app/11.2.0/grid/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 9dfe49f635724f6dbf6ad995e5c52ea9 (/dev/rhdisk11) [DG_VOTE]
2. ONLINE 2bcf065a1e0d4fcdbf9170434c90e42c (/dev/rhdisk12) [DG_VOTE]
3. ONLINE a19d1626d8494f96bf982013146631e8 (/dev/rhdisk13) [DG_VOTE]
Located 3 voting disk(s).
#ocr
#ocr在+DATA中
ntracd01[/]#/grid/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2696
Available space (kbytes) : 259424
ID : 960988113
Device/File Name : +data
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
ntracd01[/]#/grid/app/11.2.0/grid/bin/ocrconfig -add +DG_OCR
PROT-30: The Oracle Cluster Registry location to be added is not usable
PROC-50: The Oracle Cluster Registry location to be added is inaccessible on nodes ntracd02.
#报错了
#查看磁盘状态
GROUP_NUMBER ASM_GROUP_ TYPE DISK_NUMBER DISK_NAME PATH OS_MB FREE_MB FAILGROUP FAILGROUP_ MODE_STATU MOUNT_STAT STATE VOTIN
------------ ---------- -------- ----------- -------------------- -------------------- ---------- ---------- --------------- ---------- ---------- ---------- ---------- -----
1 DATA EXTERN 0 DATA_0000 /dev/rhdisk3 51200 51153 DATA_0000 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 1 DATA_0001 /dev/rhdisk4 51200 51152 DATA_0001 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 2 DATA_0002 /dev/rhdisk5 51200 51151 DATA_0002 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 3 DATA_0003 /dev/rhdisk6 51200 51152 DATA_0003 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 4 DATA_0004 /dev/rhdisk7 51200 51152 DATA_0004 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 5 DATA_0005 /dev/rhdisk8 51200 51154 DATA_0005 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 6 DATA_0006 /dev/rhdisk9 51200 51154 DATA_0006 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 7 DATA_0007 /dev/rhdisk10 51200 51154 DATA_0007 REGULAR ONLINE CACHED NORMAL N
0 DG_OCR 1 /dev/rhdisk11 10240 0 REGULAR ONLINE CLOSED NORMAL Y
0 DG_OCR 2 /dev/rhdisk12 10240 0 REGULAR ONLINE CLOSED NORMAL Y
0 DG_OCR 3 /dev/rhdisk13 10240 0 REGULAR ONLINE CLOSED NORMAL Y
0 DG_OCR 4 /dev/rhdisk14 10240 0 REGULAR ONLINE CLOSED NORMAL N
0 DG_OCR 5 /dev/rhdisk15 10240 0 REGULAR ONLINE CLOSED NORMAL N
0 DG_VOTE 1 /dev/rhdisk11 10240 0 REGULAR ONLINE CLOSED NORMAL Y
0 DG_VOTE 2 /dev/rhdisk12 10240 0 REGULAR ONLINE CLOSED NORMAL Y
0 DG_VOTE 3 /dev/rhdisk13 10240 0 REGULAR ONLINE CLOSED NORMAL Y
0 DG_VOTE 4 /dev/rhdisk14 10240 0 REGULAR ONLINE CLOSED NORMAL N
0 DG_VOTE 5 /dev/rhdisk15 10240 0 REGULAR ONLINE CLOSED NORMAL N
#2节点asm识别到的磁盘状态是不对的
dd if=/dev/zero of=/dev/rhdisk11 bs=1K count=1
dd if=/dev/zero of=/dev/rhdisk12 bs=1K count=1
dd if=/dev/zero of=/dev/rhdisk13 bs=1K count=1
dd if=/dev/zero of=/dev/rhdisk14 bs=1K count=1
dd if=/dev/zero of=/dev/rhdisk15 bs=1K count=1
#格式化磁盘
SQL> select D.GROUP_NUMBER,G.name AS ASM_GROUP_NAME,G.type,d.DISK_NUMBER, D.NAME AS DISK_NAME,D.path,D.os_mb,d.free_mb,D.FAILGROUP,D.FAILGROUP_TYPE,D.MODE_STATUS,D.mount_status,d.STATE,VOTING_FILE from V$ASM_DISKGROUP G,v$asm_disk D WHERE G.GROUP_NUMBER=d.GROUP_NUMBER order by ASM_GROUP_NAME,d.FAILGROUP,d.DISK_NUMBER;
GROUP_NUMBER ASM_GROUP_ TYPE DISK_NUMBER DISK_NAME PATH OS_MB FREE_MB FAILGROUP FAILGROUP_ MODE_STATU MOUNT_STAT STATE VOTIN
------------ ---------- -------- ----------- -------------------- -------------------- ---------- ---------- --------------- ---------- ---------- ---------- ---------- -----
1 DATA EXTERN 0 DATA_0000 /dev/rhdisk3 51200 51153 DATA_0000 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 1 DATA_0001 /dev/rhdisk4 51200 51152 DATA_0001 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 2 DATA_0002 /dev/rhdisk5 51200 51151 DATA_0002 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 3 DATA_0003 /dev/rhdisk6 51200 51152 DATA_0003 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 4 DATA_0004 /dev/rhdisk7 51200 51152 DATA_0004 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 5 DATA_0005 /dev/rhdisk8 51200 51154 DATA_0005 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 6 DATA_0006 /dev/rhdisk9 51200 51154 DATA_0006 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 7 DATA_0007 /dev/rhdisk10 51200 51154 DATA_0007 REGULAR ONLINE CACHED NORMAL N
0 DG_VOTE 1 /dev/rhdisk11 10240 0 REGULAR ONLINE CLOSED NORMAL Y
0 DG_VOTE 2 /dev/rhdisk12 10240 0 REGULAR ONLINE CLOSED NORMAL Y
0 DG_VOTE 3 /dev/rhdisk13 10240 0 REGULAR ONLINE CLOSED NORMAL Y
0 DG_VOTE 4 /dev/rhdisk14 10240 0 REGULAR ONLINE CLOSED NORMAL N
0 DG_VOTE 5 /dev/rhdisk15 10240 0 REGULAR ONLINE CLOSED NORMAL N
13 rows selected.
#asm1磁盘信息是对的，asm2不对
SQL>
SQL> alter diskgroup DG_VOTE offline disk '/dev/rhdisk14';
alter diskgroup DG_VOTE offline disk '/dev/rhdisk14'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15001: diskgroup "DG_VOTE" does not exist or is not mounted
#解决不了问题
SQL> alter diskgroup DG_VOTE dismount;
Diskgroup altered.
SQL> drop diskgroup DG_VOTE ;
drop diskgroup DG_VOTE
*
ERROR at line 1:
ORA-15039: diskgroup not dropped
ORA-15001: diskgroup "DG_VOTE" does not exist or is not mounted
dd if=/dev/zero of=/dev/rhdisk11 bs=1K count=1
dd if=/dev/zero of=/dev/rhdisk12 bs=1K count=1
dd if=/dev/zero of=/dev/rhdisk13 bs=1K count=1
dd if=/dev/zero of=/dev/rhdisk14 bs=1K count=1
dd if=/dev/zero of=/dev/rhdisk15 bs=1K count=1
SQL> select D.GROUP_NUMBER,G.name AS ASM_GROUP_NAME,G.type,d.DISK_NUMBER, D.NAME AS DISK_NAME,D.path,D.os_mb,d.free_mb,D.FAILGROUP,D.FAILGROUP_TYPE,D.MODE_STATUS,D.mount_status,d.STATE,VOTING_FILE from V$ASM_DISKGROUP G,v$asm_disk D WHERE G.GROUP_NUMBER=d.GROUP_NUMBER order by ASM_GROUP_NAME,d.FAILGROUP,d.DISK_NUMBER;
GROUP_NUMBER ASM_GROUP_ TYPE DISK_NUMBER DISK_NAME PATH OS_MB FREE_MB FAILGROUP FAILGROUP_ MODE_STATU MOUNT_STAT STATE VOTIN
------------ ---------- -------- ----------- -------------------- -------------------- ---------- ---------- --------------- ---------- ---------- ---------- ---------- -----
1 DATA EXTERN 0 DATA_0000 /dev/rhdisk3 51200 51121 DATA_0000 REGULAR ONLINE CACHED NORMAL Y
1 DATA EXTERN 1 DATA_0001 /dev/rhdisk4 51200 51152 DATA_0001 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 2 DATA_0002 /dev/rhdisk5 51200 51151 DATA_0002 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 3 DATA_0003 /dev/rhdisk6 51200 51152 DATA_0003 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 4 DATA_0004 /dev/rhdisk7 51200 51152 DATA_0004 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 5 DATA_0005 /dev/rhdisk8 51200 51154 DATA_0005 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 6 DATA_0006 /dev/rhdisk9 51200 51154 DATA_0006 REGULAR ONLINE CACHED NORMAL N
1 DATA EXTERN 7 DATA_0007 /dev/rhdisk10 51200 51154 DATA_0007 REGULAR ONLINE CACHED NORMAL N
8 rows selected.
#重新再弄一次，这次一定要2个节点的asm信息都查
create diskgroup DG_VOTE normal redundancy DISK '/dev/rhdisk11','/dev/rhdisk12','/dev/rhdisk13' ATTRIBUTE 'compatible.asm'='11.2.0.4.0','compatible.rdbms'='11.2.0.4.0';
2 DG_VOTE NORMAL 0 DG_VOTE_0000 /dev/rhdisk11 10240 10187 DG_VOTE_0000 REGULAR ONLINE CACHED NORMAL N
2 DG_VOTE NORMAL 1 DG_VOTE_0001 /dev/rhdisk12 10240 10187 DG_VOTE_0001 REGULAR ONLINE CACHED NORMAL N
2 DG_VOTE NORMAL 2 DG_VOTE_0002 /dev/rhdisk13 10240 10187 DG_VOTE_0002 REGULAR ONLINE CACHED NORMAL N
0 DG_VOTE 1 /dev/rhdisk11 10240 0 REGULAR ONLINE CLOSED NORMAL N
0 DG_VOTE 2 /dev/rhdisk12 10240 0 REGULAR ONLINE CLOSED NORMAL N
0 DG_VOTE 3 /dev/rhdisk13 10240 0 REGULAR ONLINE CLOSED NORMAL N
0 DG_VOTE 4 /dev/rhdisk14 10240 0 REGULAR ONLINE CLOSED NORMAL N
0 DG_VOTE 5 /dev/rhdisk15 10240 0 REGULAR ONLINE CLOSED NORMAL N
2节点信息不正常。
再dd一次再加
create diskgroup DG_OCR normal redundancy DISK '/dev/rhdisk14','/dev/rhdisk15' ATTRIBUTE 'compatible.asm'='11.2.0.4.0','compatible.rdbms'='11.2.0.4.0';
NOTE: Instance updated compatible.asm to 11.2.0.4.0 for grp 3
SUCCESS: diskgroup DG_OCR was mounted
SUCCESS: create diskgroup DG_OCR normal redundancy DISK '/dev/rhdisk14','/dev/rhdisk15' ATTRIBUTE 'compatible.asm'='11.2.0.4.0','compatible.rdbms'='11.2.0.4.0'
Tue Sep 25 17:30:18 2018
NOTE: diskgroup resource ora.DG_OCR.dg is online
ERROR: failed to update diskgroup resource ora.DG_OCR.dg
Tue Sep 25 17:31:45 2018
NOTE: enlarging ACD to 2 threads for group 3/0xdff6c653 (DG_OCR)
SUCCESS: ACD enlarged for group 3/0xdff6c653 (DG_OCR)
ntracd01[/dev]#/grid/app/11.2.0/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE ntracd01
ONLINE ONLINE ntracd02
ora.DG_OCR.dg
ONLINE ONLINE ntracd01
OFFLINE OFFLINE ntracd02
ora.DG_VOTE.dg
ONLINE ONLINE ntracd01
ONLINE ONLINE ntracd02
ora.asm
ONLINE ONLINE ntracd01 Started
ONLINE ONLINE ntracd02 Started
ora.gsd
OFFLINE OFFLINE ntracd01
OFFLINE OFFLINE ntracd02
ora.net1.network
ONLINE ONLINE ntracd01
ONLINE ONLINE ntracd02
ora.ons
ONLINE ONLINE ntracd01
ONLINE ONLINE ntracd02
ora.registry.acfs
ONLINE ONLINE ntracd01
ONLINE ONLINE ntracd02
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE ntracd01
ora.cvu
1 ONLINE ONLINE ntracd01
ora.ntracd01.vip
1 ONLINE ONLINE ntracd01
ora.ntracd02.vip
1 ONLINE ONLINE ntracd02
ora.oc4j
1 ONLINE ONLINE ntracd01
ora.scan1.vip
1 ONLINE ONLINE ntracd01
#添加DG_OCR资源
srvctl start diskgroup -g DG_OCR
ntracd01[/dev]#/grid/app/11.2.0/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE ntracd01
ONLINE ONLINE ntracd02
ora.DG_OCR.dg
ONLINE ONLINE ntracd01
ONLINE ONLINE ntracd02
ora.DG_VOTE.dg
ONLINE ONLINE ntracd01
ONLINE ONLINE ntracd02
ora.asm
ONLINE ONLINE ntracd01 Started
ONLINE ONLINE ntracd02 Started
ora.gsd
OFFLINE OFFLINE ntracd01
OFFLINE OFFLINE ntracd02
ora.net1.network
ONLINE ONLINE ntracd01
ONLINE ONLINE ntracd02
ora.ons
ONLINE ONLINE ntracd01
ONLINE ONLINE ntracd02
ora.registry.acfs
ONLINE ONLINE ntracd01
ONLINE ONLINE ntracd02
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE ntracd01
ora.cvu
1 ONLINE ONLINE ntracd01
ora.ntracd01.vip
1 ONLINE ONLINE ntracd01
ora.ntracd02.vip
1 ONLINE ONLINE ntracd02
ora.oc4j
1 ONLINE ONLINE ntracd01
ora.scan1.vip
1 ONLINE ONLINE ntracd01
资源总算正常了
/grid/app/11.2.0/grid/bin/crsctl replace votedisk +DG_VOTE
/grid/app/11.2.0/grid/bin/ocrcheck
ntracd01[/dev]#/grid/app/11.2.0/grid/bin/ocrconfig -add +DG_OCR
ntracd01[/dev]#/grid/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2688
Available space (kbytes) : 259432
ID : 960988113
Device/File Name : +data
Device/File integrity check succeeded
Device/File Name : +DG_OCR
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
ntracd01[/dev]#/grid/app/11.2.0/grid/bin/ocrconfig -delete +data
ntracd01[/dev]#/grid/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2688
Available space (kbytes) : 259432
ID : 960988113
Device/File Name : +DG_OCR
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
ocr和votedisk迁移完成