多么痛的一次领悟ORA-21561: OID generation failed

其他 2018-10-30 14:53:50 阅读次数: 0

环境为 2个节点的11.2.0.4的RAC环境

重启异常

数据库无法启动，出现报错如下

在日志

 /opt/app/grid/11.2.0/log/racoa02/agent/ohasd/oraagent_grid/oraagent_grid.log

2018-10-29 23:26:01.841: [ USRTHRD][4040279808]{0:0:2} checkCrsStat 2 CLSCRS_STAT ret: 184
2018-10-29 23:26:01.841: [ USRTHRD][4040279808]{0:0:2} CrsCmd::ClscrsCmdData::destroy
2018-10-29 23:26:01.841: [ USRTHRD][4040279808]{0:0:2} CrsCmd::destroy
2018-10-29 23:26:01.841: [ora.evmd][4040279808]{0:0:2} [start] (:CLSN00107:) clsn_agent::start }
2018-10-29 23:26:01.841: [    AGFW][4040279808]{0:0:2} Command: start for resource: ora.evmd 1 1 completed with status: SUCCESS
2018-10-29 23:26:01.841: [    AGFW][4067276544]{0:0:2} Agent sending reply for: RESOURCE_START[ora.evmd 1 1] ID 4098:438
2018-10-29 23:26:01.841: [ora.evmd][4040279808]{0:0:2} [check] clsdmc_respget return: status=0, ecode=0
2018-10-29 23:26:01.841: [ USRTHRD][4040279808]{0:0:2} Thread:[DaemonCheck:evmd]thread constructor
2018-10-29 23:26:01.842: [ USRTHRD][4040279808]{0:0:2} Thread:[DaemonCheck:evmd]start {
2018-10-29 23:26:01.842: [ USRTHRD][4040279808]{0:0:2} Thread:[DaemonCheck:evmd]start }
2018-10-29 23:26:01.842: [ora.evmd][4040279808]{0:0:2} [check] DaemonAgent::check returned 0
2018-10-29 23:26:01.842: [ora.evmd][4040279808]{0:0:2} [check] Deep check returned 0
2018-10-29 23:26:01.842: [    AGFW][4067276544]{0:0:2} ora.evmd 1 1 state changed from: STARTING to: PARTIAL
2018-10-29 23:26:01.843: [    AGFW][4067276544]{0:0:2} Started implicit monitor for [ora.evmd 1 1] interval=30000 delay=30000
2018-10-29 23:26:01.843: [    AGFW][4067276544]{0:0:2} Agent sending last reply for: RESOURCE_START[ora.evmd 1 1] ID 4098:438
2018-10-29 23:26:01.843: [CLSFRAME][4088489760] TM [MultiThread] is changing desired thread # to 5. Current # is 4
2018-10-29 23:26:03.504: [ora.mdnsd][4040279808]{0:0:2} [check] clsdmc_respget return: status=0, ecode=0
2018-10-29 23:26:05.515: [ora.gpnpd][4069377792]{0:0:2} [check] clsdmc_respget return: status=0, ecode=0
2018-10-29 23:26:06.526: [ora.gipcd][2613040896]{0:0:2} [check] clsdmc_respget return: status=0, ecode=0
2018-10-29 23:26:06.750: [ora.asm][4063266560]{0:0:2} [start] InstAgent::check prev clsagfw_res_status 2 current clsagfw_res_status 2
2018-10-29 23:26:06.750: [ora.asm][4063266560]{0:0:2} [start] InstAgent::stop  shutdown mode: 4
2018-10-29 23:26:06.750: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::getConnection 160 ERROR sid +ASM1 pool status 1
2018-10-29 23:26:06.750: [ora.asm][4063266560]{0:0:2} [start] InstAgent::stop pool pConnxn 00000000
2018-10-29 23:26:06.750: [ora.asm][4063266560]{0:0:2} [start] clsnInstConnection::makeConnectStr UsrOraEnv  m_oracleHome /opt/app/grid/11.2.0 Crshome /opt/app/grid/11.2.0
2018-10-29 23:26:06.750: [ora.asm][4063266560]{0:0:2} [start] makeConnectStr = (DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/opt/app/grid/11.2.0/bin/oracle)(ARGV0=oracle+ASM1)(ENVS='ORACLE_HOME=/opt/app/grid/11.2.0,ORACLE_SID=+ASM1')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(CONNECT_DATA=(SID=+ASM1)))
2018-10-29 23:26:06.751: [ora.asm][4063266560]{0:0:2} [start] InstAgent::stop non pool pConnxn 1 d00404c0
2018-10-29 23:26:06.751: [ora.asm][4063266560]{0:0:2} [start] InstConnection::connectInt: server not attached
2018-10-29 23:26:06.762: [ora.asm][4063266560]{0:0:2} [start] ORA-21561: OID generation failed

2018-10-29 23:26:06.762: [ora.asm][4063266560]{0:0:2} [start] InstConnection::connectInt (2) Exception OCIException
2018-10-29 23:26:06.762: [ora.asm][4063266560]{0:0:2} [start] InstConnection:connect:excp OCIException OCI error 21561
2018-10-29 23:26:06.762: [ora.asm][4063266560]{0:0:2} [start] InstAgent::stop: connect1 errcode 21561
2018-10-29 23:26:06.762: [ora.asm][4063266560]{0:0:2} [start] InstAgent::stop: connect2 oracleHome /opt/app/grid/11.2.0 oracleSid +ASM1
2018-10-29 23:26:06.763: [ora.asm][4063266560]{0:0:2} [start] InstConnection::connectInt: server not attached
2018-10-29 23:26:06.772: [ora.asm][4063266560]{0:0:2} [start] ORA-21561: OID generation failed

2018-10-29 23:26:06.772: [ora.asm][4063266560]{0:0:2} [start] InstConnection::connectInt (2) Exception OCIException
2018-10-29 23:26:06.772: [ora.asm][4063266560]{0:0:2} [start] InstConnection:connect:excp OCIException OCI error 21561
2018-10-29 23:26:06.772: [ora.asm][4063266560]{0:0:2} [start] InstAgent::stop: connect2 errcode 21561
2018-10-29 23:26:06.772: [ora.asm][4063266560]{0:0:2} [start] clsnUtils::error Exception type=2 string=
ORA-21561: OID generation failed

2018-10-29 23:26:06.772: [    AGFW][4063266560]{0:0:2} sending status msg [ORA-21561: OID generation failed
] for start for resource: ora.asm 1 1
2018-10-29 23:26:06.772: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::removeConnection connection count 1
2018-10-29 23:26:06.772: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::removeConnection sid  +ASM1, InstConnection d00404c0
2018-10-29 23:26:06.772: [ USRTHRD][4063266560]{0:0:2} InstConnection::breakCall pConnxn:d00404c0  DetachLock:00b01580 m_pSvcH:d00eb5f0
2018-10-29 23:26:06.772: [ USRTHRD][4063266560]{0:0:2} InstConnection:~InstConnection: this d00404c0
2018-10-29 23:26:06.773: [    AGFW][4067276544]{0:0:2} Agent sending reply for: RESOURCE_START[ora.asm 1 1] ID 4098:444
2018-10-29 23:26:06.774: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::removeConnection delete InstConnection 0x7f1fd00404c0
2018-10-29 23:26:06.774: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::removeConnection freed 1
2018-10-29 23:26:06.774: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::stopConnection
2018-10-29 23:26:06.774: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::removeConnection connection count 0
2018-10-29 23:26:06.774: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::removeConnection freed 0
2018-10-29 23:26:06.774: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::stopConnection sid +ASM1 status  1
2018-10-29 23:26:06.774: [ora.asm][4063266560]{0:0:2} [start] InstAgent::stop db/asm
2018-10-29 23:26:06.774: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::stopConnection
2018-10-29 23:26:06.774: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::removeConnection connection count 0
2018-10-29 23:26:06.774: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::removeConnection freed 0
2018-10-29 23:26:06.774: [ora.asm][4063266560]{0:0:2} [start] ConnectionPool::stopConnection sid +ASM1 status  1
2018-10-29 23:26:06.775: [ora.asm][4063266560]{0:0:2} [start] getResAttrib: attrib name USR_ORA_OPI value false len 5
2018-10-29 23:26:06.775: [ora.asm][4063266560]{0:0:2} [start] InstAgent::stop Container->stop, isShutdown = 0
2018-10-29 23:26:06.775: [ USRTHRD][4063266560]{0:0:2} CssData Destructor groupName:HB+ASM
2018-10-29 23:26:06.776: [ USRTHRD][4063266560]{0:0:2} CssData::destroyMutex &m_cssDataLock:d0036a10
2018-10-29 23:26:06.776: [ USRTHRD][4063266560]{0:0:2} CssData::Destructor
2018-10-29 23:26:06.776: [ USRTHRD][4063266560]{0:0:2} Thread:CSS Data Threadthread destructor
2018-10-29 23:26:06.776: [ USRTHRD][4063266560]{0:0:2} Thread:CSS Data Threadthread destructor exit
2018-10-29 23:26:06.776: [ora.asm][4063266560]{0:0:2} [start] InstAgent::stop prev clsagfw_res_status 2 current clsagfw_res_status 2
2018-10-29 23:26:06.776: [ora.asm][4063266560]{0:0:2} [start] InstAgent::stop: } 
2018-10-29 23:26:06.776: [ora.asm][4063266560]{0:0:2} [start] clean  }
2018-10-29 23:26:06.776: [ora.asm][4063266560]{0:0:2} [start] InstAgent::start exception }
2018-10-29 23:26:06.776: [ora.asm][4063266560]{0:0:2} [start] clsnUtils::error Exception type=2 string=
CRS-5017: The resource action "ora.asm start" encountered the following error: 
ORA-21561: OID generation failed
. For details refer to "(:CLSN00107:)" in "/opt/app/grid/11.2.0/log/racoa01/agent/ohasd/oraagent_grid//oraagent_grid.log".

2018-10-29 23:26:06.776: [    AGFW][4063266560]{0:0:2} sending status msg [CRS-5017: The resource action "ora.asm start" encountered the following error: 
ORA-21561: OID generation failed
. For details refer to "(:CLSN00107:)" in "/opt/app/grid/11.2.0/log/racoa01/agent/ohasd/oraagent_grid//oraagent_grid.log".
] for start for resource: ora.asm 1 1
2018-10-29 23:26:06.776: [ora.asm][4063266560]{0:0:2} [start] (:CLSN00107:) clsn_agent::start }
2018-10-29 23:26:06.776: [    AGFW][4063266560]{0:0:2} Command: start for resource: ora.asm 1 1 completed with status: FAIL
2018-10-29 23:26:06.776: [    AGFW][4067276544]{0:0:2} Agent sending reply for: RESOURCE_START[ora.asm 1 1] ID 4098:444
2018-10-29 23:26:06.776: [ora.asm][4063266560]{0:0:2} [check] ConnectionPool::resetConnection  s_statusOfConnectionMap 00b07d28
2018-10-29 23:26:06.777: [ora.asm][4063266560]{0:0:2} [check] ConnectionPool::resetConnection sid +ASM1 status  2
2018-10-29 23:26:06.777: [ora.asm][4063266560]{0:0:2} [check] Gimh::check OH /opt/app/grid/11.2.0 SID +ASM1
2018-10-29 23:26:06.777: [ora.asm][4063266560]{0:0:2} [check] Gimh::check condition changes to (GIMH_NEXT_NUM) 1(Instance Shutdown) exists
2018-10-29 23:26:06.777: [ USRTHRD][4063266560]{0:0:2} Gimh::destructor gimh_dest_query_ctx rc=0
2018-10-29 23:26:06.777: [ USRTHRD][4063266560]{0:0:2} Gimh::destructor gimh_dest_inst_ctx rc=0
2018-10-29 23:26:06.777: [ora.asm][4063266560]{0:0:2} [check] ConnectionPool::stopConnection
2018-10-29 23:26:06.777: [ora.asm][4063266560]{0:0:2} [check] ConnectionPool::removeConnection connection count 0
2018-10-29 23:26:06.777: [ora.asm][4063266560]{0:0:2} [check] ConnectionPool::removeConnection freed 0
2018-10-29 23:26:06.777: [ora.asm][4063266560]{0:0:2} [check] ConnectionPool::stopConnection sid +ASM1 status  1
2018-10-29 23:26:06.777: [ora.asm][4063266560]{0:0:2} [check] InstAgent::check 1 prev clsagfw_res_status 2 current clsagfw_res_status 2
2018-10-29 23:26:06.777: [ora.asm][4063266560]{0:0:2} [check] InstAgent::checkState dbType 
2018-10-29 23:26:06.777: [ora.asm][4063266560]{0:0:2} [check] InstAgent::checkState wait 5 seconds
2018-10-29 23:26:06.777: [    AGFW][4067276544]{0:0:2} Agent sending reply for: RESOURCE_START[ora.asm 1 1] ID 4098:444
2018-10-29 23:26:11.777: [ora.asm][4063266560]{0:0:2} [check] InstAgent::check prev clsagfw_res_status 2 current clsagfw_res_status 2
2018-10-29 23:26:11.777: [    AGFW][4067276544]{0:0:2} ora.asm 1 1 state changed from: STARTING to: PLANNED_OFFLINE
2018-10-29 23:26:11.777: [    AGFW][4067276544]{0:0:2} ora.asm 1 1 state details has changed from:  to: Instance Shutdown
2018-10-29 23:26:11.777: [    AGFW][4067276544]{0:0:2} Agent sending last reply for: RESOURCE_START[ora.asm 1 1] ID 4098:444
2018-10-29 23:26:11.779: [    AGFW][4067276544]{0:0:2} Agent received the message: RESOURCE_CLEAN[ora.asm 1 1] ID 4100:454
2018-10-29 23:26:11.779: [    AGFW][4067276544]{0:0:2} Preparing CLEAN command for: ora.asm 1 1
2018-10-29 23:26:11.779: [    AGFW][4067276544]{0:0:2} ora.asm 1 1 state changed from: PLANNED_OFFLINE to: CLEANING
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] (:CLSN00106:) clsn_agent::clean {
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] clean {
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop_option stop mode immediate option 1
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop { 
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] getResAttrib: attrib name USR_ORA_OPI value false len 5
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop original reason system do shutdown abort
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::resetConnection  s_statusOfConnectionMap 00b07d28
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::resetConnection sid +ASM1 status  2
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] Gimh::check OH /opt/app/grid/11.2.0 SID +ASM1
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] Gimh::check condition changes to (GIMH_NEXT_NUM) 1(Instance Shutdown) exists
2018-10-29 23:26:11.780: [ USRTHRD][4040279808]{0:0:2} Gimh::destructor gimh_dest_query_ctx rc=0
2018-10-29 23:26:11.780: [ USRTHRD][4040279808]{0:0:2} Gimh::destructor gimh_dest_inst_ctx rc=0
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::stopConnection
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::removeConnection connection count 0
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::removeConnection freed 0
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::stopConnection sid +ASM1 status  1
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::check 1 prev clsagfw_res_status 2 current clsagfw_res_status 2
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::checkState dbType 
2018-10-29 23:26:11.780: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::checkState wait 5 seconds
2018-10-29 23:26:16.780: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::check prev clsagfw_res_status 2 current clsagfw_res_status 2
2018-10-29 23:26:16.780: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop  shutdown mode: 4
2018-10-29 23:26:16.780: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::getConnection 160 ERROR sid +ASM1 pool status 1
2018-10-29 23:26:16.780: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop pool pConnxn 00000000
2018-10-29 23:26:16.780: [ora.asm][4040279808]{0:0:2} [clean] clsnInstConnection::makeConnectStr UsrOraEnv  m_oracleHome /opt/app/grid/11.2.0 Crshome /opt/app/grid/11.2.0
2018-10-29 23:26:16.780: [ora.asm][4040279808]{0:0:2} [clean] makeConnectStr = (DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/opt/app/grid/11.2.0/bin/oracle)(ARGV0=oracle+ASM1)(ENVS='ORACLE_HOME=/opt/app/grid/11.2.0,ORACLE_SID=+ASM1')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(CONNECT_DATA=(SID=+ASM1)))
2018-10-29 23:26:16.781: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop non pool pConnxn 1 a40717e0
2018-10-29 23:26:16.781: [ora.asm][4040279808]{0:0:2} [clean] InstConnection::connectInt: server not attached
2018-10-29 23:26:16.792: [ora.asm][4040279808]{0:0:2} [clean] ORA-21561: OID generation failed

2018-10-29 23:26:16.793: [ora.asm][4040279808]{0:0:2} [clean] InstConnection::connectInt (2) Exception OCIException
2018-10-29 23:26:16.793: [ora.asm][4040279808]{0:0:2} [clean] InstConnection:connect:excp OCIException OCI error 21561
2018-10-29 23:26:16.793: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop: connect1 errcode 21561
2018-10-29 23:26:16.793: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop: connect2 oracleHome /opt/app/grid/11.2.0 oracleSid +ASM1
2018-10-29 23:26:16.793: [ora.asm][4040279808]{0:0:2} [clean] InstConnection::connectInt: server not attached
2018-10-29 23:26:16.803: [ora.asm][4040279808]{0:0:2} [clean] ORA-21561: OID generation failed

2018-10-29 23:26:16.803: [ora.asm][4040279808]{0:0:2} [clean] InstConnection::connectInt (2) Exception OCIException
2018-10-29 23:26:16.803: [ora.asm][4040279808]{0:0:2} [clean] InstConnection:connect:excp OCIException OCI error 21561
2018-10-29 23:26:16.803: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop: connect2 errcode 21561
2018-10-29 23:26:16.803: [ora.asm][4040279808]{0:0:2} [clean] clsnUtils::error Exception type=2 string=
ORA-21561: OID generation failed

2018-10-29 23:26:16.803: [    AGFW][4040279808]{0:0:2} sending status msg [ORA-21561: OID generation failed
] for clean for resource: ora.asm 1 1
2018-10-29 23:26:16.803: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::removeConnection connection count 1
2018-10-29 23:26:16.803: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::removeConnection sid  +ASM1, InstConnection a40717e0
2018-10-29 23:26:16.803: [ USRTHRD][4040279808]{0:0:2} InstConnection::breakCall pConnxn:a40717e0  DetachLock:00b01580 m_pSvcH:a40c0520
2018-10-29 23:26:16.803: [ USRTHRD][4040279808]{0:0:2} InstConnection:~InstConnection: this a40717e0
2018-10-29 23:26:16.803: [    AGFW][4067276544]{0:0:2} Agent sending reply for: RESOURCE_CLEAN[ora.asm 1 1] ID 4100:454
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::removeConnection delete InstConnection 0x7f1fa40717e0
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::removeConnection freed 1
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::stopConnection
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::removeConnection connection count 0
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::removeConnection freed 0
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::stopConnection sid +ASM1 status  1
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop db/asm
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::stopConnection
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::removeConnection connection count 0
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::removeConnection freed 0
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] ConnectionPool::stopConnection sid +ASM1 status  1
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] getResAttrib: attrib name USR_ORA_OPI value false len 5
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop Container->stop, isShutdown = 0
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop calling initCssData
2018-10-29 23:26:16.804: [ USRTHRD][4040279808]{0:0:2} Thread:CSS Data Threadthread constructor
2018-10-29 23:26:16.804: [ora.asm][4040279808]{0:0:2} [clean] CssData Constructor groupName:HB+ASM cssRegistrationType:0 memno:-1
2018-10-29 23:26:16.804: [ USRTHRD][4040279808]{0:0:2} CssData::initMutex &m_cssDataLock:a410f070
2018-10-29 23:26:16.805: [ora.asm][4040279808]{0:0:2} [clean] CssGroup::regis registration of groupName:HB+ASM mbrid:-1 m_grpNum:-1 regType:0 succeeded
2018-10-29 23:26:16.805: [ora.asm][4040279808]{0:0:2} [clean] CssData::Constructor regis succeeded groupName:HB+ASM memno:-1
2018-10-29 23:26:16.805: [ora.asm][4040279808]{0:0:2} [clean] CssData::Constructor m_pCssGroup:a403d750 group HB+ASM
2018-10-29 23:26:16.805: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::init m_groupName:HB+ASM m_pCssData a410dfa0
2018-10-29 23:26:16.805: [ USRTHRD][4040279808]{0:0:2} CssData Destructor groupName:HB+ASM
2018-10-29 23:26:16.806: [ USRTHRD][4040279808]{0:0:2} CssData::destroyMutex &m_cssDataLock:a410f070
2018-10-29 23:26:16.806: [ USRTHRD][4040279808]{0:0:2} CssData::Destructor
2018-10-29 23:26:16.806: [ USRTHRD][4040279808]{0:0:2} Thread:CSS Data Threadthread destructor
2018-10-29 23:26:16.806: [ USRTHRD][4040279808]{0:0:2} Thread:CSS Data Threadthread destructor exit
2018-10-29 23:26:16.806: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop prev clsagfw_res_status 2 current clsagfw_res_status 2
2018-10-29 23:26:16.806: [ora.asm][4040279808]{0:0:2} [clean] InstAgent::stop: } 
2018-10-29 23:26:16.806: [ora.asm][4040279808]{0:0:2} [clean] clean  }
2018-10-29 23:26:16.807: [ora.asm][4040279808]{0:0:2} [clean] (:CLSN00106:) clsn_agent::clean }
2018-10-29 23:26:16.807: [    AGFW][4040279808]{0:0:2} Command: clean for resource: ora.asm 1 1 completed with status: SUCCESS
2018-10-29 23:26:16.807: [    AGFW][4067276544]{0:0:2} Agent sending reply for: RESOURCE_CLEAN[ora.asm 1 1] ID 4100:454
2018-10-29 23:26:16.807: [ora.asm][4040279808]{0:0:2} [check] ConnectionPool::resetConnection  s_statusOfConnectionMap 00b07d28
2018-10-29 23:26:16.807: [ora.asm][4040279808]{0:0:2} [check] ConnectionPool::resetConnection sid +ASM1 status  2
2018-10-29 23:26:16.807: [ora.asm][4040279808]{0:0:2} [check] Gimh::check OH /opt/app/grid/11.2.0 SID +ASM1
2018-10-29 23:26:16.807: [ora.asm][4040279808]{0:0:2} [check] Gimh::check condition changes to (GIMH_NEXT_NUM) 1(Instance Shutdown) exists
2018-10-29 23:26:16.807: [ USRTHRD][4040279808]{0:0:2} Gimh::destructor gimh_dest_query_ctx rc=0
2018-10-29 23:26:16.807: [ USRTHRD][4040279808]{0:0:2} Gimh::destructor gimh_dest_inst_ctx rc=0
2018-10-29 23:26:16.807: [ora.asm][4040279808]{0:0:2} [check] ConnectionPool::stopConnection
2018-10-29 23:26:16.807: [ora.asm][4040279808]{0:0:2} [check] ConnectionPool::removeConnection connection count 0
2018-10-29 23:26:16.807: [ora.asm][4040279808]{0:0:2} [check] ConnectionPool::removeConnection freed 0
2018-10-29 23:26:16.807: [ora.asm][4040279808]{0:0:2} [check] ConnectionPool::stopConnection sid +ASM1 status  1
2018-10-29 23:26:16.807: [ora.asm][4040279808]{0:0:2} [check] InstAgent::check 1 prev clsagfw_res_status 2 current clsagfw_res_status 2
2018-10-29 23:26:16.807: [ora.asm][4040279808]{0:0:2} [check] InstAgent::checkState dbType 
2018-10-29 23:26:16.807: [ora.asm][4040279808]{0:0:2} [check] InstAgent::checkState wait 5 seconds
2018-10-29 23:26:21.807: [ora.asm][4040279808]{0:0:2} [check] InstAgent::check prev clsagfw_res_status 2 current clsagfw_res_status 2
2018-10-29 23:26:21.807: [    AGFW][4067276544]{0:0:2} ora.asm 1 1 state changed from: CLEANING to: PLANNED_OFFLINE
2018-10-29 23:26:21.808: [    AGFW][4067276544]{0:0:2} Agent sending last reply for: RESOURCE_CLEAN[ora.asm 1 1] ID 4100:454
2018-10-29 23:26:21.814: [    AGFW][4067276544]{0:0:2} Agent received the message: RESOURCE_START[ora.asm 1 1] ID 4098:461
2018-10-29 23:26:21.814: [    AGFW][4067276544]{0:0:2} Preparing START command for: ora.asm 1 1
2018-10-29 23:26:21.814: [    AGFW][4067276544]{0:0:2} ora.asm 1 1 state changed from: PLANNED_OFFLINE to: STARTING
2018-10-29 23:26:21.814: [ora.asm][4040279808]{0:0:2} [start] (:CLSN00107:) clsn_agent::start {
2018-10-29 23:26:21.814: [ora.asm][4040279808]{0:0:2} [start] InstAgent::start { 
2018-10-29 23:26:21.814: [ora.asm][4040279808]{0:0:2} [start] ConnectionPool::stopConnection
2018-10-29 23:26:21.814: [ora.asm][4040279808]{0:0:2} [start] ConnectionPool::removeConnection connection count 0
2018-10-29 23:26:21.814: [ora.asm][4040279808]{0:0:2} [start] ConnectionPool::removeConnection freed 0
2018-10-29 23:26:21.814: [ora.asm][4040279808]{0:0:2} [start] ConnectionPool::stopConnection sid +ASM1 status  1
2018-10-29 23:26:21.814: [ USRTHRD][4040279808]{0:0:2} ConnectionPool::~ConnectionPool m_oracleHome:/opt/app/grid/11.2.0, m_oracleSid:+ASM1, m_usrOraEnv:
2018-10-29 23:26:21.814: [ora.asm][4040279808]{0:0:2} [start] AsmAgent::refresh
2018-10-29 23:26:21.814: [ora.asm][4040279808]{0:0:2} [start] AsmAgent::refresh ORACLE_HOME = /opt/app/grid/11.2.0
2018-10-29 23:26:21.814: [ora.asm][4040279808]{0:0:2} [start] clsnUtils::cmdIdIsStart CmdId:257
2018-10-29 23:26:21.814: [ora.asm][4040279808]{0:0:2} [start] AsmAgent:getOracleSid 2 oracle_sid = 
2018-10-29 23:26:21.814: [ora.asm][4040279808]{0:0:2} [start] AsmAgent:getOracleSid 4 oracle_sid = 
2018-10-29 23:26:21.819: [ora.asm][4040279808]{0:0:2} [start] Node 0 doesn't exist
2018-10-29 23:26:21.819: [ora.asm][4040279808]{0:0:2} [start] AsmAgent::getOracleSid: Exception IntException
2018-10-29 23:26:21.819: [ora.asm][4040279808]{0:0:2} [start] AsmAgent::getOracleSidAttrib getCssNodeName Error: Agents: clssnsqlnum ret = 12: 
2018-10-29 23:26:21.819: [ora.asm][4040279808]{0:0:2} [start] AsmAgent:getOracleSid 5 oracle_sid = +ASM1
2018-10-29 23:26:21.819: [ora.asm][4040279808]{0:0:2} [start] AsmAgent:getOracleSid oracle_sid = +ASM1
2018-10-29 23:26:21.819: [ora.asm][4040279808]{0:0:2} [start] AsmAgent::refresh ORACLE_SID = +ASM1
2018-10-29 23:26:21.819: [ USRTHRD][4040279808]{0:0:2} Thread:CSS Data Threadthread constructor
2018-10-29 23:26:21.819: [ora.asm][4040279808]{0:0:2} [start] CssData Constructor groupName:HB+ASM cssRegistrationType:0 memno:-1
2018-10-29 23:26:21.819: [ USRTHRD][4040279808]{0:0:2} CssData::initMutex &m_cssDataLock:a410f070
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] CssGroup::regis registration of groupName:HB+ASM mbrid:-1 m_grpNum:-1 regType:0 succeeded
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] CssData::Constructor regis succeeded groupName:HB+ASM memno:-1
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] CssData::Constructor m_pCssGroup:a4068f40 group HB+ASM
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] InstAgent::init m_groupName:HB+ASM m_pCssData a410dfa0
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] Utils::getOracleHomeAttrib getEnvVar oracle_home:/opt/app/grid/11.2.0
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] Utils::getOracleHomeAttrib oracle_home:/opt/app/grid/11.2.0
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] ConnectionPool::ConnectionPool 2 m_oracleHome:/opt/app/grid/11.2.0, m_oracleSid:+ASM1, m_usrOraEnv:
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] updating GEN_USR_ORA_INST_NAME to +ASM1
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] clsnUtils::setResAttrib nonPerX current value GEN_USR_ORA_INST_NAME value +ASM1
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] setResAttrib clsagfw_modify_attribute attr GEN_USR_ORA_INST_NAME value +ASM1 retCode 0
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] sModifyConfig for +ASM
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] Utils::getOracleHomeAttrib getEnvVar oracle_home:/opt/app/grid/11.2.0
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] Utils::getOracleHomeAttrib oracle_home:/opt/app/grid/11.2.0
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] sclsnInstAgent::sUpdateOratab oratab is /etc/oratab
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] ConfigFile::setAltName altName = +ASM
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] ConfigFile::setAltValue altValue = /opt/app/grid/11.2.0:N
2018-10-29 23:26:21.820: [ora.asm][4040279808]{0:0:2} [start] sclsnInstAgent::sUpdateOratab CssLock
2018-10-29 23:26:21.821: [ora.asm][4040279808]{0:0:2} [start] clsnUtils::cmdIdIsStart CmdId:257
2018-10-29 23:26:21.821: [ora.asm][4040279808]{0:0:2} [start] AsmAgent:getOracleSid 2 oracle_sid = 
2018-10-29 23:26:21.821: [ora.asm][4040279808]{0:0:2} [start] AsmAgent:getOracleSid 4 oracle_sid = 
2018-10-29 23:26:21.821: [ora.asm][4040279808]{0:0:2} [start] Node 0 doesn't exist
2018-10-29 23:26:21.821: [ora.asm][4040279808]{0:0:2} [start] AsmAgent::getOracleSid: Exception IntException
2018-10-29 23:26:21.821: [ora.asm][4040279808]{0:0:2} [start] AsmAgent::getOracleSidAttrib getCssNodeName Error: Agents: clssnsqlnum ret = 12: 
2018-10-29 23:26:21.821: [ora.asm][4040279808]{0:0:2} [start] AsmAgent:getOracleSid 5 oracle_sid = +ASM1
2018-10-29 23:26:21.821: [ora.asm][4040279808]{0:0:2} [start] AsmAgent:getOracleSid oracle_sid = +ASM1
2018-10-29 23:26:21.821: [ora.asm][4040279808]{0:0:2} [start] ConfigFile::getAltEntry altEntry = +ASM1
2018-10-29 23:26:21.822: [ora.asm][4040279808]{0:0:2} [start] ConfigFile::getValue name +asm1 value /opt/app/grid/11.2.0:N
2018-10-29 23:26:21.822: [ora.asm][4040279808]{0:0:2} [start] sclsnInstAgent::sUpdateOratab entry exists with dbName +ASM1 value /opt/app/grid/11.2.0:N
2018-10-29 23:26:21.822: [ora.asm][4040279808]{0:0:2} [start] sclsnInstAgent::sUpdateOratab CSS unlock
2018-10-29 23:26:21.822: [ora.asm][4040279808]{0:0:2} [start] getResAttrib: attrib name USR_ORA_OPI value false len 5
2018-10-29 23:26:21.822: [ora.asm][4040279808]{0:0:2} [start] clsnInstConnection::makeConnectStr UsrOraEnv  m_oracleHome /opt/app/grid/11.2.0 Crshome /opt/app/grid/11.2.0
2018-10-29 23:26:21.822: [ora.asm][4040279808]{0:0:2} [start] makeConnectStr = (DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/opt/app/grid/11.2.0/bin/oracle)(ARGV0=oracle+ASM1)(ENVS='ORACLE_HOME=/opt/app/grid/11.2.0,ORACLE_SID=+ASM1')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(CONNECT_DATA=(SID=+ASM1)))
2018-10-29 23:26:21.823: [ora.asm][4040279808]{0:0:2} [start] Container:start oracle home /opt/app/grid/11.2.0
2018-10-29 23:26:21.823: [ora.asm][4040279808]{0:0:2} [start] InstConnection::connectInt: server not attached
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] ORA-21561: OID generation failed

2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] InstConnection::connectInt (2) Exception OCIException
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] InstConnection:connect:excp OCIException OCI error 21561
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] InstAgent::start: 1 errcode 21561
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] InstAgent::start: untrapped error
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] InstAgent::start: 2 errcode 21561
2018-10-29 23:26:21.834: [ USRTHRD][4040279808]{0:0:2} InstConnection:~InstConnection: this a4076770
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] clean {
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] InstAgent::stop_option stop mode immediate option 1
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] InstAgent::stop { 
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] getResAttrib: attrib name USR_ORA_OPI value false len 5
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] InstAgent::stop original reason system do shutdown abort
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] ConnectionPool::resetConnection  s_statusOfConnectionMap 00b07d28
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] ConnectionPool::resetConnection sid +ASM1 status  2
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] Gimh::check OH /opt/app/grid/11.2.0 SID +ASM1
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] Gimh::check condition changes to (GIMH_NEXT_NUM) 1(Instance Shutdown) exists
2018-10-29 23:26:21.834: [ora.asm][4040279808]{0:0:2} [start] CrsCmd::ClscrsCmdData::stat entity 5 statflag 32 useFilter 1
2018-10-29 23:26:21.835: [ COMMCRS][2610939648]clsc_connect: (0x7f1fa40e1ad0) no listener at (ADDRESS=(PROTOCOL=IPC)(KEY=CRSD_UI_SOCKET))

除了报错，连接数据库出现异常，异常如下

[oracle@racoa01 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Tue Oct 30 00:47:45 2018

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

connect 

SQL >

这种现象很诡异，在mos上搜索相关文档

Connection Via Oracle Net Fails With Error ORA-21561: OID Generation Failed (文档 ID 559981.1)

Changes

It is likely that a recent upgrade has taken place.
This error is usually the result of a hostname (FQDN or shortname) resolution issue.

Cause

Incorrect /etc/hosts mapping for the client machine or no mapping at all. 

Solution

Check the client's /etc /.../hosts file and verify that the local node is correctly identified with a mapping for both the short and fully qualified name of the host.

对比主机名发现

/etc/hosts 主机名为 racoa01

而 /etc/sysconfig/network 主机名为 racoa01.bjzb

将/etc/sysconfig/network 主机名更改为 racoa01，重启即可

猜你喜欢

转载自blog.csdn.net/li66934791/article/details/83515792
oid
今日推荐
基于大语言模型的开源知识库问答系统 MaxKB GitHub Star 数量突破 5,000 个！
美国拟限制 AI 大模型出口中国和俄罗斯
苹果将与 OpenAI 达成协议，将 ChatGPT 应用于 iPhone
openKylin 社区生态委员会第六次会议圆满召开
阿里云正式发布通义千问 2.5
Python 3.13 发布首个 Beta：实验性自由线程模式和 JIT、改进交互式解释器
Stack Overflow 拿我的代码去训练 AI 大模型，还封了我的账号
Pop!_OS 的 COSMIC 桌面完成 App Store 上架工作
报告：Django 仍然是 74% 开发者的首选
《2024 年一季度互联网投融资运行情况》研究报告
15 年前上了“FFmpeg 耻辱柱”，今天他还得谢谢咱——腾讯QQPlayer一雪前耻？
TIOBE 5 月榜单：Fortran “复活”进入 Top 10
周排行
BPM为企业带来的实际利益
好程序员web前端分享css常用属性缩写
Java文件下载（excel）
css样式的动态添加及显示和隐藏等零碎用法
axios全局配置以及拦截器
使用Logstash来实时同步MySQL和log日志数据到ES
C++获取当前时间（年月日、时分秒、毫秒）
Odoo产品分析 (四) -- 工具板块(11) -- 网站即时聊天(1)
Java环境配置正确，但是java、javac、java -version均返回“不是内部或外部命令，也不是可运行的程序或批处理文件”？
01 官网下载各种CentOS教程（超详细版）
每日归档
更多
2024-05-14(0)
2024-05-13(18)
2024-05-12(0)
2024-05-11(38)
2024-05-10(38)
2024-05-09(35)
2024-05-08(42)
2024-05-07(14)
2024-05-06(40)
2024-05-05(0)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022