2008/03/14
skate
相应的版本:
os: (cent os)Linux movo2u 2.6.9-5.ELsmp #1 SMP Wed Jan 5 19:30:39 EST 2005 i686 i686 i386 GNU nux
db: Oracle Database 10g Enterprise Edition Release 10.2.0.3.0
ORA-00600: internal error code, arguments: [4097], [], [], [], [], [], [], []
4000开头的600错误大多数都和回滚段有关
今天早晨发现应用程序无法登录,返回给前端应用程序的错误是:
OERR: ORA-12519 TNS:no appropriate service handler found
这个错误前几天遇到过,推断是连接数太多,
select count(*) from v$process;
select * from v$process;
select name,value from v$parameter where name = 'processes';
select * from v$license
当我用下面的命令生成的命令:
select 'alter system kill session '''||s.SID||','||s.SERIAL#||''';'
from v$session s
where s.USERNAME='USERNAME';
删除用户会话,前端程序就不提示错误了.但由于程序的问题,这种删除会话只是暂时的,一会会话数量又上来了
有的时候我通过sql developer都登录不上去,而我用通过ssh登录服务器, 通过sqlplus登录,依旧提示
超出连接数,我于是和开发人员沟通,重启自应用服务.
我登录数据库:
我在查日志的时候发现:
alter.log:
Fri Mar 14 11:47:15 2008
Errors in file /home/oracle/admin/omovo/udump/omovo_ora_31788.trc:
ORA-00600: internal error code, arguments: [4097], [], [], [], [], [], [], []
Fri Mar 14 11:47:23 2008
Errors in file /home/oracle/admin/omovo/udump/omovo_ora_31720.trc:
ORA-00600: internal error code, arguments: [4097], [], [], [], [], [], [], []
Fri Mar 14 11:47:34 2008
/home/oracle/admin/omovo/udump/omovo_ora_31788.trc:
----------------------------------------------
*** 2008-03-14 11:32:45.614
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [4097], [], [], [], [], [], [], []
Current SQL statement for this session:
update b_weblog set click_count=:1, updateTime=:2 where id=:3
----- Call Stack Trace -----
calling call entry argument values in hex
location type point (? means dubious value)
-------------------- -------- -------------------- ----------------------------
ksedst()+27 call ksedst1() 0 ? 1 ?
ksedmp()+557 call ksedst() 0 ? 0 ? 0 ? 0 ? 0 ? 0 ?
ksfdmp()+19 call ksedmp() 3 ? BFFF07A4 ? ACC92A0 ?
C806D00 ? 3 ? C7C88D8 ?
kgeriv()+188 call 00000000 C806D00 ? 3 ?
kgesiv()+118 call kgeriv() C806D00 ? B72B11BC ? 1001 ?
0 ? BFFF0810 ?
ksesic0()+44 call kgesiv() C806D00 ? B72B11BC ? 1001 ?
0 ? BFFF0810 ? 1001 ? 0 ?
BFFF0810 ?
ktugti()+2510 call ksesic0() 1001 ? 0 ? 0 ? 0 ? 0 ? 0 ?
ktbgfi()+6705 call ktugti() 21BD602C ? BFFF0A90 ? 0 ?
BFFF0AAC ? 0 ? 0 ? 0 ? 0 ?
kdisdelete()+10297 call ktbgfi() BFFF1388 ? BFFF1440 ? 103 ?
0 ? BFFF143C ? 0 ? 45 ?
__PGOSF323_kdisnew_ call kdisdelete() B72B23E0 ? BFFF1FA0 ?
bseg_srch_cbk()+295 181898D ? 0 ?
ktspfpblk()+400 call 00000000 BFFF1FA0 ? BFFF19A8 ?
ktspfsrch()+469 call ktspfpblk() BFFF1678 ? 6 ? 1813E09 ?
34C2 ? 34C2 ? 30008 ?
ktspscan_bmb()+274 call ktspfsrch() BFFF1678 ? 1813E09 ? 0 ?
1813E09 ?
ktspgsp_cbk1()+451 call ktspscan_bmb() BFFF1678 ? B72B23E8 ? 1 ? 2 ?
1813B7A ? 0 ?
ktspgsp_cbk()+70 call ktspgsp_cbk1() B72B23E8 ? 0 ? 0 ? 2 ?
BFFF1FA0 ? 0 ?
kdisnew()+184 call ktspgsp_cbk() B72B23E8 ? 0 ? 0 ? 2 ?
BFFF1FA0 ? 0 ?
kdisnewle()+81 call kdisnew() B72B23D8 ? 181898D ?
BFFF1FA0 ? 2 ? 0 ? 0 ?
kdisle()+3884 call kdisnewle() B72B23D8 ? 181898D ?
BFFF1FA0 ? BFFF2084 ?
B72A8B90 ? C8 ?
kdiins0()+28307 call kdisle() 77F8C74C ? BFFF226C ?
BFFF2CE4 ? 1 ? 2 ? BFFFA190 ?
kdiins()+95 call kdiins0() 77F8C74C ? 0 ? 0 ? BFFFA190 ?
0 ? 0 ?
kauupd()+4591 call kdiins() 77F8C74C ? 0 ? 0 ? BFFFA190 ?
2 ? FF ? 0 ? 0 ?
updrow()+1328 call kauupd() B72C866C ? 77F8C784 ?
B72DC9E0 ? 0 ? 77F8D700 ? 5 ?
333E ? 77F8CB10 ? 18 ?
BFFFB0A4 ? BFFFB05C ?
77F8D958 ?
__PGOSF357_qerupRow call updrow() 77F8F360 ? 7FFF ? 0 ? 48 ?
Procedure()+62 77F8D220 ? BFFFB388 ?
qerupFetch()+510 call 00000000 77F8CEB0 ? 7FFF ?
updaul()+900 call 00000000 77F8CEB0 ? 0 ? 77F8CD44 ?
7FFF ?
updThreePhaseExe()+ call updaul() 77F8F360 ? BFFFB830 ? 0 ?
274 C80E614 ? BFFFB898 ?
AF7187F ?
updexe()+283 call updThreePhaseExe() 77F8F360 ? 0 ? B72DC9E0 ?
BFFFB8FC ? 77F8F360 ? 1 ?
BFFFB8FC ? 0 ?
opiexe()+4025 call updexe() 77F8F360 ? BFFFC16C ?
kpoal8()+2089 call opiexe() 49 ? 3 ? BFFFC4CC ?
opiodr()+985 call 00000000 5E ? 17 ? BFFFEDD0 ?
ttcpip()+1093 call 00000000 5E ? 17 ? BFFFEDD0 ? 0 ?
opitsk()+1031 call ttcpip() C80E500 ? 5E ? BFFFEDD0 ? 0 ?
BFFFEAAC ? BFFFEEE0 ?
opiino()+821 call opitsk() 0 ? 0 ?
opiodr()+985 call 00000000 3C ? 4 ? BFFFF9A0 ?
opidrv()+466 call opiodr() 3C ? 4 ? BFFFF9A0 ? 0 ?
sou2o()+91 call opidrv() 3C ? 4 ? BFFFF9A0 ?
opimai_real()+117 call sou2o() BFFFF984 ? 3C ? 4 ?
BFFFF9A0 ?
main()+111 call opimai_real() 2 ? BFFFF9D0 ?
__libc_start_main() call 00000000 2 ? BFFFFA94 ? BFFFFAA0 ?
+227 699B46 ? 7CBFF4 ? 0 ?
--------------------- Binary Stack Dump ---------------------
通过查看日志文件,问题是处在update b_weblog表有错误,我于是就用exp/imp这个表,结果问题解决,
日志不在报ora-600错误,应用模块一可以访问了
[oracle@skate-test ~]$ export NLS_LANG=AMERICAN_AMERICA.ZHS32GB18030
expdp skate/skate dumpfile=mo_vo_b_weblog.dmp tables=b_weblog
impdp skate/skate dumpfile=mo_vo_b_weblog.dmp tables=b_weblog remap_schema=mo_vo:mo_vo table_exists_action= replace
或
exp skate/skate tables=(b_weblog) file=/home/oracle/online_script/dmp/mo_vo_b_weblog_exp.dmp grants=y
imp skate/skate fromuser=mo_vo touser=mo_vo file=/home/oracle/online_script/dmp/mo_vo_b_weblog_exp.dmp ignore=y
这样以后,在查询alter.log日志,发现正常了,就是日志切换了
还有一种方法,就是删除掉坏的回滚段,以前我的库就有回滚段坏了,然后我就用下面的方法,但这次我没查到有错误的回滚段
参考文档: http://space.8see.com/467/viewspace-4194
查看回滚段:
select SEGMENT_NAME,STATUS from dba_rollback_segs;
Summary:
Problem: Series of Ora-00600 [4097] and Ora-00600 [25012] errors
related to undotablespace "UNDOTBS" :tsn# 33 and undo segment#10.
Ora-00600[4097] started on Fri Oct 27 01:33:04 2006
Ora-00600[25012] started on Fri Oct 27 01:33:34 2006
Ora-00600[4097]
~~~~~~~~~~~~~~~ .
The error Ora-00600[4097] started occuring on segment number 10
in tsn# 33-undotbs.
xid dump in the trace files shows higher wrap# for the slot than what is present in undo segment header for usn# 10.
From dp06_ora_7324.trc,
----------------------- .
xid: 0x000a.032.00028790
.
undo segment header 10 shows wrap# 0x282ef for slot 0x32. could not find reference to the xid "0x000a.032.00028790" in any of the data blocks dumped in trace file.
Call stack
===========
ksedmp kgeriv kgesiv ksesic0 ktugti ktsftcmove
ktsf_gsp kdtgsp kdtgrs kdtInsRow insrow insdrv
insexe opiexe opiall0 opikpr opiodr rpidrus
skgmstack rpidru rpiswu2 kprball kqlsrcchg kqlchg
kglfls ktcrcm ktdcmt k2lcom k2send xctctl
xctcom opiexe opiosq0 kpooprx kpoal8 opiodr
ttcpip opitsk opiino opiodr opidrv sou2o main start
dbv on all undo datafiles were clean.
They tried switching to a new undo tablespace UNDOTBS2. But still the errors continued.
After dropping the undo segment 10 using _smu_debug_mode, there are no Ora-00600[4097] errors seen.
However,once the load increased in database,"_SYSSMU10$ " got automatically created and then they see lots of Ora-00600[25012] errors.
After dropping undo segment 10, they are still using UNDOTBS as active undo tablespace.
Ora-00600[25012]
~~~~~~~~~~~~~~~ .
The index block had an open itl which was referencing undo segment 10 and the uba was referring to datafile 583 which is not part of UNDOTBS.
Call stack (from dp06_ora_29049.trc)
===========
ksedmp kgeriv kgesiv ksesic2 kftr2ah kftr2bz
kturbk ktrgcm ktrgtc kdiixs1 qerixFetch qertbFetchByRowID
qersoFetch opifch2 opiall0 kpoal8 opiodr ttcpip
opitsk opiino opiodr opidrv sou2o main sta
From trace files,the symptoms are index blocks have open itls referencing undo segment number 10 and dba pointing to file 583 which is not part of UNDOTBS.
Hence the CR generation fails with Ora-00600 [25012].
This error occurs from different jobs and on different objects.
Dropping and recreating indexes solves the problem.
dbv does not report any problem on the affected index datafiles.
附：如何drop回滚段_SYSSMU10$
1、做之前先做一个全库备份
2、修改pfile:
*.undo_management='MANUAL';
_offline_rollback_segments=(_SYSSMU1$,_SYSSMU2$,_SYSSMU3$,_SYSSMU4$,_SYSSMU5$,_SYSSMU6$,_SYSSMU7$,_SYSSMU8$,_SYSSMU9$,_SYSSMU10$)
3、以pfile启动然后执行下述语句：
DROP ROLLBACK SEGMENT "_SYSSMU10$";
4、将pfile中隐含参数_offline_rollback_segments去掉后再重启数据库。
再分享一下我老师大神的人工智能教程吧。零基础！通俗易懂！风趣幽默！还带黄段子！希望你也加入到我们人工智能的队伍中来！https://blog.csdn.net/jiangjunshow