某开发库的一个undo数据文件莫名丢失，前台Java程序报错
Caused by: java.sql.SQLException: ORA-00376: file 2 cannot be read at this time
ORA-01110: data file 2: '/u03/oradata/SCOUTBV/undotbs01.dbf'
数据库alert log满屏都是
ORA-00604: error occurred at recursive SQL level 1
ORA-00376: file 2 cannot be read at this time
ORA-01110: data file 2: '/u03/oradata/SCOUTBV/undotbs01.dbf'
该数据库没有备份，数据文件无法恢复，因此考虑将该undo表空间删除重建
SQL> show parameter undo
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
undo_management string AUTO
undo_retention integer 900
undo_tablespace string UNDOTBS01
数据库默认undo表空间为UNDOTBS01，而出问题的是UNDOTBS1，其中file 2在OS上被物理删除，之前已经有DBA已经执行了offline drop
SQL> select FILE_NAME,FILE_ID,status,ONLINE_STATUS from dba_data_files where TABLESPACE_NAME ='UNDOTBS1';
FILE_NAME FILE_ID STATUS ONLINE_
-------------------------------------------------------------------------------- ---------- --------- -------
/u03/oradata/SCOUTBV/undotbs01.dbf 2 AVAILABLE RECOVER
/u05/oradata/SCOUTBV/undotbs01_1.dbf 56 AVAILABLE ONLINE
/u03/oradata/SCOUTBV/undotbs1.dbf 58 AVAILABLE ONLINE
SQL> alter tablespace UNDOTBS1 offline;
alter tablespace UNDOTBS1 offline
*
ERROR at line 1:
ORA-01191: file 2 is already offline - cannot do a normal offline
ORA-01110: data file 2: '/u03/oradata/SCOUTBV/undotbs01.dbf'
SQL> select SEGMENT_NAME,TABLESPACE_NAME,file_id ,STATUS from DBA_ROLLBACK_SEGS order by 2,3;
SEGMENT_NAME TABLESPACE_NAME FILE_ID STATUS
------------------------------ ------------------------------ ---------- ----------------
SYSTEM SYSTEM 1 ONLINE
_SYSSMU14$ UNDOTBS01 57 ONLINE
_SYSSMU21$ UNDOTBS01 57 ONLINE
_SYSSMU20$ UNDOTBS01 57 ONLINE
_SYSSMU19$ UNDOTBS01 57 ONLINE
_SYSSMU18$ UNDOTBS01 57 ONLINE
_SYSSMU13$ UNDOTBS01 57 ONLINE
_SYSSMU22$ UNDOTBS01 57 ONLINE
_SYSSMU15$ UNDOTBS01 57 ONLINE
_SYSSMU16$ UNDOTBS01 57 ONLINE
_SYSSMU17$ UNDOTBS01 57 ONLINE
_SYSSMU12$ UNDOTBS1 2 OFFLINE
_SYSSMU11$ UNDOTBS1 2 NEEDS RECOVERY
_SYSSMU10$ UNDOTBS1 2 NEEDS RECOVERY
_SYSSMU9$ UNDOTBS1 2 NEEDS RECOVERY
_SYSSMU8$ UNDOTBS1 2 NEEDS RECOVERY
SQL> alter system set "_smu_debug_mode"=4;
System altered.
SQL> drop rollback segment "_SYSSMU10$";
drop rollback segment "_SYSSMU10$"
*
ERROR at line 1:
ORA-01545: rollback segment '_SYSSMU10$' specified not available
无法直接删除状态为Needs recovery的undo segment，尝试采用隐含参数
SQL> alter system set "_corrupted_rollback_segments"='(_SYSSMU11$,_SYSSMU10$,_SYSSMU9$,_SYSSMU8$)' scope=spfile;
System altered.
SQL> alter system set "_offline_rollback_segments"='(_SYSSMU11$,_SYSSMU10$,_SYSSMU9$,_SYSSMU8$)' scope=spfile;
System altered
重启后drop操作报同样错误
SQL> drop rollback segment "_SYSSMU11$";
drop rollback segment "_SYSSMU11$"
*
ERROR at line 1:
ORA-01545: rollback segment '_SYSSMU11$' specified not available
查看字典表undo$
SQL> select NAME,FILE#,STATUS$ from undo$;
NAME FILE# STATUS$
------------------------------ ---------- ----------
SYSTEM 1 3
_SYSSMU1$ 2 1
_SYSSMU2$ 2 1
_SYSSMU3$ 2 1
_SYSSMU4$ 2 1
_SYSSMU5$ 2 1
_SYSSMU6$ 2 1
_SYSSMU7$ 2 1
_SYSSMU8$ 2 5
_SYSSMU9$ 2 5
_SYSSMU10$ 2 5
_SYSSMU11$ 2 5
_SYSSMU12$ 2 1
_SYSSMU13$ 57 3
_SYSSMU14$ 57 3
_SYSSMU15$ 57 3
_SYSSMU16$ 57 3
_SYSSMU17$ 57 3
_SYSSMU18$ 57 3
_SYSSMU19$ 57 3
_SYSSMU20$ 57 3
_SYSSMU21$ 57 3
_SYSSMU22$ 57 3
/* 1 = INVALID, 2 = AVAILABLE, 3 = IN USE, 4 = OFFLINE, 5 = NEED RECOVERY,
* 6 = PARTLY AVAILABLE (contains in-doubt txs)
*/
将状态5手工改为1
SQL> update undo$ set STATUS$=1 where STATUS$=5;
SQL> commit;
SQL> select SEGMENT_NAME,TABLESPACE_NAME,file_id ,STATUS from DBA_ROLLBACK_SEGS order by 2,3;
SEGMENT_NAME TABLESPACE_NAME FILE_ID STATUS
------------------------------------------------------------ ------------------------------ ---------- ----------------
SYSTEM SYSTEM 1 ONLINE
_SYSSMU14$ UNDOTBS01 57 ONLINE
_SYSSMU15$ UNDOTBS01 57 ONLINE
_SYSSMU16$ UNDOTBS01 57 ONLINE
_SYSSMU22$ UNDOTBS01 57 ONLINE
_SYSSMU18$ UNDOTBS01 57 ONLINE
_SYSSMU19$ UNDOTBS01 57 ONLINE
_SYSSMU20$ UNDOTBS01 57 ONLINE
_SYSSMU21$ UNDOTBS01 57 ONLINE
_SYSSMU13$ UNDOTBS01 57 ONLINE
_SYSSMU17$ UNDOTBS01 57 ONLINE
SQL> drop rollback segment "_SYSSMU9$";
Rollback segment dropped.
删除操作成功执行
从alert log可以看到随着最后一个原状态为needs recovery的undo segment被删除，原来的错误信息消失了
但是drop表空间时遇到错误，10046追踪一下
SQL> oradebug set mypid
ORA-00070: command set is not valid
SQL> oradebug setmypid
Statement processed.
SQL> oradebug event 10046 trace name context forever, level 12;
Statement processed.
SQL> drop tablespace UNDOTBS1 including contents;
drop tablespace UNDOTBS1 including contents
*
ERROR at line 1:
ORA-01561: failed to remove all objects in the tablespace specified
SQL> oradebug tracefile_name
/u01/scoutbv/admin/udump/scoutbv_ora_22548.trc
最后一条执行的sql如下
---
select null from seg$ where ts#=:1 and type# != 3 and type# != 9
END OF STMT
PARSE #7:c=0,e=861,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=4,tim=8504918038993
BINDS #7:
kkscoacd
Bind#0
acdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
acflg=08 fl2=0001 frm=00 csi=00 siz=24 ff=0
kxsbbbfp=ffffffff7dba31e8 bln=22 avl=02 flg=05
value=1
---
SQL> select TYPE# from seg$ where ts#=1 and type# != 3 and type# != 9;
TYPE#
----------
10
10
10
10
/* 1 = UNDO, 2 = SAVE UNDO, 3 = TEMPORARY, 4 = CACHE, 5 = DATA, 6 = INDEX */
/* 7 = SORT 8 = LOB 9 = Space Header 10 = System Managed Undo */
再次手工修改seg$表
SQL> delete from seg$ where ts#=1 and type# != 3 and type# != 9;
4 rows deleted.
SQL> drop tablespace UNDOTBS1 including contents;
Tablespace dropped.
