In many cases, we will need to insert a large amount of data into a table, and we hope to complete the work in the shortest possible time. Here, I will share with you some of my usual experience in doing a large amount of data insert.
Premise: Before inserting data, if it is a non-production environment, please remove the indexes and constraints of the table, and then build the indexes and constraints after the insert is completed.
1.
| insert into tab1 select * from tab2; commit; |
This is the most basic insert statement. We insert the data in the tab2 table into the tab1 table. According to experience, tens of millions of data can be completed within 1 hour. However, the arch generated by this method will be very fast. It is necessary to pay attention to the amount of archives generated, and start the backup software in time to avoid the burst of the arch directory.
2.
| alter table tab1 nologging; insert /*+ append */ into tab1 select * from tab2; commit; alter table tab1 logging; |
This method will greatly reduce the generation of arch, and improve the time to a certain extent. According to experience, tens of millions of data can be completed within 45 minutes. However, please note that this method is suitable for the serial mode of a single process. If there are multiple processes running at the same time, the process initiated later will have an enqueue waiting. Note that this method must not be used on dataguard (but if you have force logging in the database, you are not afraid, huh)!!
3.
| insert into tab1 select /*+ parallel */ * from tab2; commit; |
For the case where the statement after select is a full table scan, we can add parallel hint to improve its concurrency. It should be noted here that the maximum concurrency is limited by the initialization parameter parallel_max_servers, and concurrent processes can be viewed through v$px_session, or ps -ef |grep ora_p view.
4.
| alter session enable parallel dml; insert /*+ parallel */ into tab1 select * from tab2; commit; |
Contrary to method 2, the concurrent insert has not been compared with method 2 which is more efficient (even it is estimated that method 2 is faster). Friends who have tested are welcome to add.
5.
| insert into tab1 select * from tab2 partition (p1); insert into tab1 select * from tab2 partition (p2); insert into tab1 select * from tab2 partition (p3); insert into tab1 select * from tab2 partition (p4); |
For a partitioned table, you can use tab1 to perform concurrent inserts of multiple processes. The more partitions, the more processes that can be started. I once tried to insert a table with 260 million rows of records, 8 partitions, and 8 processes. If method 2 is used, it may take 40 minutes for a single process to complete, but since there are 8 partitions and 8 processes, the later process has an enqueue , so the time required is 40 minutes × 8; but if method 5 is used, although a single process takes 110 minutes, the total time required is about 110 minutes because of the ability to execute concurrent processes.
6.
| DECLARE TYPE dtarray IS TABLE OF VARCHAR2(20) INDEX BY BINARY_INTEGER; v_col1 dtarray; v_col2 dtarray; v_col3 dtarray; BEGIN SELECT col1, col2, col3 BULK COLLECT INTO v_col1, v_col2, v_col3 FROM tab2; FORALL i IN 1 .. v_col1.COUNT insert into tab1 WHERE tab1.col1 = v_col1; END; |
用批量绑定(bulk binding)的方式。当循环执行一个绑定变量的sql语句时候，在PL/SQL 和SQL引擎(engines)中，会发生大量的上下文切换(context switches)。使用bulk binding，能将数据批量的从plsql引擎传到sql引擎，从而减少上下文切换过程，提升效率。该方法比较适合于在线处理，不必停机。
7.
| sqlplus -s user/pwd< runlog.txt set copycommit 2; set arraysize 5000; copy from user/pwd@sid - to user/pwd@sid - insert tab1 using select * from tab2; exit EOF |
用copy的方法进行插入，注意此处insert没有into关键字。该方法的好处是可以设置copycommit和arrarysize来一起控制commit的频率，上面的方法是每10000行commit一次。
转自:http://www.cnblogs.com/husam/p/5586168.html