As a part of daily importing a job i am TRUNCATING the data for tables and inserting data from raw file.
Some tables import usually large data like in the range 200,000 rows but some tables import less data in the range 5,000 to 30,000 rows.
For large table import it will take lot of time during insertion and at the end sometimes it will end with performance issue.
I am using Java for processing this imports into Oracle table. So this table is not hardcoded and import according to their job.
jdbcTemplate.execute("TRUNCATE TABLE " + tableName); --for truncating table
return "INSERT INTO " + tableName + " (" +
columnData.stream().filter(Objects::nonNull).map(ColumnData::getName).collect(joining(", ")) +
", SYSTEM_INSERTED_AT) VALUES (" +
values.stream().collect(joining(", ")) +
", CURRENT_TIMESTAMP)"; --for insertion into table
I would like to know is it good idea to use /*+ APPEND */ hint during insertion and does it improve performance during insertion ? As read in the oracle article for less insertion its not good idea to use /*+ APPEND */ as it will end up affecting the performance issue.
Your program almost certainly will not benefit from the APPEND hint. Batching commands is more likely to improve performance.
The APPEND hint is helpful when inserting a huge amount of data. It improves performance by writing data directly to data files, and bypassing updating memory structures, REDO, and UNDO. But we must be willing to accept the consequences of those shortcuts - the operation locks the entire table, and the data will not be recoverable.
200,000 rows is not a huge amount of data. (Unless each row contains large files stored in LOB columns. For performance questions, it's often more useful to speak about bytes instead of rows.) If we make a wild assumption that each row has 1,000 bytes, that's still only 200 megabytes of data. Writing 200 megabytes to memory, UNDO, and REDO, shouldn't add more than a few seconds to the run time. Eliminating those seconds won't help much.
Batching is the best way to improve the performance of your program. Every database command has a significant amount of overhead, mostly for network lag and parsing. Combining multiple commands can significantly reduce that overhead and boost performance.
Check if your database API or framework has a batching option. Enabling it might be as simple as a configuration change. You don't need to batch everything, just combining one hundred rows at a time will eliminate 99% of the overhead
If you don't have a batching option, can build your own solution by concatenating strings. The below code demonstrates two different ways to insert multiple rows in a single command.
create table table1(a number, b number);
insert into table1
select 1, 1 from dual union all
select 2, 2 from dual;
insert all
into table1 values(3, 3)
into table1 values(4, 4)
select * from dual;