Author | JiekeXu
Source|Public Account JiekeXu DBA Road (ID: JiekeXu_IT)
If you need to reprint, please contact for authorization | (Personal WeChat ID: JiekeXu_DBA)
Hello everyone, I am JiekeXu. I am very glad to meet you again. Today, I will take a look at the introduction of Oracle SQL*Loader with you. Welcome to click on the blue word "JiekeXu DBA Road" above to follow my official account, star or put it on the top , more dry goods will arrive as soon as possible!
The previous article briefly introduced the installation and use of Oracle's large data volume export tool - sqluldr2 . The birth of sqluldr2 is mainly used to quickly export large quantities of Oracle data into CSV/Text text format, which is convenient for importing into other databases. The transformation is in full swing. This tool is also widely used in domestic database migration. It is worth learning and using. Today I will talk about the data import tool SQL*Loader (sqlldr) that comes with the Oracle database. As long as you install it Oracle database, then this tool exists in the ORACLE_HOME/bin directory, and its function is to load the DAT/CSV/Text files exported from other databases into the Oracle database. The data pump import requires a dmp file, and the execution of the insert statement requires a .sql file. Of course, the form of an external table is also possible, but the external table cannot be edited and the file is located outside the database. If you cannot update and edit data, consider directly loading it into sqlldr It is more convenient in Oracle database. 
Principle of SQL*Loader
Import external data (such as text) into the Oracle database. (Can be used for different types of database data migration) The essence is to re-insert records in the segment (segment table).
①conventional: To insert records into blocks below the HWM (high water mark) of the segment, first access the bitmap to determine which blocks have free space
②direct path: Insert records into unused blocks above the segment's HWM (high water mark), bypass db_buffer, and do not check constraints. Redo can also be turned off, and parallel operations are also supported to speed up insertion.
Traditional insert data and direct insert data:
SQL> create table emp1 as select * from emp where 1=2;
SQL> insert into emp1 select * from emp; traditional way data
SQL> insert /*+ APPEND */ into emp1 select * from emp; direct data, you must commit to view the data
[Experimental] Direct Path Insert Data
SQL>createtableemp1asselect*fromempwhere1=2;
SQL>insertintoemp1select*from emp; conventional Traditional way data
SQL> insert /*+ APPEND */ into emp1 select * from emp; direct data, must be committed to view the data
create table insert data
SCOTT@PROD>create table test as select * from emp;
SCOTT@PROD>insert into test select * from test;
SCOTT@PROD>/
SCOTT@PROD>delete from test;
SCOTT@PROD>commit;
SCOTT@PROD>analyze table test compute statistics;
SCOTT@PROD>selecttable_name,blocks,empty_blocks from user_tables where table_name='TEST';
normal insert
SCOTT@PROD>insert into test select * from emp;
SCOTT@PROD>commit;
SCOTT@PROD>analyze table test compute statistics;
SCOTT@PROD>select table_name,blocks,empty_blocks from user_tables where table_name='TEST';
direct path insertion
SCOTT@PROD>insert /*+append*/ into test select * from emp;
SCOTT@PROD>analyze table test compute statistics;
SCOTT@PROD>selec ttable_name,blocks,empty_blocks from user_tables where table_name='TEST';
Sqlldr usage
--示例sqlldr test/mypassword@pdb1 /home/oracle/test.ctl log=test.log
[oracle@test19crac1 bin]$ sqlldr -help
SQL*Loader: Release 19.0.0.0.0 - Production on Wed Aug 30 00:22:53 2023
Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle and/or its affiliates. All rights reserved.
Usage: SQLLDR keyword=value [,keyword=value,...]
Valid Keywords:
userid -- ORACLE username/password
control -- control file name
log -- log file name
bad -- bad file name
data -- data file name
discard -- discard file name
discardmax -- number of discards to allow (Default all)
skip -- number of logical records to skip (Default 0)
load -- number of logical records to load (Default all)
errors -- number of errors to allow (Default 50)
rows -- number of rows in conventional path bind array or between direct path data saves
(Default: Conventional path 250, Direct path all)
bindsize -- size of conventional path bind array in bytes (Default 1048576)
silent -- suppress messages during run (header,feedback,errors,discards,partitions)
direct -- use direct path (Default FALSE)
parfile -- parameter file: name of file that contains parameter specifications
parallel -- do parallel load (Default FALSE)
file -- file to allocate extents from
skip_unusable_indexes -- disallow/allow unusable indexes or index partitions (Default FALSE)
skip_index_maintenance -- do not maintain indexes, mark affected indexes as unusable (Default FALSE)
commit_discontinued -- commit loaded rows when load is discontinued (Default FALSE)
readsize -- size of read buffer (Default 1048576)
external_table -- use external table for load; NOT_USED, GENERATE_ONLY, EXECUTE
columnarrayrows -- number of rows for direct path column array (Default 5000)
streamsize -- size of direct path stream buffer in bytes (Default 256000)
multithreading -- use multithreading in direct path
resumable -- enable or disable resumable for current session (Default FALSE)
resumable_name -- text string to help identify resumable statement
resumable_timeout -- wait time (in seconds) for RESUMABLE (Default 7200)
date_cache -- size (in entries) of date conversion cache (Default 1000)
no_index_errors -- abort load on any index errors (Default FALSE)
partition_memory -- direct path partition memory limit to start spilling (kb) (Default 0)
table -- Table for express mode load
date_format -- Date format for express mode load
timestamp_format -- Timestamp format for express mode load
terminated_by -- terminated by character for express mode load
enclosed_by -- enclosed by character for express mode load
optionally_enclosed_by -- optionally enclosed by character for express mode load
characterset -- characterset for express mode load
degree_of_parallelism -- degree of parallelism for express mode load and external table load
trim -- trim type for express mode load and external table load
csv -- csv format data files for express mode load
nullif -- table level nullif clause for express mode load
field_names -- field names setting for first record of data files for express mode load
dnfs_enable -- option for enabling or disabling Direct NFS (dNFS) for input data files (Default FALSE)
dnfs_readbuffers -- the number of Direct NFS (dNFS) read buffers (Default 4)
sdf_prefix -- prefix to append to start of every LOB File and Secondary Data File
help -- display help messages (Default FALSE)
empty_lobs_are_null -- set empty LOBs to null (Default FALSE)
defaults -- direct path default value loading; EVALUATE_ONCE, EVALUATE_EVERY_ROW, IGNORE, IGNORE_UNSUPPORTED_EVALUATE_ONCE, IGNORE_UNSUPPORTED_EVALUATE_EVERY_ROW
direct_path_lock_wait -- wait for access to table when currently locked (Default FALSE)
PLEASE NOTE: Command-line parameters may be specified either by
position or by keywords. An example of the former case is 'sqlldr
scott/tiger foo'; an example of the latter is 'sqlldr control=foo
userid=scott/tiger'. One may specify parameters by position before
but not after parameters specified by keywords. For example,
'sqlldr scott/tiger control=foo logfile=log' is allowed, but
'sqlldr scott/tiger control=foo log' is not, even though the
position of the parameter 'log' is correct.SQL*Loader Control File
SQL*Loader To load data into an Oracle database, a SQL*Loader control file is required, which is a text file written in a language that SQL*Loader understands.
Control files tell SQL*Loader where to find data, how to analyze and interpret data, where to insert data, and so on.
Typically, a control file has three main sections, in the following order:
Session scoped information
Table and field list information
Enter data (optional section)
Some control file syntax considerations to keep in mind include:
Syntax is free-form (statements can extend across multiple lines).
The syntax is case-insensitive; however, strings enclosed in single or double quotes are taken literally, including case.
In control file syntax, comments are extended from two hyphens ( ) that mark the beginning of a comment to the end of the line. The third section of the optional control file is interpreted as data rather than control file syntax; therefore, comments in this section are not supported.--
Keywords have special meaning to SQL*Loader and are therefore reserved. To avoid potential conflicts, Oracle recommends that you do not use or as the name of any table or column.CONSTANTZONECONSTANTZONE
The SQL*Loader program reads data from one or more data files (or operating system equivalents for files) specified in the control file .
From SQL*Loader's perspective, the data in a data file is organized into records. Specific data files can be in fixed record format, variable record format, or stream record format. The record format can be specified in the control file using parameters. If no record format is specified, it defaults to the stream record format.
Example: Using sqlldr, import emp1.dat into the common table emp1 under scott
create control file
[oracle]$vi /home/oracle/dir1/emp1.ctl
load data
infile '/home/oracle/dir1/emp1.dat'
insert
into table emp1
fields terminated by ',' (empno,ename,sal,comm,deptno)
Create emp1 table structure (internal table) under scott
SCOTT@PROD>create table emp1 as select * from emp1_ext where 1=2;
ORACLE_LOADER engine import (normal mode)
[oracle@oracle~]$ cd/home/oracle/dir1
[oracle@oracledir1]$ll
[oracle@oracledir1]$sqlldr
[oracle@oracle]$sqlldrscott/tigercontrol=emp1.ctllog=emp1.logdata=emp1.dat
Validation results
SCOTT@PROD>select * from emp1;
The data source and control file are merged and described in .ctl
[oracle@oracledir1]$vi emp2.ctl
load data
infile *
append
into table emp1
fields terminated by ','
optionally enclosed by '"'
(empno,ename,sal,comm,deptno)
begindata
8888,Jieke,1000,,"10"
9999,ORACLE,8000,100,20
[oracle@oracledir1]$sqlldrscott/tigercontrol=emp2.ctllog=emp2.log
Official document link:
https://docs.oracle.com/en/database/oracle/oracle-database/19/sutil/oracle-sql-loader-concepts.html#GUID-F99250F9-F720-45EE-8159-9B8A8E5A77DAfter the full text, I hope it can help you who are reading. If you think this article is helpful to you, you can share it with your friends, colleagues, and whoever you care about. Let’s learn and make progress together~~~
Welcome to follow my official account [JiekeXu DBA Road], and learn new knowledge together as soon as possible! You can find me at the following three addresses. The other addresses are pirated copyright infringement crawling my articles, and the code format, pictures, etc. are all messed up, making it inconvenient to read. Welcome to my official account or Motianlun address to follow me, the first time Get the latest news.
Welcome to follow my official account [JiekeXu DBA Road], and learn new knowledge together as soon as possible!
————————————————————————————————
Public account: JiekeXu DBA Road
CSDN: https://blog.csdn.net/JiekeXu
Motian Wheel: https://www.modb.pro/u/4347
———————————————————————————————
Share several database backup scripts
Oracle table fragmentation check and defragmentation scheme
OGG|Oracle GoldenGate 基础2022 年公众号历史文章合集整理Several problems encountered by Oracle 19c RAC
OGG|Oracle 数据迁移后比对一致性OGG|Oracle GoldenGate Microservice Architecture
Oracle query table space usage is extremely slow
Domestic database|TiDB 5.4 stand-alone quick installation first experience
Oracle ADG standby database shutdown maintenance process and incremental recovery
Linux environment to build MySQL8.0.28 master-slave synchronization environment