Oracle principle: data loading, SQLldr, external table

One, import SQLldr

SQL*LOADER can import txt files and Excel files into the database. Import and export using SQLloader requires a data file and a control file. The data file contains the data you need to import, and the control file writes how you need to import the data.

LOAD DATA
infile 'e:\aa.csv'                ## 源文件路径,路径不要包括中文
into table xx_temp  <impcmd>        ## 要导入的表 
(
　　id terminated by whitespace　  ## id 为列名,whitespace　表示列之间使用空格来区分,如果是其他的 ‘|’ 方式则使用 terminated by '|' 逗号则用逗号.以此类推
)
##   换行 也是自动终止字段读的标识

Among them, impcmd can be replaced by 4 values: insert (default default value), append, replace, truncate

insert: insert data, the table is required to be empty

append: append data, insert data on the basis of the original table

replace: delete the original table data, and then insert

truncate: delete the original table data, and then insert, more efficient than replace

For example, there is such a table

create  table salary_tbl(
   employer_nm varchar(20) ,
   department varchar(20) not null,
   salary number not null,
   leader_nm varchar(20)
)

Create a new .txt file for storing data, you can customize the separator to distinguish the fields

Create a new ctl separated by the character '|' 

Or it’s easier to write like this

load data
infile 'data1.txt'
into table SALARY_TBL append
fields terminated by '|'(
  employer_nm ,
  department ,
  salary	 ,
  leader_nm  
)

Execute cd in cmd to the corresponding directory for execution. After execution, you can view the contents of the log at the corresponding location

sqlldr userid=system/voapd@orcl control=control1.ctl log=11.log

Execute sqlldr in cmd to view the help of sqlldr

userid	ORACLE username/password
control	Control file name
log	Log file name
bad	Error file name
data	Data file name
discard	Obsolete file name
discardmax	The number of files allowed to be discarded (all default)
skip	The number of logical records to skip (default 0)
load	The number of logical records to be loaded (all default)
errors	Number of allowed errors (default 50)
rows	The number of rows in the regular path binding array or between the data saved by the direct path (default: regular path 64, all direct paths)
bindsize	The size of the regular path binding array (in bytes) (default 256000)
silent	Hide messages during operation (title, feedback, error, obsolete, partition)
direct	Use direct path (default FALSE)
parfile	Parameter file: the name of the file containing the parameter description
parallel	Perform parallel loading (default FALSE)
file	Files to allocate area from
skip_unusable_indexes	Disallow/allow useless indexes or index partitions (default FALSE)
skip_index_maintenance	No index is maintained, mark the affected index as useless (default FALSE)
commit_discontinued	Submit the line that was loaded when the load was interrupted (default FALSE)
readsize	The size of the read buffer (default 1048576)
external_table	Use external tables for loading; NOT_USED, GENERATE_ONLY, EXECUTE (default NOT_USED)
columnarrayrows	The number of rows of the direct path column array (default 5000)
streamsize	The size of the direct path stream buffer (in bytes) (default 256000)
multithreading	Use multithreading in direct path
resumable	Enable or disable the current resumable session (default FALSE)
resumable_name	A text string that helps identify the recoverable statement
resumable_timeout	RESUMABLE waiting time (in seconds) (default 7200)
date_cache	The size of the date conversion cache (in entries) (default 1000)
no_index_errors	Aborting loading when any index error occurs (default FALSE)
PLEASENOTE	Command line parameters can be specified by position or keywords. An example of the former is'sqlldrscott/tigerfoo'; an example of the latter is'sqlldrcontrol=foouserid=scott/tiger'. The time of the position specified parameter must be earlier than but not later than the parameter specified by the keyword. For example,'sqlldrscott/tigercontrol=foologfile=log' is allowed, but'sqlldrscott/tigercontrol=foolog' is not allowed, even if the position of the parameter'log' is correct.

 

------------------------------------------

Two, export spool

Enter in SQLplus or SQl command line

spool c:\test\spool.txt
  select st.employer_nm||'|'||st.salary||'|'||st.department from salary_tbl st  where LEADER_NM='雇佣者4';
spool off

You can export

此时ctl 文件该怎么写才能正确导入呢 ？

其中 options skip 是选择跳过的行数  ，  顺便使用支持中文导入的字符编码

options(skip=3)        
load data
CHARACTERSET ZHS16GBK 
infile 'spool.txt'
into table SALARY_TBL truncate
fields terminated by '|'(
  employer_nm ,
  salary	 ,
  department 
)

 

二、外部表

  外部表的数据不装入数据库中，数据库中只存储外部表的定义。实际数据位于操作系统中的平面文件中。外部表只读，可以通过select 进行查询。外部表可以由数据泵引擎生成的外部表。也可以通过文本文件生成的外部表

create  table salary_tbl_external(
   employer_nm ,
   department ,
   salary ,
   leader_nm  
)
 organization external   -----指明外部表
(
  type oracle_datapump    --利用数据泵来创建
  default directory MY_DIR    --D:\DIRTEST1
  location ('sal1.dmp','sal2.dmp')
) parallel
  as 
  select salary_tbl.employer_nm,
         salary_tbl.department,
         salary_tbl.salary,
         salary_tbl.leader_nm from salary_tbl

在MY_DIR 文件中有 SAL1.dmp 和SAL2.dmp文件。现在有了dmp文件可以通过外部表来创建外部表

create  table salary_tbl_external2(
   employer_nm varchar2(20) ,
   department  varchar2(20),
   salary number,
   leader_nm  varchar2(20)
) organization external(
  type oracle_datapump
  default directory MY_DIR    --D:\DIRTEST1
  location ('sal1.dmp','sal2.dmp')
)

利用文本文件来创建外部表  现有txt文件

create directory C_test as 'C:\test';
--使用oracle_loader创建外部表,数据文件中每一行为数据行，字段按照 ‘|'划分
create  table salary_tbl_external3(
   employer_nm varchar2(20) ,
   department  varchar2(20),
   salary number,
   leader_nm  varchar2(20)
) organization external(
  type oracle_loader
  default directory C_test    
  access parameters(
    records delimited by newline  
    fields terminated  by '|'     
  )
  location ('data1.txt')
)

select * from salary_tbl_external3

就可以查询了。

 注意： 如果在access parameters 中  注释一些没有用的代码，系统认为这是不符合规则的语句，会产生错误ORA-29913