KETTLE4 useful working examples --2 complex, two table data comparison, comparing the automatic synchronization (sector data synchronization units)
Second, the two tables comparing and checking data, automatically synchronized to the collation target table
Objective: Comparison t_bm t_bm_target data table and the data table to table t_bm subject, the data to t_bm_target automatic synchronization;
1, in order to give you a more intuitive display, loudspeakers Fun [database] first create two tables in the database, table structure is as follows:
- t_bm sector units table;
- t_bm_target sector units target table;
. 1 - the Create Table
2 Create Table T_BM . 3 ( . 4 organize_code VARCHAR2 ( 200 is), - unit code . 5 organize_name VARCHAR2 ( 200 is), - Name . 6 cjsj DATE - creation time . 7 ) . 8 TABLESPACE ZFQLC . 9 PCTFREE 10 10 INITRANS . 1 . 11 MAXTRANS 255 12 is Storage 13 is ( 14 Initial 64K 15 Next 1M 16 MINEXTENTS . 1 . 17 maxextents unlimited 18 );
. 1 - the Create Table
2 Create Table T_BM_TARGET . 3 ( . 4 organize_code VARCHAR2 ( 200 is), - unit code . 5 organize_name VARCHAR2 ( 200 is), - Name . 6 cjsj DATE - creation time . 7 ) . 8 TABLESPACE ZFQLC . 9 PCTFREE 10 10 INITRANS . 1 . 11 MAXTRANS 255 12 is Storage 13 is ( 14 Initial 64K 15 Next 1M 16 MINEXTENTS . 1 . 17 maxextents unlimited 18 );
Wherein t_bm (unit table) data as shown below:
t_bm_target (target table units) data as shown below:
2, the entire job process:
Need to use a conversion operation operates and 4 2 tables.
The following figure shows the entire work process:
The above is a flowchart of the entire process of operation, it uses three converters and two JS script to achieve this requirement.
3, access to agency information data
Data acquisition source below 3.1, sql statement must be sorted by means Code
Data acquisition source below 3.2, sql statement must be sorted by means Code
3.3 Steps 3.1 and 3.2 of the hops through wires connected to the connection control recordset
Selecting steps 1 and 2, the connection type left outer, the step of creating a data source table connection basis, connected organize_code field selection field.
3.4 Setting filter conditions, if organize_code is empty, then do nothing, not empty words, into the data stream
3.5 select field, the field will not be placed in the selected empty data, and copy the record to the result, for the next conversion step using
3.6 JavaScript script validation, data acquisition and set into a variable
Field test values of 3.7, obtaining the data set into a variable and
3.8 accumulation counter, obtaining the variable i, and the result is placed in detail in detail
3.9 synchronous processing flow mechanism
3.9.1 Sync mechanism - data acquisition, data on the detail of the step into variable
3.9.2 sql script insertion t_bm_target table
Note: Due to certain restrictions, the latest source code and the subsequent general configuration of Data Extraction author has been placed on the public number, please pay attention to micro-channel public number: big horn school database, reply Keywords: [instance], to obtain kettle installer and run examples (table structure and kjb, ktr file) .
