The fields of different data sources of kettle are inconsistently merged and inserted into the database

The fields of different data sources of kettle are inconsistently merged and inserted into the database

Problem Description

When migrating data, it is necessary to add a fixed value to each row of data in the data list of a data source. This fixed value is not a constant. This fixed value needs to be obtained by querying another data source. Finally, the data list with the fixed value must be inserted into
the new table.
The detailed description is as follows :

Before the merge, the fields are completely different.

 select t.id,   t.name,t.sex       from   T t

Data source 1: T table

Data source 2: T1 table

 select   t.group_name    from   T1 t  where   group_code='x'

Now expect the transformed effect
Insert into new table T2

INSERT INTO T2(id,name,sex,group_name ) VALUE(1,'xiaozhang',1,'A组');

try to solve

You might think of this as grouping the name of the configurable variable (core object-job- setting-variable ) and using this variable when merging.

1. You may use the core object-connection- merge record in the kettle component

		这个使用前提是2张表的字段和行列要一致才行。

2. You may use the core object-connection- recordset connection for the kettle component

		这个使用前提是2张表的字段和行列要一致才行。

3. You may use the core object-conversion- increase constant in the kettle component

		这个无法获取设置变量的值 ,因为要用${group_name}。

4. You may use the core object-conversion- set field value in the kettle component

先查询这个group_name,同时设置成变量,然后插入/更新(核心对象-输出-插入/更新),就可以选这个这个变量,
没想到,插入时会找不到这个字段值。

final solution

Through a lot of previous attempts, I finally found the core object-job- get variable component

Core Object-Job- Set Variable
Core Object-Job**-Get Variable**
This get variable can get the value of ${group_name}. Basset board.

Perfect solution! ! !

Summarize

I am not very familiar with the basic components of the kettle. I hope to learn more about the use of the basic components of the kettle through project practice in the future. It took a long time to solve this problem this time. Hope to solve the problem later and improve efficiency.

I also learned the operation skills of a Kettle variable.

If you have data migration issues, welcome to private message to discuss learning together.

Guess you like

Origin blog.csdn.net/fxzzq/article/details/130432373