Kettle- dynamic data link to the multiplex JOB

Dynamic data connection, to JOB multiplexed so 

background

Mobile law enforcement system in the current main deployment strategy for the 1 + N way that deploy a corps, to deploy a city each, and basically in the private network environment. Around the city of business data systems need to be pushed to the Corps, the Corps to do the whole system of supervision and decision-making. In the course of the entire data butt, butt-based ministries and agencies around the city program Kettle ETL development, except for data link information, additional information is basically the same. Duplication of development costs are too high. Is it possible to switch the database link information to reuse the same data docking procedure?

Design ideas

Obtaining data link information is saved to the results, and then the cycle call data link information to dynamically configure data extraction procedures to achieve JOB reuse.

kettle achieve
  • Increase the dynamic data link (way to use variables to achieve dynamic access)
  • JOB execution entry
  • Reusable JOB
  1. TR_LOAD_DB:获取数据链接信息,并将数据链接信息保存的记录(内存中);
  2. JOB_LOAD_ZFSJ_ITEM:可复用的数据对接程序
  3. TR_SET_DB_PRO:将传递过来的数据链接信息保存到变量中。
  4. TR_LOAD_[数据类型]:加载指定的业务数据
  5. TR_EXCUTE_SQL:各地市数据加载成功之后的后操作(非必须)
  • ! Key setup instructions

The LOAD-DB-TR:

the JOB-the LOAD-ZFSJ the ITEM-:
for data link information may call one by one, must be configured to "perform each input line"

TR-DB the SET-PRO-:

TR-LOAD- [Data Type]:
Source data link using dynamic data link

After this configuration is complete, for the system 1 + N deployed, the need for data docking consolidation, we only need to develop a data docking program, and then configure the data link information, you can complete the data docking of N systems, multiple copies compared to the previous way, a lot of convenience, but also easy to maintain in the future.

Guess you like

Origin www.cnblogs.com/xiaotao726/p/11600949.html