Instructions for the use of kettle universal plugin [kettlePlugins]

1. Download the plug-in source code

Project address: https://gitee.com/nivalsoul/kettlePlugins

Download via git or directly download the zip file.

2. Environmental preparation

2.1 Install maven, jdk8

Make sure maven and jdk (preferably 1.8) are installed, and environment variables are set, the process is slightly...

2.2 Install the kettle jar package to the local maven repository

1. First download the release package of kettle6 or 7, and extract it to the specified directory
2. Use the following command to install the jar (locate to kettle/lib or modify the path of the jar package in the script, or copy the following jar to a custom directory) After execution, swt-6.1.0.1-196.jar is the swt.jar in the corresponding Windows platform file in the libswt directory. In order to unify the need to copy and rename it, the following takes the 6.1.0.1-196 version as an example):

mvn install:install-file -Dfile=./kettle-core-6.1.0.1-196.jar -DgroupId=org.pentaho.di -DartifactId=kettle-core -Dversion=6.1.0.1-196 -Dpackaging=jar
mvn install:install-file -Dfile=./kettle-dbdialog-6.1.0.1-196.jar -DgroupId=org.pentaho.di -DartifactId=kettle-dbdialog -Dversion=6.1.0.1-196 -Dpackaging=jar
mvn install:install-file -Dfile=./kettle-engine-6.1.0.1-196.jar -DgroupId=org.pentaho.di -DartifactId=kettle-engine -Dversion=6.1.0.1-196 -Dpackaging=jar
mvn install:install-file -Dfile=./kettle-ui-swt-6.1.0.1-196.jar -DgroupId=org.pentaho.di -DartifactId=kettle-ui-swt -Dversion=6.1.0.1-196 -Dpackaging=jar
mvn install:install-file -Dfile=./pentaho-metadata-6.1.0.1-196.jar -DgroupId=org.pentaho.di -DartifactId=pentaho-metadata -Dversion=6.1.0.1-196 -Dpackaging=jar
mvn install:install-file -Dfile=./metastore-6.1.0.1-196.jar -DgroupId=org.pentaho.di -DartifactId=metastore -Dversion=6.1.0.1-196 -Dpackaging=jar
mvn install:install-file -Dfile=./swt-6.1.0.1-196.jar -DgroupId=org.pentaho.di -DartifactId=swt -Dversion=6.1.0.1-196 -Dpackaging=jar

3. Development process

1. Modify the kettle version <kettle.version> number in the pom (the same as the one specified by mvn install above) and the kettle installation directory <kettle.home>
2. Use Eclipse or IDEA to import the project
3. After compiling directly (mvn clean package) , start the kettle to check if it is successful: under the converted plugin category CommonPlugin :

4. The implemented plugin demos include [custom input type] (read data stream from rest and read data stream from csv through sax) and [output to json file] (support output as array or object with specified field name) Format):

4. Custom plug-in development

1. First create a class to inherit nivalsoul.kettle.plugins.common.CommonStepRunBase
2. Rewrite the disposeRow() method to process each row of data. This part can refer to the implementation of existing plugins.
3. If necessary, it can be rewritten
The init() and end() methods do some initialization and cleaning operations

• The poi dependency version used is 3.17 and httpmine is 4.5.1. If you need to use custom input excel/rest input, you need to put it under kettle/lib
• In addition, fastjson and guava are also used. If not, they need to be placed under kettle/lib.

5. Restart the kettle, and drag the plug-in under the converted plug-in category CommonPlugin to the conversion diagram flow chart. The configuration instructions are as follows:

• Plug-in type directly select "custom" (if you want to specify a unique name, you can add the corresponding name to the nivalsoul.kettle.plugins.common.PluginType enumeration)
• Fill in the full path of the above self-created class for the custom class name
• The format of the configuration part is json, and the content is the configuration field information used in the custom plug-in (if you add your own unique plug-in name in the plug-in type, you can also add the same name as the plug-in enumeration name under resource. Configure the json file so that the default configuration can be automatically loaded when the plugin category is selected)

Note: This section can refer to the implemented plugin.

5. Others

If you encounter any problems in the development, you can add QQ group (195548102) to consult.