Kettle学习（3）：JAVA调用Kettle脚本使用JNDI配置文件

其他 2020-06-23 10:17:42 阅读次数: 0

背景

工作上遇到了一个前同事写的bug，关于Kettle这个文件的调用的问题，之前的同事可能也是着急实现这个功能，大体就是迁移核心的数据到ncpi系统的这一过程写的ktr的转换脚本是手动配置数据库，而且好几个ktr脚本这么配置起来就比较麻烦，再加上，如果换成生产环境的情况，就要把数据库的配置在重新配置一遍，但是我在本地配置了JNDI的连接方式，可是要怎么使用这个JNDI的配置就是一个问题了，毕竟服务器里面使用java调用Kettle的脚本是可以实现的，但是配置这一块，如果你用编辑软件打开就会发现这样通过JNDI的配置的方式是看不到数据库的连接方式的，如下图所示

通过普通的配置数据库的方式通过配置JDNI的方式连接的数据库
这样的两张图片可以对比的来说，如果你使用的是配置数据库地址，端口的那种方式，而不是配置文件JDNI的方式去写的ktr脚本，那么等你把脚本放大服务器上的时候，你要每一个ktr的文件都要进行修改，很麻烦，但是如果以配置文件的方式使用的话，就可以变得很简单，只要把数据库的配置改了就可以了，并且还可以解决mysql对应java的时间的问题，这个以后再说

虽然配置数据的方式是好用的，但是根据上图也能看出来，JNDI的方式你是看不到配置数据库的基本信息的，也就是说，这个配置文件如果你不上传到服务器上的话，这个脚本是会跑失败的，所以怎么让服务器调用这个脚本文件的配置文件就成了问题

解决问题流程思考

我先卖个关子，先看一下Kettle脚本在正常调用的时候是怎么个流程的，看看源码
JAVA调用Kettle代码的初始化代码
点击进入这个代码，发现这里是默认是true的
继续，发现如果是true的话，是直接进入到这个初始化里面
仔细看这段代码，发现在这个位置是配置了一个jndi的路径的，如果是空的话，就会默认的找系统下面的simple-jndi的文件夹

所以看到这里我有一个大胆的想法，就是在初始化之前，就把这个path给改了，改成我们放在服务器地址jndi配置，我们直接就把Kettle里面的文件夹，给拖到我们服务器的地址试试
正常Kettle配置数据库文件地址的地方
然后我把代码改了，改成如下所示了

    @Test
    public void test_path() throws KettleException {
        //初始化之前配置一下jndi的实际位置路径
        File jndiFile = FileUtil.file("C://Users//Y//Desktop//simple-jndi");
        String jndiFilePath = jndiFile.getPath();
        Const.JNDI_DIRECTORY = jndiFilePath;
        
        //这是我本地进行测试的，所以我把文件放到了桌面上了，写了根目录
        File ktrFile = FileUtil.file("C://Users//Y//Desktop//test2.ktr");
        String ktrFilePath = ktrFile.getPath();
        //初始化
        KettleEnvironment.init();
        //加载文件
        TransMeta transMeta = new TransMeta(ktrFilePath);
        Trans trans = new Trans(transMeta);
        //放入参数，这里其实可以从数据库中取到
        //如果没有参数，可以把这步忽略
        trans.setParameterValue("stade", "2019-04-24");
        trans.prepareExecution(null);
        trans.startThreads();
        //等待执行完毕
        trans.waitUntilFinished();
    }

但是当我跑起来的时候报了错误，告诉我找不到这个配置项

2019/05/25 20:56:36 - localhost_mysql_jndi - ERROR (version 7.0.0.0-25, build 1 from 2016-11-05 15.35.36 by buildguy) : Unable to find datasource by JNDI name: localhost_mysql_jndi
2019/05/25 20:56:36 - localhost_mysql_jndi - ERROR (version 7.0.0.0-25, build 1 from 2016-11-05 15.35.36 by buildguy) : Unable to find datasource by JNDI name: localhost_mysql_jndi
2019/05/25 20:56:36 - localhost_mysql_jndi - ERROR (version 7.0.0.0-25, build 1 from 2016-11-05 15.35.36 by buildguy) : org.pentaho.di.core.database.DataSourceNamingException: 
2019/05/25 20:56:36 - localhost_mysql_jndi - javax.naming.NoInitialContextException: Cannot instantiate class: org.osjava.sj.SimpleContextFactory [Root exception is java.lang.ClassNotFoundException: org.osjava.sj.SimpleContextFactory]
2019/05/25 20:56:36 - localhost_mysql_jndi - Cannot instantiate class: org.osjava.sj.SimpleContextFactory
2019/05/25 20:56:36 - localhost_mysql_jndi - 
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.core.database.util.DatabaseUtil.getNamedDataSource(DatabaseUtil.java:180)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.core.database.util.DatabaseUtil.getNamedDataSource(DatabaseUtil.java:193)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.core.database.Database.normalConnect(Database.java:431)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.core.database.Database.connect(Database.java:370)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.core.database.Database.connect(Database.java:341)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.trans.steps.tableoutput.TableOutput.init(TableOutput.java:529)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.trans.step.StepInitThread.run(StepInitThread.java:69)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at java.lang.Thread.run(Thread.java:748)
2019/05/25 20:56:36 - localhost_mysql_jndi - Caused by: javax.naming.NoInitialContextException: Cannot instantiate class: org.osjava.sj.SimpleContextFactory [Root exception is java.lang.ClassNotFoundException: org.osjava.sj.SimpleContextFactory]
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:674)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:313)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at javax.naming.InitialContext.init(InitialContext.java:244)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at javax.naming.InitialContext.<init>(InitialContext.java:192)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.core.database.util.DatabaseUtil.getNamedDataSource(DatabaseUtil.java:178)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	... 7 more
2019/05/25 20:56:36 - localhost_mysql_jndi - Caused by: java.lang.ClassNotFoundException: org.osjava.sj.SimpleContextFactory
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at java.lang.Class.forName0(Native Method)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at java.lang.Class.forName(Class.java:348)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at com.sun.naming.internal.VersionHelper12.loadClass(VersionHelper12.java:72)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at com.sun.naming.internal.VersionHelper12.loadClass(VersionHelper12.java:61)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:672)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	... 11 more
2019/05/25 20:56:36 - localhost_mysql_jndi - ERROR (version 7.0.0.0-25, build 1 from 2016-11-05 15.35.36 by buildguy) : org.pentaho.di.core.database.DataSourceNamingException: 
2019/05/25 20:56:36 - localhost_mysql_jndi - javax.naming.NoInitialContextException: Cannot instantiate class: org.osjava.sj.SimpleContextFactory [Root exception is java.lang.ClassNotFoundException: org.osjava.sj.SimpleContextFactory]
2019/05/25 20:56:36 - localhost_mysql_jndi - Cannot instantiate class: org.osjava.sj.SimpleContextFactory
2019/05/25 20:56:36 - localhost_mysql_jndi - 
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.core.database.util.DatabaseUtil.getNamedDataSource(DatabaseUtil.java:180)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.core.database.util.DatabaseUtil.getNamedDataSource(DatabaseUtil.java:193)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.core.database.Database.normalConnect(Database.java:431)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.core.database.Database.connect(Database.java:370)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.core.database.Database.connect(Database.java:341)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.trans.steps.tableinput.TableInput.init(TableInput.java:342)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.trans.step.StepInitThread.run(StepInitThread.java:69)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at java.lang.Thread.run(Thread.java:748)
2019/05/25 20:56:36 - localhost_mysql_jndi - Caused by: javax.naming.NoInitialContextException: Cannot instantiate class: org.osjava.sj.SimpleContextFactory [Root exception is java.lang.ClassNotFoundException: org.osjava.sj.SimpleContextFactory]
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:674)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:313)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at javax.naming.InitialContext.init(InitialContext.java:244)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at javax.naming.InitialContext.<init>(InitialContext.java:192)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at org.pentaho.di.core.database.util.DatabaseUtil.getNamedDataSource(DatabaseUtil.java:178)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	... 7 more
2019/05/25 20:56:36 - localhost_mysql_jndi - Caused by: java.lang.ClassNotFoundException: org.osjava.sj.SimpleContextFactory
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at java.lang.Class.forName0(Native Method)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at java.lang.Class.forName(Class.java:348)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at com.sun.naming.internal.VersionHelper12.loadClass(VersionHelper12.java:72)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at com.sun.naming.internal.VersionHelper12.loadClass(VersionHelper12.java:61)
2019/05/25 20:56:36 - localhost_mysql_jndi - 	at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:672)

解决这个问题的方法就是，找到Kettle的源码里面有一个jar包是没有安装的
使用jndi配置的时候需要导入这个jar包，所以会需要使用mvn的命令安装到本地
mvn安装命令：

mvn install:install-file 
-Dfile=D:\1\simple-jndi-1.0.0.jar
-DgroupId=pentaho-kettle 
-DartifactId=simple-jndi 
-Dversion=1.0.0
-Dpackaging=jar

然我们在pom文件里面引入这个依赖

        <dependency>
            <groupId>pentaho-kettle</groupId>
            <artifactId>simple-jndi</artifactId>
            <version>1.0.0</version>
        </dependency>

然后我们在试试这个类跑起来，，看日志输出


2019-05-25 21:02:26.565  INFO 10620 --- [           main] com.example.demo.MyTest                  : Starting MyTest on DESKTOP-IJQDUR1 with PID 10620 (started by Y in D:\03-work\02_test_work\webservicetest)
2019-05-25 21:02:26.569  INFO 10620 --- [           main] com.example.demo.MyTest                  : No active profile set, falling back to default profiles: default
2019-05-25 21:02:28.417  INFO 10620 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.ws.config.annotation.DelegatingWsConfiguration' of type [org.springframework.ws.config.annotation.DelegatingWsConfiguration$$EnhancerBySpringCGLIB$$e914c28f] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2019-05-25 21:02:28.512  INFO 10620 --- [           main] .w.s.a.s.AnnotationActionEndpointMapping : Supporting [WS-Addressing August 2004, WS-Addressing 1.0]
2019-05-25 21:02:29.620  INFO 10620 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2019-05-25 21:02:30.306  INFO 10620 --- [           main] com.example.demo.MyTest                  : Started MyTest in 4.275 seconds (JVM running for 6.263)
2019-05-25 21:02:31.663  INFO 10620 --- [           main] o.a.c.v.impl.StandardFileSystemManager   : Using "C:\Users\Y\AppData\Local\Temp\vfs_cache" as temporary files store.
2019/05/25 21:02:32 - test2 - 为了转换解除补丁开始  [test2]
Sat May 25 21:02:32 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Sat May 25 21:02:32 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2019/05/25 21:02:34 - 表输出.0 - Connected to database [localhost_mysql_jndi] (commit=1000)
2019/05/25 21:02:34 - 表输入.0 - Finished reading query, closing connection.
2019/05/25 21:02:34 - 表输入.0 - 完成处理 (I=5, O=0, R=0, W=5, U=0, E=0)
2019/05/25 21:02:34 - 表输出.0 - 完成处理 (I=0, O=5, R=5, W=5, U=0, E=0)
2019-05-25 21:02:34.371  INFO 10620 --- [       Thread-2] o.s.s.concurrent.ThreadPoolTaskExecutor  : Shutting down ExecutorService 'applicationTaskExecutor'

可以正常的运行起来了

总结

通过这次的学习，踩坑，发现通过这样的配置方式可以很好的解决多次配置Kettle脚本数据库的问题，终于摆脱傻逼的代码了

猜你喜欢

转载自blog.csdn.net/lovePaul77/article/details/90548114
今日推荐
Apache Doris 2.0.10 版本正式发布！
开源日报 | 大模型开战；大模型独角兽被曝卖身；周鸿祎建议谷歌开源所有产品；最大开源AI社区提供1000万美元共享GPU
开源日报 | Chrome内置Gemini的意义不在于Gemini；中国AI追随之路的五大误区；ECharts创始人“下海”养鱼；谷歌I/O开发者大会什么都有，只是没有惊喜
微软回应中国区AI团队“打包赴美”传闻
基于大语言模型的开源知识库问答系统 MaxKB GitHub Star 数量突破 5,000 个！
美国拟限制 AI 大模型出口中国和俄罗斯
苹果将与 OpenAI 达成协议，将 ChatGPT 应用于 iPhone
周排行
阿里云短信服务平台注册
Windows下的字符串处理(1)
sqoop: mysql导入数据到hdfs, hive, hbase
commons.lang中常用的工具类
离线安装PostgreSQL11.6
使用PyTorch简单实现卷积神经网络模型
一文彻底搞定谱聚类
一道面试题引发的血案
One Chat for Mac(聊天工具)
TCP/IP的底层队列是如何实现的？
每日归档
更多
2024-05-17(34)
2024-05-16(6)
2024-05-15(24)
2024-05-14(0)
2024-05-13(18)
2024-05-12(0)
2024-05-11(38)
2024-05-10(38)
2024-05-09(35)
2024-05-08(42)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022