kettle7.0写入elasticsearch6.3.2插件
之前写了5.x的插件,但是6.x的api改动也较大,主要介绍改动部分,源码更改编译见前篇。
客户端获取,ElasticSearchBulk中
private TransportClient client;
private void initClient() throws UnknownHostException {
Settings.Builder settingsBuilder = Settings.builder();
settingsBuilder.put( Settings.Builder.EMPTY_SETTINGS ); // keep default classloader
//---------------changed---------------//
Map<String, String> kv = meta.getSettingsMap();
//不支持直接put map,遍历出来,再put
for (Map.Entry<String, String> entry : kv.entrySet()) {
settingsBuilder.put( entry.getKey(),entry.getValue() );
}
Settings settings = settingsBuilder.build();
// 注意client为TransportClient;没有node,删掉所有node的获取与关闭
client = new PreBuiltTransportClient(settings);
if ( !meta.servers.isEmpty() ) {
for ( ElasticSearchBulkMeta.Server s : meta.servers ) {
client.addTransportAddress( s.getAddr() );
}
}
}
索引与集群的管理,ElasticSearchBulkDialog中
AdminClient admin = client.admin();
switch ( testType ) {
case INDEX:
if ( StringUtils.isBlank( tempMeta.getIndex() ) ) {
showError( BaseMessages.getString( PKG, "ElasticSearchBulk.Error.NoIndex" ) );
break;
}
// First check to see if the index exists
IndicesExistsRequestBuilder indicesExistBld = admin.indices().prepareExists( tempMeta.getIndex() );
IndicesExistsResponse indicesExistResponse = indicesExistBld.execute().get();
if ( !indicesExistResponse.isExists() ) {
showError( BaseMessages.getString( PKG, "ElasticSearchBulkDialog.Error.NoIndex" ) );
return;
}
RecoveryRequestBuilder indicesBld = admin.indices().prepareRecoveries( tempMeta.getIndex() );
//---------------changed---------------//
ActionFuture<RecoveryResponse> execute = indicesBld.execute();
// 分片数量的获取
String shards = "" + execute.get().getSuccessfulShards() + "/" + execute.get().getTotalShards();
showMessage( BaseMessages.getString( PKG, "ElasticSearchBulkDialog.TestIndex.TestOK", shards ) );
break;
case CLUSTER:
//---------------changed---------------//
ClusterHealthResponse healths = admin.cluster().prepareHealth().get();
//集群名称与节点数的获取
String clusterName = healths.getClusterName();
int numberOfNodes = healths.getNumberOfNodes();
showMessage( BaseMessages.getString( PKG, "ElasticSearchBulkDialog.TestCluster.TestOK", clusterName, numberOfNodes ) );
break;
default:
break;
}
源码及插件见:链接: https://pan.baidu.com/s/1yEjLaWJZLcMElGtKYKXtug 密码: pb8j
不指定id字段时会报错:an id must be provided if version type or value are set
解决方法见:https://blog.csdn.net/ukakasu/article/details/82422700