ES&Logstash报错备案:使用logstash-input-jdbc导入数据到ES中出的错误

错误1:原因,配置文件格式问题,可能是少了空格之类的。导致配置文件无法正确读取。

检查配置文件的空格,以及驱动库的位置、sql文件位置路径是否正确。

错误2:报错。但是不影响数据导入。可以忽略。在此Mark一下。

logstash报错如下。

09:16:33.882 [[main]>worker6] ERROR logstash.outputs.elasticsearch - Retrying individual actions
09:16:33.882 [[main]>worker2] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@333b75e7 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})
09:16:33.882 [[main]>worker5] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@3cb3eeba on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})
09:16:33.902 [[main]>worker6] ERROR logstash.outputs.elasticsearch - Action
09:16:33.902 [[main]>worker3] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@1ae963b6 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})
09:16:33.912 [[main]>worker6] ERROR logstash.outputs.elasticsearch - Action
09:16:33.892 [[main]>worker4] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@4f3848a on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})
09:16:33.912 [[main]>worker6] ERROR logstash.outputs.elasticsearch - Action
09:16:33.912 [[main]>worker5] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@6ff3fb37 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})
09:16:33.902 [[main]>worker2] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@333b75e7 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})
09:16:33.922 [[main]>worker3] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@d5b137a on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})

ES报错如下

[2019-11-12T09:21:29,490][DEBUG][o.e.a.b.TransportShardBulkAction] [node-1] [gcrz3][3] failed to execute bulk item (index) index {[gcrz3][EEA_S_USER_ROLE][109876], source[{"@timestamp":"2019-11-12T01:11:00.485Z","user_id":"702996","role_id":"440001","@version":"1","id":"109876","type":"EEA_S_USER_ROLE"}]}
org.elasticsearch.cluster.metadata.ProcessClusterEventTimeoutException: failed to process cluster event (put-mapping) within 30s
        at org.elasticsearch.cluster.service.ClusterService.lambda$onTimeout$5(ClusterService.java:497) ~[elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:527) ~[elasticsearch-5.2.0.jar:5.2.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_73]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_73]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]
[2019-11-12T09:21:29,490][DEBUG][o.e.a.b.TransportShardBulkAction] [node-1] [gcrz3][2] failed to execute bulk item (index) index {[gcrz3][EEA_S_USER_ROLE][917566], source[{"@timestamp":"2019-11-12T01:11:00.482Z","user_id":"968327","role_id":"440001","@version":"1","id":"917566","type":"EEA_S_USER_ROLE"}]}
org.elasticsearch.cluster.metadata.ProcessClusterEventTimeoutException: failed to process cluster event (put-mapping) within 30s
        at org.elasticsearch.cluster.service.ClusterService.lambda$onTimeout$5(ClusterService.java:497) ~[elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:527) ~[elasticsearch-5.2.0.jar:5.2.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_73]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_73]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]

初步判断是jdk和导入Oracle的ojdbc版本不对应的问题。显示时debug调试模式。暂时不影响数据的导入。就是logstash报错看着有点难受。不过ES要求jdk8以上,Oracle11的要求ojdbc6,应该是jdk版本对应的问题。

发布了38 篇原创文章 · 获赞 20 · 访问量 2万+

猜你喜欢

转载自blog.csdn.net/weixin_40165004/article/details/103022409