版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/xvshu/article/details/81984137
线上服务经过一段时间跑之后，发现es有丢消息的情况，经过排查发现，出现了一些bug，以下是bug全貌：
[8663]: index [logstash_apifile-trans-usercenter-22], type [perform_log_trans-userCenter], id [AWVfVq-CIB0x6NjELW1y], message [RemoteTransportException[[10.1.115.181-1][inet[/10.1.115.181:9301]][indices:data/write/bulk[s]]]; nested: EsRejectedExecutionException[rejected execution (queue capacity 50) on org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase$1@1fe2ac24]; ]
[8664]: index [logstash_apifile-trans-message-22], type [perform_log_trans-message], id [AWVfVq-CIB0x6NjELW1z], message [RemoteTransportException[[10.1.114.221-2][inet[/10.1.114.221:9311]][indices:data/write/bulk[s]]]; nested: EsRejectedExecutionException[rejected execution (queue capacity 50) on org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase$1@701913c5]; ]
at org.apache.flume.sink.elasticsearch.client.ElasticSearchTransportClient.execute(ElasticSearchTransportClient.java:181)
at org.apache.flume.sink.elasticsearch.ElasticSearchSink.process(ElasticSearchSink.java:207)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
at java.lang.Thread.run(Thread.java:745)经过排查，应该是es配置的批量提交线程不够，等待队列使用默认的大小（50），导致消息抛弃，解决方法也非常简单，在es配置文件末尾加入如下配置
threadpool.bulk.type: fixed
threadpool.bulk.size: 60
threadpool.bulk.queue_size: 3000具体意思就是启动一个固定大小线程池，线程树为60，等待队列最大3000条，问题解决，再跑一段时间卡看看效果！