optimization
es installation and configuration is very lightweight, to meet a variety of different application scenarios, the underlying data structures provide a variety of support and a lot of default configuration optimization, using part of the configuration for a specific user scenarios may be redundant and may even result in performance degradation, it is necessary to make the appropriate choice based on actual business scenarios, combined with their own usage scenarios we made the following optimization (article omissions or incorrect place also welcome reviews correction).
- Environment Configuration
sudo swapoff -a
# 禁用swapping，开启服务器虚拟内存交换功能会对es产生致命的打击
vm.max_map_count
# 在/etc/sysctl.conf文件中找到该参数，修改为655300后 执行sysctl -p，不然启动时会报值太小Common configuration in the two documents, namely elasticsearch.yml and jvm.options (configuration memory)
- jvm.options
jvm.options main memory configuration is the official recommended memory allocated to es do not exceed 50% of the system memory, set aside half to Lucene, because Lucene caches segment data to enhance the retrieval performance; memory configuration should not exceed 32g, If your server memory is not much more than 64g, it's not recommended es jvm memory is set to 32g, because after more than 32g length of each object pointer jvm will be doubled, leading to memory and cpu overhead increases.
-Xms10g
-Xmx10g
- elasticsearch.yml
Basic Configuration:
cluster.name
# 配置es集群名称，相同名称的集群会自动识别
node.name
# es7.0集群节点名称会自动获取本机hostname，如果不是多实例部署，可不配置该项
path.data
# 指定数据存放目录，多目录逗号分隔
path.logs
# 指定日志存放目录
network.host
# 指定本机ip地址
http.port
# 指定http协议端口 ，多实例部署时需要修改
transport.tcp.port
# 指定tcp协议端口，多实例部署时需要修改
cluster.initial_master_nodes: [" "]
# 指定主节点列表，需要在每个节点上配置该参数
discovery.zen.ping.unicast.hosts: []
# 广播节点
Optimization:
bootstrap.memory_lock: true
#设置为true锁住内存，当服务混合部署了多个组件及服务时，应开启此操作，允许es占用足够多的内存。
indices.breaker.request.limit: 10%
#设置单个request请求的内存熔断限制，默认是jvm堆的60%（es7.0引入了新的内存熔断机制，会智能判断，规避OOM）。
index.merge.scheduler.max_thread_count: 1
#设置segment合并时占用的线程数，配置线程数越多对磁盘io消耗就越大（SSD忽略）。
indices.queries.cache.size：20%
#query请求可使用的jvm内存限制，默认是10%。
indices.requests.cache.size：2%
#查询request请求的DSL语句缓存，被缓存的DSL语句下次请求时不会被二次解析，可提升检索性能，默认值是1%。
indices.fielddata.cache.size：30%
#设置字段缓存的最大值，默认无限制。
node.attr.box_type: hot
#用来对索引数据进行冷热分离，需要注意的是 setting 中也要进行相关配置 "index.routing.allocation.require.box_type": "hot"
Need to learn how to build es, you can watch the other articles in this blog, the source of this article technology micro-channel public number java micro technology