Rollover scroll using aliases and create index index

In ElasticSearch6.3.2 cluster nodes do cold (warm) hot (hot) separation , the realization of the separation of hot and cold ElasticSearch cluster node, the newly created index must be assigned only to the hot nodes, and over time, the old historical index data need to migrate to the warm node. Therefore, the data stored on the ES index is generally divided in time: for example, to automatically generate a monthly index, it is used to store all the data produced by this month. In order to more easily manage the time generated by the index, the index can be used in conjunction with the template function ES Rollover to easily manage index.

Create a template index, which index the index template is not defined alias for the rollover. Because the index template inside the index will generally point to multiple alias "physical" index. Specific Reference: rollOver-the failing . Directed to more "physical" index of index aliases can not be used to write data.

PUT _template/pubchat
{
  "index_patterns": "pubchat-*",
  "settings": {
    "index": {
      "number_of_shards": "3",
      "number_of_replicas": "1",
      "routing": {
        "allocation": {
          "exclude": {
            "box_type": "warm"
          },
          "require": {
            "box_type": "hot"
          }
        }
      }
    }
  },
  "mappings": {
    "_doc": {
      "_source": {
        "enabled": true
      },
      "properties": {
        "uid": {
          "type": "keyword"
        },
        "nick": {
          "type": "keyword"
        }，
        "chatTime": {
          "type": "date",
          "format": "yyyy-MM-dd HH:mm:ss"
        }
      }
    }
  },
  "aliases": {
    "pubchat-search-alias": {}
  }
}

Alias index pubchat-search-alias used primarily for search (read operation). To distinguish between type-2 indexes alias: one is the write index, the other is read index, specifically refer to ES Index Alias official documents.

Create a new index, the automatic index hit a template, and create a special alias is used to search the index pubchat-202001: pubchat-search-alias

#创建一个索引
PUT pubchat-202001

Specifically designated a rollover (scroll written) of index aliases. Alias index pubchat-write-alias is used to write data

#为索引再指定一个只用来 rollover 的别名
POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "pubchat-202001",
        "alias": "pubchat-write-alias"
      }
    }
  ]
}

specify the time when the rollover alias pubchat-write-alias only need to create the first index , behind pubchat-write-alias automatically scrolls point: pubchat-202002, pubchat-202003 ......

The main role of rollover index is the alias of "rolling writes," so rollover index aliases can only point to a specific index.

An alias that points to one and only one index can be used to read and write data. An alias that points to more than one index is read-only.

A detailed description of the index aliases can refer to the official documentation: the Write Index

Specify the scroll policy for the index. Here To facilitate testing, when an index of more than two documents, you create a new index.

# 指定rollover 的策略
POST /pubchat-write-alias/_rollover 
{
  "conditions": {
    "max_docs":  2
  }
}

Because the new index is created, there is no data, and therefore should be returned as follows:

{
  "acknowledged": false,
  "shards_acknowledged": false,
  "old_index": ""pubchat-202001",
  "new_index": "pubchat-202002",
  "rolled_over": false,
  "dry_run": false,
  "conditions": {
    "[max_docs: 2]": false
  }
}

Finally, only the alias pubchat-write-alias to write data to the rollover index, when specified to meet the rollover policy, the next new index will be created automatically.

# 写一点测试数据进去
PUT pubchat-write-alias/_doc/1
{
    "uid" : "111",
    "nick" : "test"
}

It is worth noting: index.refresh_interval parameters affect the accuracy of the scroll policy. 2 is arranged such max_docs, by refresh_interval influence, the number of documents contained in the index may be greater than that of two.

An example illustrates:

In order to generate a month index (pubchat-yyyyMM), such as: pubchat-202001, pubchat-202002, pubchat-202003 ......

After the introduction of the Rollover, just write to write data on pubchat-write-alias index (alias), when data is written to the specified trigger rollover strategy, will automatically generate a new index (go to match the index has been defined template )

restHighLevelClient.bulkWrite("pubchat-write-alias");

Rollover if employed, it needs its own code logic to write data, generates an index name is determined monthly.

restHighLevelClient.bulkWrite("pubchat-202001");//1月份
restHighLevelClient.bulkWrite("pubchat-202002");//2月份
....

Obviously rollover simplifies the management of the index. Another, ES6.7 version introduces Index LifeCycle Management, should be able to more easily manage the index of the bar.

Original: https://www.cnblogs.com/hapjin/p/11386965.html