API series of ES -- usage of dynamic template (dynamic template) (with examples)

Original website: API series of ES -- usage of dynamic template (dynamic template) (with examples)

Introduction

illustrate

        This article introduces the usage of ElasticSearch's dynamic template (dynamic template) (with examples).

Official website URL

https://www.elastic.co/guide/en/elasticsearch/reference/8.0/dynamic-templates.html

The role of dynamic templates

Function overview

        The dynamic template is set for a specific index, that is, specified when creating a specific index.

scenes to be used

According to the feature of ES automatically identifying field types, set the names of some special fields to certain types. For example:

• Set all fields of string type to keyword
• Strings starting with is are all set to boolean type

match-mapping-type

overview

        When _mapping is not set, ES performs dynamic mapping according to the type of document field. In this way, the ES data type is dynamically defined for the JSON data type.

JSON data type	ES data type
null	do not add field
true / false	boolean
double	float
long	long
object	object
array	According to the type of the first non-null value in the array
String to pass date detection	date
String to pass number detection	float or long
Strings that did not pass the above 2 tests	text type with .keyword subfield

example

PUT /es-learn-doc-person/_create/1
{
    "name":"zhang san",
    "age":29
}

Mapping under dynamic mapping:

{
  "es-learn-doc-person" : {
    "mappings" : {
      "properties" : {
        "age" : {
          "type" : "long"
        },
        "name" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    }
  }
}

        In this document, age is automatically mapped to long, but the integer type fully satisfies the storage of current data. Assuming such a scenario, for the integer value part of the company's data, most integers can meet the storage requirements, then we can use Dynamic Template to dynamically map. the way is:

PUT es-learn-000002
{
  "mappings": {
    "dynamic_templates": [
      {
        "integers": {
          "match_mapping_type": "long",
          "mapping": {
            "type": "integer"
          }
        }
      }
    ]
  }
}

For the dynamic template used by index es-learn-000002, we insert the same user information. 

PUT es-learn-000002/_doc/1
{
    "name":"zhang san",
    "age":29
}

View the corresponding mapping

match与unmatch

overview

        match and unmatch is to match the field name.

        For nested objects, match and unmatch only work on the last level of field names.

• match: the field name matches a rule
• unmatch : the field name does not match a rule
• match_pattern: The pattern used for matching. If set to regex, match and unmatch support regular expressions. For example:

  "match_pattern": "regex",
  "match": "^profit_\d+$"

example

need

• Set all fields of string type to keyword
• Strings starting with is are all set to boolean type

method

PUT my_index
{
    "mappings": {
        "dynamic_templates": [{
                "strings_as_boolean": {
                    "match_mapping_type": "string",
                    "match": "is*",
                    "mapping": {
                        "type": "boolean"
                    }
                }
            }, {
                "strings_as_keywords": {
                    "match_mapping_type": "string",
                    "mapping": {
                        "type": "keyword"
                    }
                }
            }
        ]
    }
}

path_match 和 path_unmatch

overview

        match and unmatch only work on the field name of the last level. For a multi-layer embedded object, path_match and path_unmatch can be used. (ES supports storing objects), for example: some_object.*.some_field.

example

If the value is an object type, it starts with name and does not end with middle, then copy it to the full_name field.

Set mapping:

PUT my_index
{
  "mappings": {
    "dynamic_templates": [
      {
        "full_name": {
          "path_match":   "name.*",
          "path_unmatch": "*.middle",
          "mapping": {
            "type":       "text",
            "copy_to":    "full_name"
          }
        }
      }
    ]
  }
}

insert data

PUT my_index/_doc/1
{
  "name": {
    "first":  "John",
    "middle": "Winston",
    "last":   "Lennon"
  }
}

view data

GET my_index/_search?q=full_name:John

Combination of index templates and dynamic templates

I haven't tried it here, but I guess that index templates and dynamic templates can be used at the same time, for example:

PUT  _index_template/event_log_template
{
    "index_patterns": ["event_log_*"],
    "template": {
        "settings": {
            "number_of_shards": 5,
            "number_of_shards": 1
        },
        "mappings": {
            "dynamic_templates": [{
                    "dynamicFields": {
                        "match_mapping_type": "string",
                        "path_match": "dynamicFields.*_sku_attr",
                        "mapping": {
                            "type": "keyword"
                        }
                    }
                }
            ],
            "properties": {
                "id": {
                    "type": "keyword"
                },
                "category_first_id": {
                    "type": "keyword"
                },
                "category_first": {
                    "type": "keyword"
                }
            }
        }
    }

}