ElasticSearch basis using CRUD, DSL and document mapping

Four .ElasticSearch basis

4.1. The index database CRUD

4.1.1 The increase in the index Library

PUT shopping
{
	"settings":{
		"number_of_shards":5,
		"number_of_replicas":1
	}
}

4.1.2 query index Library

GET _cat/indices?v

Check the specified index Library

GET _cat/indices/aigou

[Image dump the chain fails, the source station may have security chain mechanism, it is recommended to save the picture down uploaded directly (img-NfNnHdbX-1581164132995) (images / 1573717333201.png)]

4.1.3. Delete the index Library

DELETE 名字

4.1.4. Modify the index Library

删除再添加

4.2. CRUD document

4.2.1. Documentation concept

IT IS	Mysql
_index (index Library)	database
_type (document type)	table
_document (document object)	A row of data
_id (document ID)	Primary Key ID
field (field)	Row

other

_source	(Document raw data - is the real data stored in the ES)
_all	(All fields in the connection string)
metadata	(Document metadata - document out real accident data to other data)

4.2.2. Adding documents

grammar

PUT 索引库/文档类型/文档id
{
	文档原始数据
}

Case:

PUT crm/user/11	
{
	"id":11,
	"username":"zs"
}

Explanation: Adding a user id to 11, the index library crm, type User

Note: If you do not specify a document's id, ES will automatically generate a document id

4.2.3. Obtaining Documentation

a. Gets the specified document

GET 索引库/类型/文档ID 

b. Specify columns returned

GET /itsource/employee/123?_source=fullName,email

4.2.4. Modify the document

a. Overall modification

PUT {index}/{type}/{id}
{
	修改后的内容
}

ps: As with the full amount of the modified syntax to add document syntax, if the document is to add already exists, or is modified,

Document modification process: 1. Delete the old mark documents, add 2 new documents.

b. partial modification

POST itsource/employee/123/_update
{
    "doc":{
        "email" : "[email protected]", 
        "salary": 1000
    }
}

ps: document modification process: 1. retrieve old documents, 2. modify the document, delete the old document marked 3, 4. Add a new document.

4.2.5. Delete Document

DELETE {index}/{type}/{id}

V. query documents

5.1. Common Queries

5.1.1. All inquiries

GET _search

5.1.2. Query specified index Library

GET crm/_search

5.1.3. Specify the type of query

GET crm/user/_search

5.1.4. Query specified document

GET crm/user/11

5.1.5. Paging query

&size=2&from=2

• size: the number of page
• form: How many pieces of data from the beginning of the investigation

5.1.6. Query String

+ + Sorting criteria query pagination

GET crm/user/_search?q=age:17&size=2&from=2&sort=id:desc&_source=id,username

Query string (query string) is actually way behind the url string splicing a variety of search criteria, this approach is not recommended, because too many conditions, stitching up more trouble

5.1.7. Batch query

Batch query is very important, compared to a single query, the higher-volume query performance.

a. Different index database queries

GET _mget
{
    "docs" : [
        {
            "_index" : "itsource",
            "_type" : "blog",
            "_id" : 2
        },
        {
            "_index" : "itsource",
            "_type" : "employee",
            "_id" : 1,
            "_source": "email,age"
        }
    ]
}

b with the same type of database index - Recommended

GET itsource/blog/_mget
{
	"ids" : [ "2", "1" ]
}

Six .DSL DSL filter query

6.1. Basic Concepts

6.1.1. Basic Concepts

DSL DSL filter statements and queries are very similar, but their purpose is different: DSL filter query document for the way I like conditional "yes" or "no" (equal; not equal to), but the DSL query such as "how like" (fuzzy search).

6.1.2. Difference between a query and filtering

DSL DSL filter and query the difference in performance:

• The results can be cached by filtration and applied to subsequent requests. -> take accurate results filtered fuzzy query performance high
• The documents that match the query, the correlation is calculated, it is more time-consuming, and does not cache.
• Filter statement can effectively meet the query is completed document filtering.

Summary: The need to use fuzzy queries DSL query, use DSL requires precise query filter, in combination used in the development (combination search) -> query keyword query using DSL, others are filtered with DSL.

6.2. Integrated Case

6.2.1. Syntax Introduction

GET crm/user/_search
{
    "query": {
       "bool": { 
              "must": [
                	{"match": {"description": "search" }}
               ],
               "filter": {
                   "term": {"tags": "lucene"}
               }
        }
    },
    "from": 20, 
    "size": 10,
    "_source": ["fullName", "age", "email"],
    "sort": [{"join_date": "desc"},{"age": "asc"}]
}

Explanation:

query: Query

bool: a combination of inquiry, including DSL and DSL filter query

MUST: must match: and (MUST) or (should) Non (must_not)

match: word match the query, the query would word, multi_match: multi-field match

filter: filter conditions

term: Yuan word query, the query is not word for

from, size: Paging

_source: which columns the query results need

sort: Sorting

6.2.2 Integrated Case

Bool can combine multiple combinations of search query is a query object name (name) in the "baby" of commodity prices (minPrice) between 10000-20000, brand id is 14, according to sales (saleCount) Reverse Order, queries page 1, page 10, the query results only: id, name, minPrice, brandId

GET aigou/product/_search 
{
	"query":{
		"bool": {
			"must": [{
				"match": {
					"name": "春季"
				}
			}],
			
			"filter": [
				{
                    "range":{	//范围查询
                        "minPrice":{
                            "gte":10000,
                            "lte":20000
                        }
                    }
				},
				{
					"term": {	//词元查询
						"brandId": 14
					}
				}
			]
		}
	},
	"from": 1,
	"size": 10,
	"_source": ["id", "name", "minPrice","brandId"],
	"sort": [{
		"saleCount": "desc"
	}]
}

ps: other ways to search

• term: the word / term yuan inquiry
• match: word queries (fuzzy match)
• multi_match: multi-field match
• range: range queries
• exists: there is
• missing: there is no
• prefix: prefix match
• wildcard: wildcard matching, * for 0 ~ N number, use on behalf of a?.

VII. Mapping document

7.1. Basic Concepts

7.1.1. What is a document mapping

ES document mapping (Mapping) mechanisms for acknowledgment field type, matching each field of the data for determining a type.

• Mysql built Table process: Create Database -> Create a table (the specified field type) -> crud data
• ES index creation: Create an index Library -> Document type mapping -> crud Documents
• Mapping the word: In order to facilitate the search field, and I it would be word some is stored in the ES in the field, but some field types can be divided into words, some field types can not be sub-word, so for the type of field we need themselves to be specified.

7.1.2. Default field type

See index type mapping configuration: GET {indexName} / _ mapping / {typeName}

a. The basic field type

• String

  text (word), keyword (regardless of the word), StringField (regardless word text), TextFiled (word to text), text defaults to full text, keyword default for non-full text

• 数字(long,integer,short,double,float)

• Date date

• Boolean logic

b. Complex Field Type

• Object type: object
• Array Type: array
• Location: geo_point, geo_shape

c. default mapping

In the absence of new ES Configuration Mapping document, ES will try to guess the field type and dynamically generate a mapping between fields and types.

JSON type	Field type
Boolean: true or false	“boolean”
Whole number: 123	“long”
Floating point: 123.45	“double”
String, valid date:“2014-09-15”	“date”
String: “foo bar”	“string”

7.1.3. Mapping Rules

Property configuration list of commonly used field mapping - that can specify the following attributes to perform a field of class time

Property name	Explanation
type	Type field: basic data types, integer, long, date, boolean, keyword, text ...
enable	Is enabled: The default is true. false: not index, not a search filter.
boost	Weight lifting Multiples: weighted to calculate the final score when used in a query, such as title search to search for content than content with the introduction to the important, you can upgrade
index	Index mode: analyzed (index and segmentation, text default mode), not_analyzed (regardless of the index word, keyword default mode), no (not indexed)
analyzer	An index word: word to use when you create an index, such as ik_smart, ik_max_word, standard
search_analyzer	Search word is: When searching for value of the field, passing the word queries content.
fields	Multi-field index: Used when the field requires the use of multiple index mode. Such as: urban search New York "city":

"city":{  
	"type": "text",   
	"analyzer": "ik_smart",   
	"fields": {      
		"raw": {         
			"type":  "keyword"       
		}   
	}
}

Explanation: city corresponds to an alias taken city.raw, city keyword type is text, the type of city.raw
search word city; city.raw search terms regardless of filtering and sorting search then later can be used city.raw field name

7.2. Add Mapping

Note: If the index database already contains data, you can not add mapped

7.2.1. Create a new index database

put aigou

7.2.2. Creating a single type mapping

put aigou/goods/_mapping
{
	"goods": {
        "properties": {
            "id": {
                "type": "long"
            },
            "name": {
                "type": "text",
                "analyzer": "ik_smart",
                "search_analyzer": "ik_smart"
            }
        }
    }
}

Explanation: The index database to aigou is the type of goods to create a mapping, id specified as a long type, name specified as a text type (for word), analyzer using the word ik, also use the query word ik

7.2.3. Create a multi-type maps

PUT aigou
{
  "mappings": {
    "user": {
      "properties": {
        "id": {
          "type": "integer"
        },
        "info": {
          "type": "text",
          "analyzer": "ik_smart",
          "search_analyzer": "ik_smart"
        }
      }
    },
    "dept": {
      "properties": {
        "id": {
          "type": "integer"
        },
        ....更多字段映射配置
      }
    }
  }
}

Explanation: Create documents and maps to the user dept

7.3. Array / object mapping

Basic types of field mapping is very simple, straightforward configuration corresponding to the type, but how arrays and objects of the specified type it?

7.3.1. Object Mapping

{
  "id" : 1,
  "girl" : {
      "name" : "王小花",
      "age"  : 22
  }
}

Document Map

{ 
  "properties": {
       "id": {"type": "long"},
       "girl": {
           "properties":{
           		"name": {"type": "keyword"},
           		"age": {"type": "integer"}
           }
        }
  }
}

7.3.2. Mapping Array

{
	"id" : 1,
	"hobby" : ["王小花","林志玲"]
}

Document Map

{ 
		"properties": {
            "id": {"type": "long"},
            "hobby": {"type": "keyword"}
     }
}

Explanation: The mapping array only needs to be mapped to one element, because the array element type is the same.

7.3.3. An array of objects

{
	"id" : 1,
	"girl":[{"name":"林志玲","age":32},{"name":"赵丽颖","age":22}]
}

Document Map

"properties": {
        "id": {
            "type": "long"
        },
        "girl": {
            "properties": {
              "age": { "type": "long" },
              "name": { "type": "text" }
            }
        }
}

7.4. Global Mapping

Index database multiple types (Table) field is the same map, as all of the ID can be specified as integer types, based on this idea, we can make a global map, so that all documents are mapped using global document. Global mapping can be set in two ways through dynamic templates and default.

7.4.1 default:. Default

All types mapping configuration will inherit the configuration _default_ index, such as:

PUT {indexName}
{
 "mappings": {
    "_default_": { 
       "_all": {
        	"enabled": false
       }
  },
  "user": {}, 
  "dept": { 
     "_all": {
      	"enabled": true
     }
  }
 }

Close the default _all, dept custom open all

7.4.2. Dynamic Templates

dynamic_templates

PUT _template/global_template  //创建名为global_template的模板
{
  "template":   "*",  //匹配所有索引库
  "settings": { "number_of_shards": 1 }, //匹配到的索引库只创建1个主分片
  "mappings": {
    "_default_": {
      "_all": { 
        "enabled": false //关闭所有类型的_all字段
      },
      "dynamic_templates": [
        {
          "string_as_text": { 
            "match_mapping_type": "string",//匹配类型string
            "match":   "*_text", //匹配字段名字以_text结尾
            "mapping": {
              "type": "text",//将类型为string的字段映射为text类型
              "analyzer": "ik_max_word",
              "search_analyzer": "ik_max_word",
              "fields": {
                "raw": {
                  "type":  "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        },
        {
          "string_as_keyword": { 
            "match_mapping_type": "string",//匹配类型string
            "mapping": {
              "type": "keyword"//将类型为string的字段映射为keyword类型
             }
          }
        }
      ]
    }
  }}

PS: Mapping the way priorities (Low -> High): Default -> Global -> Customize