Table of contents
- 1. Document batch operations
- 2. Advanced query in DSL language
-
- 1 Overview
- 2. No query conditions
- 3. There are query conditions
- 4.query example
-
- 4.1term query
- 4.2match fuzzy query
- 4.3 multi_match multi-field fuzzy matching query and specified field query
- 4.4query_string unspecified field and specified field condition query, including AND and OR conditions
- 4.5range range query
- 4.6 Paging, outputting specified fields, and sorting queries by fields
- 5.Filter query example
- 6.match、term、match_phase、query_string总结
- 3. Document Mapping
- 8.Elasticsearch optimistic concurrency control
1. Document batch operations
1. _mget (get document data in batches)
(1) Request method without specifying index and type in the URL
: GET
request address: _mget
Function description: Obtain data without index (index library) and type (table) through id
Request parameters:
"docs":[
{
"_index":"索引库名称",
"_type":"类型默认是_doc",
"_id":主键
}, #多个条件使用逗号分割
{
.............
}
]
_type is the default after es7.
Example ES statement:
我是查询的两个索引库（test_index01，test_index02）
GET _mget
{
"docs":[
{
"_index":"test_index01",
"_id":1
},
{
"_index":"test_index02",
"_id":1
}
]
}
result
{
"docs" : [
{
"_index" : "test_index01",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"_seq_no" : 3,
"_primary_term" : 1,
"found" : true,
"_source" : {
"name" : "秀儿",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
},
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "1",
"_version" : 3,
"_seq_no" : 2,
"_primary_term" : 1,
"found" : true,
"_source" : {
"name" : "李四",
"sex" : 1,
"age" : 28,
"address" : "北京",
"remark" : "java"
}
}
]
}
(2) Specify the index request method in the URL
: GET
request address: GET /index library name/_mget
function description: batch query the contents of the specified index library
Request parameters:
GET /索引库名称/_mget
{
"docs":[
{
"_id":主键id
},
{
"_id":主键id
}
]
}
Example
GET /test_index01/_mget
{
"docs":[
{
"_id":1
},
{
"_id":2
}
]
}
result
{
"docs" : [
{
"_index" : "test_index01",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"_seq_no" : 3,
"_primary_term" : 1,
"found" : true,
"_source" : {
"name" : "秀儿",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
},
{
"_index" : "test_index01",
"_type" : "_doc",
"_id" : "2",
"found" : false
}
]
}
2. _bulk (create document data in batches)
The put and post methods have been discussed in the previous article. Here you can perform the two methods yourself to see the difference. The following will be explained in the form of a post.
(1) Create documents in batches.
Do not specify index and type in the URL.
Request method: POST
request address: _bulk
Function description: Add, delete, and modify document data.
Request parameters:
Generally, there are two lines of parameters (or many pairs of parameters).
The first Line: Specify the operation type (addition, deletion, modification) and the object of the operation (index, type, id)
The second line: the data of the operation
1 {
"actionName":{
"_index":"indexName", "_type":"typeName","_id":"id"}}
2 {
"field1":"value1", "field2":"value2"}
POST _bulk
{
"操作类型(create,index,delete,update)":{
"_index":"索引库名称", "_type":"类型（7.x以后版本可省略）", "_id":指定主键}}
{
"title":"文档数据具体内容","content":"文档数据具体内容","tags":["文档数据具体内容", "文档数据具体内容"]}
Example:
The id:3 here is the ID in the business, not the primary key of es. The primary key of es is _id:9like this
POST _bulk
{
"create":{
"_index":"test_index03", "_id":9}}
{
"id":3,"title":"1","content":"111","tags":["java", "面向对象"]}
{
"create":{
"_index":"test_index03","_id":810}}
{
"id":4,"title":"2","content":"222","tags":["java", "面向对象"]}
result
{
"took" : 3,
"errors" : false,
"items" : [
{
"create" : {
"_index" : "test_index03",
"_type" : "_doc",
"_id" : "9",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 12,
"_primary_term" : 1,
"status" : 201
}
},
{
"create" : {
"_index" : "test_index03",
"_type" : "_doc",
"_id" : "810",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 13,
"_primary_term" : 1,
"status" : 201
}
}
]
}
(2) Normally create or replace index.
If the original document does not exist, create it
. If it exists, modify the original document in full
. Example
POST _bulk
{
"index":{
"_index":"article", "_id":99}}
{
"title":"1","content":"2222","tags":["333", "444"]}
result
{
"took" : 5,
"errors" : false,
"items" : [
{
"index" : {
"_index" : "article",
"_type" : "_doc",
"_id" : "99",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 2,
"_primary_term" : 1,
"status" : 201
}
}
]
}
When executed again, the result becomes "result": "updated"
{
"took" : 2,
"errors" : false,
"items" : [
{
"index" : {
"_index" : "article",
"_type" : "_doc",
"_id" : "99",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 3,
"_primary_term" : 1,
"status" : 200
}
}
]
}
3. _bulk (Delete document data in batches)
Do not specify index and type in the URL
. Request method: POST
request address: _bulk
Function description: Delete the document content of the index library in batches.
Request parameters:
POST _bulk
{
"delete":{
"_index":"索引库名称", "_id":es主键}}
Example
POST _bulk
{
"delete":{
"_index":"test_index03", "_id":3}}
{
"delete":{
"_index":"test_index03", "_id":4}}
result
{
"took" : 3,
"errors" : false,
"items" : [
{
"delete" : {
"_index" : "test_index03",
"_type" : "_doc",
"_id" : "3",
"_version" : 7,
"result" : "deleted",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 14,
"_primary_term" : 1,
"status" : 200
}
},
{
"delete" : {
"_index" : "test_index03",
"_type" : "_doc",
"_id" : "4",
"_version" : 7,
"result" : "deleted",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 15,
"_primary_term" : 1,
"status" : 200
}
}
]
}
4. _bulk (modify document data in batches)
Do not specify index and type in the URL
Request method: POST
request address: _bulk
Function description: Modify document data in batches
Request parameters:
POST _bulk
{
"update":{
"_index":"索引库名称", "_id":es主键}}
{
"doc":{
"更新的字段名称":"更新内容"}}
Example
POST _bulk
{
"update":{
"_index":"test_index03", "_id":3}}
{
"doc":{
"title":"ES"}}
{
"update":{
"_index":"test_index03", "_id":4}}
{
"doc":{
"content":"修改内容"}}
The result (I deleted it before, and I modified it after I added it)
{
"took" : 35,
"errors" : false,
"items" : [
{
"update" : {
"_index" : "test_index03",
"_type" : "_doc",
"_id" : "3",
"_version" : 3,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 20,
"_primary_term" : 1,
"status" : 200
}
},
{
"update" : {
"_index" : "test_index03",
"_type" : "_doc",
"_id" : "4",
"_version" : 3,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 21,
"_primary_term" : 1,
"status" : 200
}
}
]
}
2. Advanced query in DSL language
1 Overview
Domain Specific Language
Elasticsearch provides a JSON-based DSL to define queries.
DSL consists of two clauses: leaf query clause and compound query clause.
2. No query conditions
Unconditional query, default query all
GET /test_index03/_doc/_search
{
"query":{
"match_all":{
}
}
}
3. There are query conditions
3.1 Leaf condition query (single field query condition)
(1) Fuzzy query
Fuzzy matching is mainly for text type fields. Text type fields will segment the text content. When querying, the content passed by the conditions you query will also be segmented, and then the matching data will be found through the inverted index. Fuzzy matching is mainly implemented through parameters such as match. .
①match: Fuzzy matching condition content through match keyword
②prefix: prefix matching
③regexp: match data through regular expressions
match的复杂用法
- The match condition also supports the following parameters:
- query: specifies the matching value
- operator: matching condition type
- and: All conditions must be matched after word segmentation
- or : There is only one match after the conditional word segmentation (default)
- minmum_should_match : Specify the minimum number of matches
(2) exact match
- term: single condition is equal
- terms: a single field belongs to a value in a value array
- range: The field belongs to a range of values
- exists : Whether the value of a certain field exists
- ids: batch query by ID
3.2 Combined condition query (multi-condition query)
Combined conditional query is a complete query condition formed by combining leaf conditional query statements.
- bool: There is an and, or or not relationship between the conditions
- must: Each condition must be met, that is, each condition is related to and
- should: Only one of each condition is satisfied, that is, each condition is in an OR relationship
- must_not: Not all conditions are met, that is, each condition is not.
- filter: does not calculate the relevance score. It does not calculate the _score, which is the relevance score, and is more efficient.
- constant_score: Do not calculate relevance score
must/filter/shoud/must_notThe sub-condition of equal is passed
term/terms/range/ids/exists/matchas parameter by equal leaf condition.
3.3 Connection query (multi-document merge query)
- Parent-child document query: parent/child
- Nested document query: nested
3.4 Difference between query DSL (query DSL) and filter DSL (filter DSL)
There are two types of DSL query languages: query and filter.
4.query example
4.1term query
term查询不会对字段进行分词查询, will use an exact match
Use term for precise query, and the query field mapping type is keyword (please see the catalog for mapping related information)
Index content
GET /test_index02/_search
{
"query":{
"match_all":{
}
}
}
result
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"name" : "秀儿",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
},
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "C9bd_IYBuSAClcwx8fGZ",
"_score" : 1.0,
"_source" : {
"name" : "李四",
"sex" : 1,
"age" : 28,
"address" : "北京",
"remark" : "java"
}
}
]
}
}
Example 1
POST /test_index02/_search
{
"query":{
"term":{
"age":"28"
}
}
}
result
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"name" : "李四",
"sex" : 1,
"age" : 28,
"address" : "北京",
"remark" : "java"
}
}
]
}
}
示例2
POST /test_index02/_search
{
"query":{
"term":{
"name":"秀儿"
}
}
}
result
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 0,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
}
}
示例3
POST /test_index02/_search
{
"query":{
"term":{
"name.keyword":"秀儿"
}
}
}
result
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.6931471,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.6931471,
"_source" : {
"name" : "秀儿",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
}
]
}
}
4.2match fuzzy query
match会根据该字段的分词器，进行分词查询
Example
POST /test_index02/_search
{
"from":0,
"size":2,
"query":{
"match":{
"name":"秀儿"
}
}
}
result
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.3862942,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.3862942,
"_source" : {
"name" : "秀儿",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
}
]
}
}
4.3 multi_match multi-field fuzzy matching query and specified field query
This is just a simple demonstration. More detailed methods need to be understood by yourself.
index all data
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"name" : "秀儿",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
},
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "DNbg_IYBuSAClcwx4fEv",
"_score" : 1.0,
"_source" : {
"name" : "李四",
"sex" : 1,
"age" : 28,
"address" : "北京",
"remark" : "java"
}
},
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "Ddbm_IYBuSAClcwxbfER",
"_score" : 1.0,
"_source" : {
"name" : "王五",
"sex" : 1,
"age" : 28,
"address" : "北京",
"remark" : "秀儿多字段"
}
},
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "Dtbv_IYBuSAClcwx5fFw",
"_score" : 1.0,
"_source" : {
"name" : "赵六",
"sex" : 1,
"age" : 28,
"address" : "秀儿",
"remark" : "python"
}
}
]
Example (specify fields)
POST /test_index02/_search
{
"query":{
"multi_match":{
"query":"秀儿",
"fields":[
"remark",
"name"
]
}
}
}
result
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.9616582,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.9616582,
"_source" : {
"name" : "秀儿",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
},
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "Ddbm_IYBuSAClcwxbfER",
"_score" : 1.3921447,
"_source" : {
"name" : "王五",
"sex" : 1,
"age" : 28,
"address" : "北京",
"remark" : "秀儿多字段"
}
}
]
}
}
Example 2
POST /test_index02/_search
{
"query":{
"multi_match":{
"query":"秀儿"
}
}
}
result
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 2.4079456,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "1",
"_score" : 2.4079456,
"_source" : {
"name" : "秀儿",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
},
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "Dtbv_IYBuSAClcwx5fFw",
"_score" : 1.9333868,
"_source" : {
"name" : "赵六",
"sex" : 1,
"age" : 28,
"address" : "秀儿",
"remark" : "python"
}
},
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "Ddbm_IYBuSAClcwxbfER",
"_score" : 1.5779729,
"_source" : {
"name" : "王五",
"sex" : 1,
"age" : 28,
"address" : "北京",
"remark" : "秀儿多字段"
}
}
]
}
}
Example 3 (fields using wildcards)
POST /test_index02/_search
{
"query":{
"multi_match":{
"query":"秀儿",
"fields":[
"address",
"na*"
]
}
}
}
result
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 2.4079456,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "1",
"_score" : 2.4079456,
"_source" : {
"name" : "秀儿",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
},
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "Dtbv_IYBuSAClcwx5fFw",
"_score" : 1.9333868,
"_source" : {
"name" : "赵六",
"sex" : 1,
"age" : 28,
"address" : "秀儿",
"remark" : "python"
}
}
]
}
}
4.4query_string unspecified field and specified field condition query, including AND and OR conditions
AND 和 OR 需要大写
Unspecified example
POST /test_index02/_search
{
"query":{
"query_string":{
"query":"王五 OR 李四"
}
}
}
result
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 2.4079456,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "DNbg_IYBuSAClcwx4fEv",
"_score" : 2.4079456,
"_source" : {
"name" : "李四",
"sex" : 1,
"age" : 28,
"address" : "北京",
"remark" : "java"
}
},
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "Ddbm_IYBuSAClcwxbfER",
"_score" : 2.4079456,
"_source" : {
"name" : "王五",
"sex" : 1,
"age" : 28,
"address" : "北京",
"remark" : "秀儿多字段"
}
}
]
}
}
Unspecified example
POST /test_index02/_search
{
"query":{
"query_string":{
"query":"王五 AND 秀儿"
}
}
}
result
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 3.9859185,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "Ddbm_IYBuSAClcwxbfER",
"_score" : 3.9859185,
"_source" : {
"name" : "王五",
"sex" : 1,
"age" : 28,
"address" : "北京",
"remark" : "秀儿多字段"
}
}
]
}
}
Example of a specified field (wildcards are used in the field na*, only one name field matches here)
POST /test_index02/_search
{
"query":{
"query_string":{
"query":"王五 OR 张三",
"fields":[
"address",
"na*"
]
}
}
}
result
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 2.4079456,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "Ddbm_IYBuSAClcwxbfER",
"_score" : 2.4079456,
"_source" : {
"name" : "王五",
"sex" : 1,
"age" : 28,
"address" : "北京",
"remark" : "秀儿多字段"
}
}
]
}
}
4.5range range query
gte is greater than or equal to, lte is less than or equal to, gt is greater than, lt is less than.
Example
POST /test_index02/_search
{
"query":{
"range":{
"age":{
"gte":25,
"lte":26
}
}
}
}
result
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"name" : "秀儿",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
}
]
}
}
4.6 Paging, outputting specified fields, and sorting queries by fields
Example
POST /test_index02/_search
{
"query":{
"range":{
"age":{
"gte":25,
"lte":28
}
}
},
"from":0,
"size":2,
"_source":[
"name",
"age"
],
"sort":{
"age":"desc",
"sex":"desc"
}
}
result
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 4,
"relation" : "eq"
},
"max_score" : null,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "DNbg_IYBuSAClcwx4fEv",
"_score" : null,
"_source" : {
"name" : "李四",
"age" : 28
},
"sort" : [
28,
1
]
},
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "Ddbm_IYBuSAClcwxbfER",
"_score" : null,
"_source" : {
"name" : "王五",
"age" : 28
},
"sort" : [
28,
1
]
}
]
}
}
5.Filter query example
Filter的查询不会计算相关性分值，也不会对结果进行排序, 因此效率会高一点，查询的结果可以被缓存
Example
POST /test_index02/_search
{
"query":{
"bool":{
"filter":{
"term":{
"age":25
}
}
}
}
}
result
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.0,
"hits" : [
{
"_index" : "test_index02",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.0,
"_source" : {
"name" : "秀儿",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
}
]
}
}
6.match、term、match_phase、query_string总结
English word segmentation: Split individual words according to spaces
Take hello july as an example
. If the finest granularity of word segmentation is: hello, july. There is an example of this in the first blog (word segmentation process).
- For fuzzy match
matching, you need to specify the field name, but the input query content will be segmented. For example, if you query "hello july", the word will be segmented first and split into two parts: hello and july. Then the corresponding search in the index library will be performed according to the rules of the inverted index. Content. If the field contains hello or july, or hello july, it will be queried. - term
Sometimes the query results of term and match are the same. For example, if term queries the single word july, the result will be the same as that of match. The term query will not segment the content, which means that july will not be split into finer parts. word. When querying July with match, although match will segment the query content, July is already the most fine-grained segmentation. Then, July is used for inverted index query, so the results of the two queries are the same.
But if the term query hello july is different, hello july is used as a whole for the inverted index query and will not be word segmented, while match will be word segmented, so the query results are not the same. - match_phase
match_phase will also segment the query content, but the result must also contain all the segmented words, and the order must be the same. Taking the query "hello july" as an example, the result must contain hello and july, and they must be consecutive. Appearing together, the order is also fixed. It can be understood that "hello july" is a fuzzy query for the finest granularity. And "july hello" will not appear in the results if the order is wrong and the query conditions are not met. - query_string
is similar to match, but match requires specifying field names, while query_string searches in all fields.
3. Document Mapping
ElasticSearch mapping is divided into two types: dynamic mapping and static mapping.
1.Dynamic mapping
In a relational database (such as mysql), you need to create a database and data table first, then define fields, field types, etc., and finally configure them before inserting data into this table. In ElasticSearch, there is no need to define mappings (tables, fields, types in mysql, etc.). When a document is written to ElasticSearch, the type will be automatically identified based on the document fields. This mechanism is called dynamic mapping.
The dynamic mapping rules are as follows:
| json data | automap type |
|---|---|
| null | No fields added |
| true or false | boolean type |
| decimal | float type |
| number | long type |
| date | date or text |
| string | text type |
| array | Determined by the first non-null value of the array |
| json object | object type |
2. Static mapping
In ElasticSearch, you can also define mappings in advance, including each field type of the document, word segmenter, etc. This method is static mapping. ( 设置完的字段映射关系，是不支持修改的)
3. Get document mapping
Inquire
GET /test_index02/_mapping
result
{
"test_index02" : {
"mappings" : {
"properties" : {
"address" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"age" : {
"type" : "long"
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"remark" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"sex" : {
"type" : "long"
}
}
}
}
}
Set static mapping
"index":true: whether to perform word segmentation indexing on this field
"store":true: whether to save the original data on this field
PUT /test_index05
{
"mappings":{
"properties":{
"name":{
"type":"keyword",
"index":true,
"store":true
},
"sex":{
"type":"integer",
"index":true,
"store":true
},
"age":{
"type":"integer",
"index":true,
"store":true
},
"address":{
"type":"text",
"index":true,
"store":true
},
"remark":{
"type":"text",
"index":true,
"store":true
}
}
}
}
Inquire
GET /test_index05/_mapping
result
{
"test_index05" : {
"mappings" : {
"properties" : {
"address" : {
"type" : "text",
"store" : true
},
"age" : {
"type" : "integer",
"store" : true
},
"name" : {
"type" : "keyword",
"store" : true
},
"remark" : {
"type" : "text",
"store" : true
},
"sex" : {
"type" : "integer",
"store" : true
}
}
}
}
}
4. Core types
- String: string, the string type includes text and keyword.
- text: This type is used to index long texts. Before creating the index, these texts will be segmented into words, converted into word combinations, and indexed; es is allowed to retrieve these words. The text type cannot be used for sorting and aggregation.
- Keyword: This type cannot be used for word segmentation and can be used for search filtering, sorting and aggregation. The keyword type cannot be used for word segmentation fuzzy retrieval using text. (That is to say, if you save a piece of data as "hello july", if it is of text type, it will be segmented and stored in the index, while keyword will directly store hello july as a whole in the index without splitting it.)
- Numeric type: long, integer, short, byte, double, float
- Date type: date
- Boolean: boolean
5.The difference between keyword and text mapping types
Index mapping type
name is now keyword( 只能精准查询, 不能分词查询，能聚合、排序)
"mappings" : {
"properties" : {
"address" : {
"type" : "text",
"store" : true
},
"age" : {
"type" : "integer",
"store" : true
},
"name" : {
"type" : "keyword",
"store" : true
},
"remark" : {
"type" : "text",
"store" : true
},
"sex" : {
"type" : "integer",
"store" : true
}
}
}
}
}
index all data
"hits" : [
{
"_index" : "test_index05",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"name" : "天王盖地虎",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
}
]
term query
POST /test_index05/_search
{
"query":{
"term":{
"name":"天王盖地虎"
}
}
}
result
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.2876821,
"hits" : [
{
"_index" : "test_index05",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.2876821,
"_source" : {
"name" : "天王盖地虎",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
}
]
}
}
match query
POST /test_index05/_search
{
"query":{
"match":{
"name":"地虎"
}
}
}
result
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 0,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
}
}
这里需要注意一点：当name的type为keyword类型时，使用match进行搜索时，如果搜索内容和存贮内容完全一致，是可以搜到结果的。明明match会分词，为啥有能搜到呢？这个问题有些困扰到我。于是多方了解有了答案
Please read the following carefully
① When the match query type is keyword data in ElasticSearch, the exact match query will be performed directly, and word segmentation will not be performed.
②When using match to query data of type text, the standard analyzer will be used by default for word segmentation, and the query string will be segmented before querying.
③The standard word segmenter will segment the query string according to delimiters such as spaces and punctuation marks. It will also convert words into lowercase and remove common stop words (such as a, an, the, etc.). After word segmentation, ElasticSearch will query each term in the query string after word segmentation in the inverted index and return matching documents.
If you need to use other tokenizers for query, you can specify the corresponding tokenizers in the match query.
For example, specify a word segmenter for word segmentation
{
"query": {
"match": {
"content": {
"query": "天王盖地虎",
"analyzer": "ik_max_word"
}
}
}
}
Index structure
"name" : {
"type" : "keyword",
"store" : true
}
Index content
"hits" : [
{
"_index" : "test_index05",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"name" : "天王盖地虎",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
}
]
Example
POST /test_index05/_search
{
"query":{
"match":{
"name":"天王盖地虎"
}
}
}
result
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.2876821,
"hits" : [
{
"_index" : "test_index05",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.2876821,
"_source" : {
"name" : "天王盖地虎",
"sex" : 1,
"age" : 25,
"address" : "上海",
"remark" : "java"
}
}
]
}
}
6. Specify the ik word segmenter of text type when creating static mapping
Specify the name field as the ik word segmenter
PUT /test_index05
{
"mappings":{
"properties":{
"name":{
"type":"keyword",
"index":true,
"store":true
},
"sex":{
"type":"integer",
"index":true,
"store":true
},
"age":{
"type":"integer",
"index":true,
"store":true
},
"address":{
"type":"text",
"index":true,
"store":true,
"analyzer":"ik_smart",
"search_analyzer":"ik_smart"
},
"remark":{
"type":"text",
"index":true,
"store":true
}
}
}
}
7. Modify the existing mapping
Simple way:
①Create a new static mapping index that meets your needs
②Migrate the old index data to the new index
POST _reindex
{
"source":{
"index":"index_old"
},
"dest":{
"index":"index_new"
}
}
③ If you still want to use the index name to perform related business operations, you need to modify the alias (optional)
and set the alias of the new index to the name of the old index, which is used here 别名和索引一对一关系. (An alias can point to multiple indexes, which will be discussed later)
PUT /index_new/_alias/index_old
Later, we will explain how to deal with this kind of problem in the company.
8.Elasticsearch optimistic concurrency control
-
pessimistic concurrency control
This approach, widely used by relational databases, assumes that change conflicts may occur and therefore blocks access to the resource to prevent conflicts. A typical example is to lock a row of data before reading it to ensure that only the thread that placed the lock can modify the row of data.
-
Optimistic concurrency control
This approach used in Elasticsearch assumes that conflicts are impossible and does not block the operation being attempted. However, if the source data is modified during reading and writing, the update will fail. The application then decides how to resolve the conflict. For example, updates can be retried, using new data, or reported to the user.
ES新版本(7版本及以后版本)不使用version进行并发版本控制，使用 if_seq_no=版本值&if_primary_term=文档位置
Add a new piece of data
PUT /test_index05/_doc/1
{
"name": "天王盖地虎",
"sex": 1,
"age": 25,
"address": "上海",
"remark": "java"
}
Update this data content
POST /test_index05/_update/1
{
"doc":{
"name":"宝塔镇河妖"
}
}
Result
updated again
POST /test_index05/_update/1/?if_seq_no=1&if_primary_term=1
{
"doc":{
"name":"秀儿"
}
}
Result
Simulate concurrency
Execute the command again
POST /test_index05/_update/1/?if_seq_no=1&if_primary_term=1
{
"doc":{
"name":"秀儿"
}
}
result
