elasticsearch data organization structure
1. mapping
1.1 Introduction
mapping: mapping relationship means, in particular, refers to organizational structure. Understood in this context as a data structure, comprising a table structure, the table constraints, data types. (Non-native environment not afford to hurt ... very obscure, half an hour before the circle turned to)
1.2. mapping type
Each index has a type map, which determines the document index approach.
Mapping is divided into two types:
- Meta fields: _index, _type, _id, _source
- Attribute or value field:
Field Value Data Type - the type of data corresponding to mysql
有text,keywork,date,boolean,object,nested,geo_point等
See in particular other documents.
1.3. Mapping Constraints
Too many in a defined index fields may lead to memory overflow, it is not as thought it would be rare.
There are some settings used to constrain
index.mapping.total_fields.limit
The number of index fields, including the field count, object map field alias. The default value of 1000
index.mapping.depth.limit
The maximum depth of field, refers to the depth of the object reference. The default is 20.
index.mapping.nested_fields.limit
The maximum number of distinct nested mappings in an index, defaults to 50.
The number of non-repetitive nested mappings, default 50
index.mapping.nested_objects.limit
The maximum value of a single document nested json object, the default 10000
index.mapping.field_name_length.limit
Length limit field names, default is unlimited.
1.4 Dynamic map
And field mapping types do not need to define in advance. Automatically created when you add.
In the top-level mapping, will the internal object and nested fields so.
1.5. Explicit mapping display map
Set command syntax:
PUT /<index>/_mapping
Creating index, specifying the map
curl -X PUT "localhost:9200/my-index?pretty" -H 'Content-Type: application/json' -d'
{
"mappings": {
"properties": {
"age": { "type": "integer" },
"email": { "type": "keyword" },
"name": { "type": "text" }
}
}
}
'
Add a field mapping
Case:
curl -X PUT "localhost:9200/my-index/_mapping?pretty" -H 'Content-Type: application/json' -d'
{
"properties": {
"Employee-id": {# field name
"Type": "keyword", # Field Type
"Index": false # This field representatives do not participate in index
}
}
}
note:
Existing mapping can not be changed, the following Exception:
- Add properties to object fields
- Value can be changed is ignore_above.
Modify existing maps would have been invalid index data. If you want to modify reflect organizational relationships, create a new index and reindex data. If you just want to change the field names, it is recommended to add an alias field.
1.6. Related command
View index mapping
GET /my-index/_mapping
rv = es.indices.get_mapping(index_name)
1.7. Precautions
_doc problem
7.X and later did not provide the type argument, but in the type position 7.x, part of the command to be written in _doc.
2. field datatypes
Elasticsearch supports a number of different datatypes for the fields in a document:
-----Core datatypes
2.1. string
text and keyword
2.1.1. text
It will be resolved to the individual terms before being indexed.
2.1.2. keyword
Field for indexing structured content, such as email addresses, hostnames, status code.
Case
PUT my_index
{
"mappings": {
"properties": {
"tags": {
"type": "keyword"
}
}
}
}
2.2. Numeric
long, integer, short, byte, double, float, half_float, scaled_float
2.3. Date
date
2.4. Date nanoseconds
date_nanos
2.5. Boolean
boolean
2.6. Binary
binary
2.7. Range
integer_range, float_range, long_range, double_range, date_range
------Complex datatypes
2.8. Object
object for single JSON objects
2.9. Nested
nested for arrays of JSON objects
3. meta-field meta fields
Each document has its own metadata fields.
3.1. identity meta-fields
_index
_type
_id
3.2. document source meta-fields
_source: Source data JSON
_size: _source size, in bytes, provided by the mapper-size plugin.
3.3. indexing meta-fields
_fields_names
_igonred
3.4. routing meta-field
_routing
3.5. other meta-field
_meta
4. analyzer
document: https://www.elastic.co/guide/en/elasticsearch/reference/current/analyzer.html
Here is a word index case set parameters and application:
PUT my_index
{
"settings":{
"analysis":{
"analyzer":{
"my_analyzer":{
"type":"custom",
"tokenizer":"standard",
"filter":[
"lowercase"
]
},
"my_stop_analyzer":{
"type":"custom",
"tokenizer":"standard",
"filter":[
"lowercase",
"english_stop"
]
}
},
"filter":{
"english_stop":{
"type":"stop",
"stopwords":"_english_"
}
}
}
},
"mappings":{
"properties":{
"title": {
"type":"text",
"analyzer":"my_analyzer",
"search_analyzer":"my_stop_analyzer",
"search_quote_analyzer":"my_analyzer"
}
}
}
}
PUT my_index/_doc/1
{
"title":"The Quick Brown Fox"
}
PUT my_index/_doc/2
{
"title":"A Quick Brown Fox"
}
GET my_index/_search
{
"query":{
"query_string":{
"query":"\"the quick brown fox\""
}
}
}