es to do the search engines, but also to build a common language ELK do log analysis system.
Search engines:
es There are three basic terms we need to know: index, type of document. Database index corresponds to the type corresponding to the data table, the mapping table corresponding to the structure, the line corresponding to the document data.
Index:
inverted index: also known as an inverted index. According to the document content indexing keywords, called inverted index.
Type:
String string, there is a string that represents the keyword, text;
keyword: built directly inverted index
text: first word, after the construction of a reverse index
number refers to the type: long, integer, short, byte , double, float
Date type: date
Boolean: boolean
binary type: binary
array types (array datatype)
Mapping:
Mapping used to create the document, used to define the type of field, word, properties and so on. Divided into dynamic and static mapping maps.
Dynamic:
relational database, we need to create a database, and then build the database table, insert data; and does not require pre-designed mapping es, es when the document is written, according to the document will automatically recognize the type field, known as dynamic mapping .
Static:
In es can be defined first map, which contain various fields and their types of documents, etc., in this way is called static mapping.
es lucene on the basis of the package, a distributed search engine.
Distributed es principle:
Cluster Cluster:
cluster multiple nodes, including a master node, the master node is selected by election, this is from the inside of the cluster is. es is a concept to the center, it is outside of the cluster, is logically full, you communicate with the entire cluster es communication with any node equivalent.
Shards fragmentation:
es can be a complete index into a plurality of fragments when large index, split into multiple, distributed to different nodes can share the pressure to improve the search efficiency and other, es improve usability. Default create five slices, slice number can not be changed after the index, but the backup can be changed.
Replicas:
copies of the index, the index es may be provided a plurality of copies, a copy function: one is to improve the fault tolerance of the system can be restored from the copy of a node when a damaged or missing fragment; es second is to improve the efficiency of the query , es will automatically search request load balancing.
Recovery:
data recovery or data call redistribution. es will be reassigned according to the load index slicing machines as nodes join or leave, will hang data recovery when a node restart.
es a master-slave architecture to achieve fragmentation and backup:
indexing, first established in the master, then the slave to synchronize.
But this will not lead to a low write performance?
No, only need to go through the type of index or master, data is written there is a simple routing rules, it can route to any node in the cluster, the pressure is distributed in the cluster.
Why ES cluster to achieve?
We can handle large indexes can not be run on a single server, the index over the General Assembly cause efficiency problems, may be the cause can not run or may be stored in memory, each fragment can have multiple copies, by a copy distributed to more than a server, you can improve the load capacity of the query.
Cluster analysis of core principles:
1, each index is stored into multiple shards, created by default index is assigned five slices for storage. Each slice will be distributed deployment to deploy a plurality of different nodes on the slice is called primary shards.
2, in order to achieve high availability, the main fragment with a corresponding fragment backup replicas shards, replica shards slice carries responsible for fault tolerance and load balancing requests. (Each master slice has its own backup fragments corresponding to the primary backup can not be on the same server, the master primary shards replicas shards and a node on the same node)
ELK do log analysis system:
analysis of the log what's the use?
If a distributed system has 1,000 machines, the system fails, I want to see the log, but also on a log in to see one, but if the log access to the ELK, the system is running, suddenly appeared abnormal, in the log can be timely feedback, we can see the log in the case of direct Kibana, if then access real-time calculation module, you can also do real-time alarm.
Summary:
This article Reference: http://developer.51cto.com/art/201904/594615.htm?pc
bloggers to write easy to understand, Xiao Bian highly recommended.
