1. Background
You must be familiar with the technology stack of ELK. EKL is distributed search Elasticsearch, log collection LogStash, and Kibana that provides a friendly interface. ELK technology stack is widely used in log data analysis, real-time monitoring and other fields'.
2. Get to know Elasticsearch first
Elasticsearch is a very powerful open source search engine that can help us quickly find what we need from massive amounts of data.
Why should we learn Elasticsearch?
Take a look at the global search technology rankings:
3. Comparison between Elasticsearch and traditional database search
Search method:
The difference between ELasticsearch and traditional database searches is that ELasticsearch uses an inverted index, while traditional databases use a forward index .
So what is an inverted index and what is a forward index?
Forward index: Forward index means that the document ID is key, and the number of occurrences of each keyword is recorded in the table. When searching, the word information in each document in the table is scanned until all documents containing the query keyword are found. Whenever If I add a piece of data, I must add a corresponding unique ID
Inverted index: Inverted index is just the opposite. The ID is searched through the entries in the document. In the inverted index, each piece of data is a document. The words in the document are divided into semantics called entries.
Storage structure:
Elasticsearch is oriented to document storage, which can be a piece of commodity data or an order information in the database. Document data will be serialized into json format and stored in elasticsearch.
The structure of traditional databases is mainly tabular data. Each row serves as a piece of data in the current table.
Architecture comparison:
Take Mysq as an example: good at transaction type operations, can ensure data security and consistency
Elasticsearch: good at searching, analyzing and calculating massive data
4. Install Elasticsearch+kibana
Because Elasticsearch needs to be used in conjunction with kibana to have a good experience, it is best to install both together when installing.
Here we recommend using the docket installation. Ordinary virtual machine installation can refer to my blog.
ElasticSearch installation - Programmer Sought
4.1 Single point deployment ES
1. Create a network
Because we also need to deploy the kiana container, we need to interconnect the es and kiana containers. Here first create a network
docker network create es-net2. Load the image: the image of Elasticsearch is about 1G, and the index should be downloaded in advance as much as possible, and then go to the program to open the jar package
//下载命令
docker pull elasticsearch:7.12.1
//加载tar包命令
docker load -i es.tar #压缩包名称3. Run the docker command - single-node (single-point mode) to start Elasticsearch
Port 9200 is an exposed port
Port 9300 is the port that provides interconnection
docker run -d \
--name es \
-e "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" \
-e "discovery.type=single-node" \
-v es-data:/usr/share/elasticsearch/data \
-v es-plugins:/usr/share/elasticsearch/plugins \
--privileged \
--network es-net \
-p 9200:9200 \
-p 9300:9300 \
elasticsearch:7.12.14.2 Single-point deployment of Kibana
1. Pull the image or download the image in advance
// 拉取镜像
docker pull kibana:7.12.1
//打开压缩文件
docker load -i kibana.tar
2. start
docker run -d \
--name kibana \
-e ELASTICSEARCH_HOSTS=http://es:9200 \
--network=es-net \
-p 5601:5601 \
kibana:7.12.1
explain:
--network es-net: Join a network named es-netf, in the same network as elasticsearch -e ELASTICSEARCH_HOSTS=HTTP://es:9200 Set the address of elasticsearch, because kiana is already in the same network as elasticsearch, so You can directly access elasticsearch with the container name
3. Access kibana ip+port (5601)
4. Send a request for testing
The next article explains the basic knowledge and use of Elasticsearch