Elasticsearch (skill points)

Install

insert image description here

Enter the /elasticsearch-6.1.0 directory and modify elasticsearch.yml so that any host can access ES

cd ./elasticsearch-6.1.0
vim ./config/elasticsearch.yml

# 将network.host：前的#去掉，修改为
network.host:0.0.0.0

insert image description here

Change the sysctl.conf file command:

sudo vim /etc/sysctl.conf  
# 增加如下一行
vm.max_map_count=655360

insert image description here

Execute the following command to make it effective

sudo sysctl -p

insert image description here

Start ES, use a browser to access port 9200,

./bin/elasticsearch

insert image description here

Access port 9200 of the virtual machine address, [virtual machine IP]: 9200

insert image description here

Indicates that the startup is successful.

Output Logstash results to ES

After ES starts, you can add data. The common way is to use Logstash to monitor data files and output the results to ES.

Modify the Logstash-plain.conf configuration file and set the input configuration to ES,

cd /usr/local/logstash
vim logstash-plain.conf

# 配置文件内容如下
input{  
	file{    
		path => ['/opt/lampp/logs/access_log']    
		type => 'logstash_access_log'    	
		start_position => "beginning" 
	}
}
filter{  
	grok{    
		match => {    
		"message"=>"%{IP:ip}"    
		}  
	}
}
output{  
	elasticsearch{   
		hosts => ["localhost:9200"]    
		index => "httpd_logdata-%{+YYYY.MM.dd}"  
	}
}

Save the configuration file.
Start the web service (apache)

cd /opt/lampp
sudo ./xampp startapache

insert image description here

Test the web service:
Enter in the address bar of the browser: the IP address of the virtual machine.
If you can open the test web page, it means that the web service starts normally.

execute logstash

cd /usr/local/logstash
logstash -f ./logstash-plain.conf

insert image description here

Kibana visualization platform

Start Kibana, make sure that Elasticsearch has been started before starting, the command is as follows

kibana

insert image description here

Access port 5601 through a browser to view the Kibana interface

Kibana high frequency function

create index

insert image description here

After using Logstash to collect data into Elasticsearch, if you want to analyze and search the data in Kibana, you need to create an index in Kibana.

The first step of creating an index is pattern matching, which matches the index in the Logstash output configuration. When there are multiple indexes that meet the matching pattern entered by the user, the data in multiple indexes will be created into the same index. After the matching is completed Prompt "success"
Click "Next step" to set the time filter field name, select "@timestamp",