Recently we are learning to build the knowledge application server platform. Have learned to use Elastic kit from a friend with the book, it took me two days to the latest suite deployed on my server, an intermediate step on countless pit. I recorded the whole process down and you need friends to share.
Update
After testing the program has been installed cracked version compatible Elasticsearch6.4.2 to Kibana6.5.1
Description Environment
Operating System: CentOS 7.5 Tencent public cloud mirrored
Minimum: 1 core 2G (easy to collapse)
Recommended: 2-core 4G (stable)
Recommended: the stronger the better (nice money)
Software Source: Official source yum , official web site: https: //www.elastic.co/
Code Method used herein have actually been tested in Tencent cloud server, the installation process using yum install, if installed with rpm should be little difference.
The last time the service is successfully configured: 2018-10-22, because if the updated version of this program has led to the configuration entering or failure, please forgive me.
If configuration maintenance and other issues are welcome to discuss (in fact, I was just getting started), contact me: [email protected]
preparation
First of all you can give yourself a good name from the host, easy to distinguish between hosts in the future
hostnamectl set-hostname xxx
is best to apply the existing system update to the latest version of
yum -y update
elasticsearch requires Java version 1.8.0_131 or higher, convenience can be installed directly OpenJDK, especially if there is a friend in need can also choose OracleJDK
yum install java- 1.8.0 * -y
from now on we will start the installation Elastic kit, and attention to detail, tips and order
First, install Elasticsearch 6.4.2
official documentation of the process, please refer to the following link
installation process overview: HTTPS: //www.elastic.co/products
elasticsearch download: https: //www.elastic.co/downloads/elasticsearch
official document: https://www.elastic.co/guide/en/elasticsearch/reference/6.4/index.html
The following steps according to the guidelines of the document:
Import GPG key tips for avoiding the key issue of trust during the installation
rpm https://artifacts.elastic.co/GPG-KEY-elasticsearch --import
add the source in the source directory elasticsearch /etc/yum.repos.d/ in. repo
can be performed directly vim /etc/yum.repos.d/elasticsearch.repo, and save after completion of
writing the source information to elasticsearch.repo:
[elasticsearch-6.x]
name = elasticsearch Repository Packages for 6.x
= HTTPS BaseURL: //artifacts.elastic.co/packages/6.x/yum
gpgcheck. 1 =
gpgkey = HTTPS: //artifacts.elastic.co/GPG-KEY-elasticsearch
Enabled. 1 =
the AutoRefresh =. 1
type = RPM-MD
. 1
2
. 3
. 4
. 5
. 6
. 7
. 8
after the addition was complete the source and the source data should be updated via yum mounting elasticsearch
yum yum the install elasticsearch && update -Y -Y
two, initial configuration elasticsearch 6.4.2
Official documentation of the process, please refer to the following link
configuration instructions: https: //www.elastic.co/guide/en/elasticsearch/reference/current/settings.html
Follow the prompts to the document, I extracted some key information
The default settings are sufficient Getting Started, should be as little as possible to modify the configuration file
yum install Elasticsearch run directory: / usr / report this content share / elasticsearch /
yum install Elasticsearch profile directory: / etc / elasticsearch /
elasticsearch.yml file is used to configure Elasticsearch
the JVM .options file is used to configure elasticsearch JVM settings
log4j2.properties file is used to configure the logging elasticsearch
you might want to change:
the need to modify the host IP public network IP, IP network or any IP: network.host: 0.0.0.0
modify the port: http.port: 9200
data storage location: path.data: / var / lib / elasticsearch
log storage location: path.logs: / var / log / elasticsearch
modify the runtime memory limitation: see below
program from the start: systemctl enable elasticsearch.service
start the main program: systemctl start elasticsearch.service
modify the runtime memory on
the official document focuses on the translation:
You should rarely need to change the Java Virtual Machine (JVM) options. If you need to modify, change is most likely set the heap size. Set JVM options preferred method (including JVM system properties and logo) is modified by jvm.options profile.
You may want to set the minimum heap size (Xms) and maximum heap size (Xmx) is set to be equal to each other.
Elasticsearch heap more available, the more it can be used for cache memory. Note, however, too much of the heap may make your machine into a garbage collection pauses a long time.
Xmx set to not more than 50% of physical RAM, to ensure sufficient physical RAM left kernel file system cache.
Official documentation on the JVM: https: //www.elastic.co/guide/en/elasticsearch/reference/current/jvm-options.html
default location of this configuration file modifications /etc/elasticsearch/jvm.options, as the case Xms and Xms, take up too much or too little memory can cause programs to crash or fail to start.
Example values:
-Xms2g
-Xmx2g
1
2
try to start Elasticsearch
after Elasticsearch installation and initial configuration can be in no hurry to start, you can use the command systemctl start elasticsearch.service start Elasticsearch.
If you want to test and verify the installation, you can start the visit http: // <host_ip>: <host_part > ( default address http: // domain or IP: 9200) to verify, taking care to avoid because the local IP configuration error caused by not the connection.
The normal access to the display data of the following format json
{
"name" : "qEgqyT5",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "wow17Li0SK-hgw-bOszN9g",
"version" : {
"number" : "6.4.2",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "04711c2",
"build_date" : "2018-09-26T13:34:09.098244Z",
"build_snapshot" : false,
"lucene_version" : "7.4.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Third, the installation kibana 6.4.2
The process of official documents refer to the link below
Kibana download: https: //www.elastic.co/downloads/kibana
official document: https: //www.elastic.co/guide /en/kibana/6.4/index.html
The following steps according to the guidelines of the document:
Installation through yum Kibana: yum install kibana -y
profile Review: /etc/kibana/kibana.yml
# the machine to fill the public IP network IP, IP network or any IP personal
server.host: "0.0.0.0"
# Configure ES cluster link Kibana connected
elasticsearch.url: "HTTP: // localhost: 9200"
1
2
3
4
5
setup software boot from Kai and start the program
systemctl enable kibana.service
systemctl Start kibana.service
1
2
attempts to start Kibana
before the start Kibana must configure and start elasticsearch, otherwise an error message appears on the page. After you configure and start by http: // <host_ip>: < host_part> access Kibana, it can be accessed with a browser. The default address http: // domain or IP: 5601, the effect shown.
Four, Beats and Logstash installation
can be installed according to individual needs yum install Beats and Logstash, by the above configuration, the formation of a common configuration are relatively simple, usually you can set the host address and certificate.
For details refer to the official document: HTTPS: //www.elastic.co/guide/en/beats/metricbeat/current/index.html
Beats Description: HTTPS: //www.elastic.co/cn/products/beats
Logstash Profile : https: //www.elastic.co/cn/products/logstash
Examples simple
installation metricbeat
yum the install metricbeat -Y
configuration metricbeat
directory: /etc/metricbeat/metricbeat.yml
configuration and host address elasticsearch kibana and opening panel, content:
output.elasticsearch:
the hosts: [ "xxx.xxx.xxx. xxx: 9200 "]
setup.kibana:
host: "xxx.xxx.xxx.xxx:5601"
setup.dashboards.enabled: to true
. 1
2
. 3
. 4
. 5
. 6
. 7
boot and boot program
settings boot: chkconfig --add metricbeat
start the program manually: systemctl start metricbeat
five, provided x-pack
To crack Platinum service, we need to start security settings, that is, to install the certificate on each node
Official documentation of the process, please refer to the following link
https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-security.html
I will focus on methods and presentation of different points of the previous version of the version in the following content, only a simple statement of the reasons
Install x-pack
6.4 version does not require installation of x-pack
since version 6.4 in x-pack is already a built-in set up, do not like other online tutorial to install their own as the x-pack.
First enter trial mode
very strange problem I encountered, if the certificate and then configure the password will cause Kibana can not connect to Elasticsearch, the following tips.
When you set a password:
Unexpected the Response code [403] Calling from GET http://10.10.1.10:9200/_xpack/security/_authenticate?pretty
It does not look like the Feature at The Security Pack the X-IS-ON the this the Available elasticsearch the Node.
Please check if you have installed a license that allows access to X-Pack Security feature.
ERROR: X-IS-Pack Security Not Available.
. 1
2
. 3
. 4
. 5
will open Kibana
Can Not Connect to Cluster The elasticsearch Currently Configured for Kibana.
. 1
FIG.
To avoid this problem, please click the trial and then configure the relevant certificate of x-pack.
Generated CA certificate
path trap to be careful
certutil script most of the tutorials described on the web, in the path / usr / share / elasticsearch / bin / x-pack / in. However, it has been unavailable (you will be prompted to run the file exists but is not available) in 6.4, the new CA certificate generation program in / usr / share / elasticsearch / bin / directory, named elasticsearch-certutil
we can enter / usr / share / elasticsearch / bin / directory execute the following commands to generate the certificate.
./elasticsearch-certutil ca --ca-dn "CN = WolfBolin Elatic CA" --out /etc/elasticsearch/certs/wolfbolin-elastic-ca.p12
meanings of the parameters and instructions on their own can Baidu, or can Baidu see to understand. If you need to use TLS / SSL certificate can refer to the official documentation.
Generating certificate cert
same, can / bin / directory execute the following commands in / usr / share / elasticsearch to generate the certificate.
./elasticsearch-certutil cert -ca /etc/elasticsearch/certs/wolfbolin-elastic-ca.p12 --out /etc/elasticsearch/certs/wolfbolin-elastic-certificates.p12
the certificate generation program will be completed in an interactive way generate a certificate, it is recommended cert certificate without a password is generated to reduce configuration complexity (like when directly enter a password)
reasonable saved certificate
two things to note when you save the certificate:
cert certificate contains the password for the CA certificate, it should not be free to read, pay attention to adjust the authority
may be due to limitations of Java or the program itself, Elasticsearch can not read non-configuration directory / etc / elasticsearch / file, so you cert certificate should be stored in the / etc / elasticsearch / certs / folder, and given the appropriate permissions, my setting is chmod 660 / etc / elasticsearch / certs / *
if less access java.nio will appear in the log. file.AccessDeniedException: the error prompt
configuration in Elasticsearch certificate file
to modify the configuration file /etc/elasticsearch/elasticsearch.yml, add at the end of the file:
xpack.security.transport.ssl.enabled: to true
xpack.security.transport.ssl.verification_mode : Certificate
xpack.security.transport.ssl.keystore.path: certs / wolfbolin-Elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs / wolfbolin-Elastic-certificates.p12
. 1
2
. 3
. 4
Note: two certificates are the same, are cert generated certificate. Certificate the name and path no mistake! ! !
If the certificate has a password refer to the documentation https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#tls-transport
After configuration, reboot elasticsearch
systemctl restart elasticsearch.service
If there is an error in the restart process can not start with a lead, you can analyze the log /var/log/elasticsearch/elasticsearch.log find out why.
Initial results
after configuring the security certificate, if it will try to access tips on the page, is not accessible at this time Kibana of Elasticsearch
CAN not Connect to at The elasticsearch Cluster Rate this page Currently the Configured for Kibana.
The Refer to at The Kibana logs for More the Details and Refresh to the try Again.
1
2
user passwords
program setup-passwords used on the Internet other tutorials have been unavailable in 6.4 (exists but the file will prompt unavailable runtime), the new password setup in / usr / share / elasticsearch / bin / directory named setup-passwords
we can enter / usr / share / elasticsearch / bin / directory execute the following command to generate a password .
./elasticsearch-setup-passwords auto (auto-generated) or ./elasticsearch-setup-passwords interactive (manual setting)
if generated automatically, suggest the following procedure, given refer to as the second step.
[@ Sbox the root-wolfbolin bin] # ./elasticsearch-setup-passwords Auto
Initiating The Setup Reserved for Users of Passwords Elastic, kibana, logstash_system, beats_system.
of The Will BE RANDOMLY Generated Passwords and to Printed The Console.
Please confirm that you would like to continue [y/N]y
Changed password for user kibana
PASSWORD kibana = nEeVHfxms4Q4S6mWmzzH
Changed password for user logstash_system
PASSWORD logstash_system = zmb8xXkFk7KlLWYPHfO0
Changed password for user beats_system
PASSWORD beats_system = ejSOoRx87tx43IfokIot
Elastic User password for the Changed
PASSWORD = Elastic RFWJ2dN0crlHk0ebUHN4
. 1
2
. 3
. 4
. 5
. 6
. 7
. 8
. 9
10
. 11
12 is
13 is
14
15
16
. 17
the credentials with other applications connected Elasticsearch credentials, save
Will be deployed to the application password
to edit the configuration file Kibana /etc/kibana/kibana.yml
elasticsearch.username: "Elastic"
elasticsearch.password: "RFWJ2dN0crlHk0ebUHN4"
1
2
This is the account password Kibana connection Elasticsearch credentials, if you can not use kibana account link .
The password is deployed to the collector
to metricbeat for example, modify the configuration file /etc/metricbeat/metricbeat.yml, to amend the relevant field installation in the following format, in accordance with their own password to modify the code above to fill:
output.elasticsearch:
hosts: [ "myEShost: 9200 "]
username:" beats_system "
password:" ejSOoRx87tx43IfokIot "
setup.kibana:
Host:" kibana_host: 5601 "
username:" kibana "
password:" nEeVHfxms4Q4S6mWmzzH "
. 1
2
. 3
. 4
. 5
. 6
. 7
. 8
to restart the application to refresh the configuration: systemctl restart metricbeat
Restart Kibana
restart the application to refresh the configuration: systemctl restart kibana.service
this time to refresh the page to see Kibana already need login in, this time using kibana account login just generated.
Six, x-pack crack
instructions
different from the x-pack installed manually, x-pack contained within Elasticsearch6.4 located modules / x-pack-core, namely: / usr / share / elasticsearch / modules / x-pack -core / x-pack-core- 6.4.2.jar file.
Decompile jar file
In this step, you can skip the process decompile directly using a Java program given in the article I compiled and replaced. But if necessary can transfer the files out and back on the jar with luyten compiler software package decompile, luyten Project address: https: //github.com/deathmarine/Luyten
Modified x-pack source
of our concern focused on projects in the two documents
org.elasticsearch.license.LicenseVerifier.java
org.elasticsearch.xpack.core.XPackBuild.java
previous two files compared to the version there are some changes, but does not affect the crack process. I will not put up the original file, given directly to Java modified file. You can create code gives a Java file with the same name in a local file and copied to the top, so you have two Java source files modified x-pack program.
org.elasticsearch.license.LicenseVerifier.java
Package Penalty for org.elasticsearch.license;
import java.nio.*;
import org.elasticsearch.common.bytes.*;
import java.security.*;
import java.util.*;
import org.elasticsearch.common.xcontent.*;
import org.apache.lucene.util.*;
import org.elasticsearch.core.internal.io.*;
import java.io.*;
public class LicenseVerifier
{
public static boolean verifyLicense(final License license, final byte[] publicKeyData) {
return true;
}
public static boolean verifyLicense(final License license) {
return true;
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
org.elasticsearch.xpack.core.XPackBuild.java
package org.elasticsearch.xpack.core;
import org.elasticsearch.common.io.*;
import java.net.*;
import org.elasticsearch.common.*;
import java.nio.file.*;
import java.io.*;
import java.util.jar.*;
public class XPackBuild
{
public static final XPackBuild CURRENT;
private String shortHash;
private String date;
@SuppressForbidden(reason = "looks up path of xpack.jar directly")
static Path getElasticsearchCodebase() {
final URL url = XPackBuild.class.getProtectionDomain().getCodeSource().getLocation();
try {
return PathUtils.get(url.toURI());
}
catch (URISyntaxException bogus) {
throw new RuntimeException(bogus);
}
}
XPackBuild(final String shortHash, final String date) {
this.shortHash = shortHash;
this.date = date;
}
public String shortHash() {
return this.shortHash;
}
public String date() {
return this.date;
}
static {
final Path path = getElasticsearchCodebase();
String shortHash = null;
String date = null;
Label_0157: {
shortHash = "Unknown";
date = "Unknown";
}
CURRENT = new XPackBuild(shortHash, date);
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31 is
32
33 is
34 is
35
36
37 [
38 is
39
40
41 is
42 is
43 is
44 is
45
46 is
47
48
49
50
java compiler program
you need two files compiled into class files compiled file dependency has been given in the command.
Compile LicenseVerifier.java
javac -cp "/ usr / report this content share / elasticsearch / modules / the X-Core-Pack-/ *: / usr / report this content share / elasticsearch / lib / *" LicenseVerifier.java
compiled XPackBuild.java
javac -cp "/ usr / share / elasticsearch / modules / x- pack-core / *: / usr / share / elasticsearch / lib / * "XPackBuild.java
Tip: If you compile the delay in the end, you may need to elasticsearch off and then recompile.
Compile replacement process needs to close Elasticsearch.
Getting the source
After the completion of the compiler need to add the compiled program to the x-pack-core-6.4.2.jar file, decompress first and then we can use to build a new replacement jar files in compressed form. Enter the file directory, backup file, copy it
cd / usr / report this content share / elasticsearch / modules / the X-Core-Pack-/
cp-Pack-the X-Core-6.4.2.jar-the X-Pack-Core-6.4.2.jar. BAK
CP-X-Pack Core-6.4.2.jar /home/x-pack-core-6.4.2.jar
. 1
2
. 3
-extracting jar file
extracting jar file: jar -xvf x-pack-core -6.4.2. the jar command will extract to the current directory
unzip x-pack-core-6.4.2.jar -d ./x-pack-core-6.4.2 this command to specify the directory
to replace the class file
will be compiled just cracked class replacement to the same position
CP LicenseVerifier.class ./x-pack-core-6.4.2/org/elasticsearch/license/
CP XPackBuild.class ./x-pack-core-6.4.2/org/elasticsearch/xpack/core /
re-packaged packet jar
compressed jar file:. jar -cvf x-pack- core-6.4.2.crack.jar -C x-pack-core-6.4.2 / ( periods that can not be ignored)
replace x-pack file
We will be generated to replace the cracked jar Elasticsearch directory:
cp-Pack-the X-Core-6.4.2.crack.jar / usr / report this content share / elasticsearch / modules / the X-Core-Pack-/ the X--Pack-core- 6.4.2.jar
Also note the need to replace the cluster all x-pack-core-6.4.2.jar
restart elasticsearch
systemctl restart elasticsearch.service
seven or upgrade to platinum
apply for a license
in a license application for the official website: https : //register.elastic.co/marvel_register
licenses for new versions of applications are ordinary, we need to change information in the file so that the software think we are platinum. And because we crack the jar file verification certificate, so the software can not verify our certificate is not true.
General certificate reads as follows:
{
"License": {
"uid": "c6570128-85c2-4f72-8d8f-b1425455b9ee",
"of the type": "Basic",
"issue_date_in_millis": 1.54008 trillion,
"expiry_date_in_millis": 1,571,702,399,999,
"max_nodes ": 100,
" ISSUED_TO "
"Signature": "AAAAAwAAAA07qIy5rp9i1qa5VS3vAAAB ...",
"start_date_in_millis": 1.54008 trillion
}
}
. 1
2
. 3
. 4
. 5
. 6
. 7
. 8
. 9
10
. 11
12 is
13 is
modified as Platinum type field indicates Platinum
modified expiry_date_in_millis 2147482800000 field indicating the end of time
to modify max_nodes field is 1000, represents the number of clusters
{
"license": {
"uid": "c6570128-85c2-4f72-8d8f-b1425455b9ee",
"type": "basic",
"issue_date_in_millis": 1540080000000,
"expiry_date_in_millis": 1571702399999,
"max_nodes": 100,
"issued_to": "elastic",
"issuer": "elastic",
"signature": "AAAAAwAAAA07qIy5rp9i1qa5VS3vAAAB...",
"start_date_in_millis": 1540080000000
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
注：“2147482800” 表示 “北京时间2038-1-19 11:00:00”
Update the license
after the successful modification can upload a new license in Kibana page
Note: You must use elastic account to log in to upload the license and license update