In today's article, we'll describe how to make our Elasticsearch start https services. In many cases this is very useful. Particularly in the area of security Elastic SIEM, we need to become Elasticsearch visit https access, so making our data more secure and reliable.

Installation Elastic Stack

First of all, we can follow the previous article " : Elastic rookie Start Guide to install Elasticsearch and Kibana". Once we installed Elasticsearch and Kibana, we were in localhost: 9200 and localhost: 5601 to see the output we want:

To activate the safety Elasticsearch

We can follow my previous article " : elasticsearch Elastic account security settings to set the security for our Elasticsearch". We can not create new users, just use the default super user elastic.

Once we set the above settings, when we start Kibana It requires us to enter a user name and password to log in. If you've seen the picture above, it means that our security account settings have been successful.

P12 production certificate

We can refer to link " the Generate Certificates ", in Elasticsearch installation directory, use the following command:

./bin/elasticsearch-certutil ca

$ pwd
/Users/liuxg/elastic9/elasticsearch-7.6.0
liuxg:elasticsearch-7.6.0 liuxg$ ./bin/elasticsearch-certutil ca
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.bouncycastle.jcajce.provider.drbg.DRBG (file:/Users/liuxg/elastic9/elasticsearch-7.6.0/lib/tools/security-cli/bcprov-jdk15on-1.61.jar) to constructor sun.security.provider.Sun()
WARNING: Please consider reporting this to the maintainers of org.bouncycastle.jcajce.provider.drbg.DRBG
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'ca' mode generates a new 'certificate authority'
This will create a new X.509 certificate and private key that can be used
to sign certificate when running in 'cert' mode.

Use the 'ca-dn' option if you wish to configure the 'distinguished name'
of the certificate authority

By default the 'ca' mode produces a single PKCS#12 output file which holds:
    * The CA certificate
    * The CA's private key

If you elect to generate PEM format certificates (the -pem option), then the output will
be a zip file containing individual files for the CA certificate and private key

Please enter the desired output file [elastic-stack-ca.p12]: 
Enter password for elastic-stack-ca.p12 :

In the above we accept the default file name, and enter a familiar password (for my case, I accept empty). We at Elasticsearch installation directory, we can see a production certificate file:

$ ls
LICENSE.txt          config               lib
NOTICE.txt           data                 logs
README.asciidoc      elastic-stack-ca.p12 modules
bin                  jdk.app              plugins

We then run the following command to generate a certificate:

bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

The above command will use our CA to generate a certificate elastic-certificates.p12:

$ pwd
/Users/liuxg/elastic9/elasticsearch-7.6.0
liuxg:elasticsearch-7.6.0 liuxg$ ls
LICENSE.txt              data                     logs
NOTICE.txt               elastic-certificates.p12 modules
README.asciidoc          elastic-stack-ca.p12     plugins
bin                      jdk.app
config                   lib

We put the above elastic-certificates.p12 certificate to copy into the config subdirectory under Elasticsearch installation directory.

$ pwd
/Users/liuxg/elastic9/elasticsearch-7.6.0
liuxg:elasticsearch-7.6.0 liuxg$ ls config/
elastic-certificates.p12 jvm.options              roles.yml
elasticsearch.keystore   log4j2.properties        users
elasticsearch.yml        role_mapping.yml         users_roles

We use the following command:

openssl pkcs12 -in elastic-stack-ca.p12 -out newfile.crt.pem -clcerts -nokeys

It will generate a file called newfile.crt.pem. We copyed this file to the config subdirectory under the Kibana installation directory:

$ pwd
/Users/liuxg/elastic9/kibana-7.6.0-darwin-x86_64
liuxg:kibana-7.6.0-darwin-x86_64 liuxg$ ls config/
apm.js                   kibana.yml
elastic-certificates.p12 newfile.crt.pem

Configuration Elasticsearch

We next Elasticsearch configuration in the config / elasticsearch.yml. We refer Elastic official documents " Encrypting Communication in elasticsearch ", we add the following configuration:

xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.authc.api_key.enabled: true
xpack.security.http.ssl.keystore.path: /Users/liuxg/elastic9/elasticsearch-7.6.0/config/elastic-certificates.p12
xpack.security.http.ssl.truststore.path: /Users/liuxg/elastic9/elasticsearch-7.6.0/config/elastic-certificates.p12

We restart Elasticsearch:

./bin/elasticsearch

So that our Elasticsearch have successfully run in https mode. We enter the address in the chrome HTTPS: // localhost: 9200 , we can see:

Clearly, we can no longer access the local address of 9200 as before. When the above screen appears, we enter the following string (in the case of the current chrome window):

thissisunsafe

We Elasticsearch into elastic and password before we set, then click the " Sign an In ":

We can see from the above, the correct time our Elasticsearch installation.

We can also use the Safari browser to open:

We click on the Show Details button:

Click on our visit this website link:

As above, before we enter the user when creating the security account name and password elastic, then we can access Elasticsearch:

If we use Postman, we can make the following configuration through the "Settings" in the certificate of inspection to avoid:

We turn off the switch on top of the SSL certificate verification:

After the above settings, we can access Elasticsearch have https in the Postman. Postman on how to access Elasticsearch, please refer to my previous article " Elastic: Postman use to access the Stack Elastic ."

Placed Kibana

In order to make our Kibana able to successfully access the Elasticsearch with https. We also need to do the appropriate configuration. We open the config / kibana.yml. Add the following settings:

elasticsearch.hosts: ["https://localhost:9200"]
elasticsearch.ssl.certificateAuthorities: ["/Users/liuxg/elastic9/kibana-7.6.0-darwin-x86_64/config/newfile.crt.pem"]
elasticsearch.ssl.verificationMode: none

We put on top of previously generated certificate newfile.crt.pem filled to the top of the path, and in order that we can easily access, we do not enable verificationMode for kibana.

When we finished the configuration, we restart kibana:

We enter the elastic user's password, we can enter into Kibana the interface:

In the above we can see that we have successfully entered into Kibana of the interface.

Beats the incoming data to https in the Elasticsearch

Since the introduction of https, then how do we put our incoming data to Elasticsearch in it? When you import data, we have to configure the certificate to the beats of the configuration file. We have to filebeat example. We Elasticsearch installation directory, enter the following command:

bin/elasticsearch-certutil cert --pem elastic-stack-ca.p12

The above command will generate called "certificate-bundle.zip" file.

$ pwd
/Users/liuxg/elastic9/elasticsearch-7.6.0
liuxg:elasticsearch-7.6.0 liuxg$ ls
LICENSE.txt            certificate-bundle.zip lib
NOTICE.txt             config                 logs
README.asciidoc        data                   modules
bin                    jdk.app                plugins

We can put a decompress the file and put inside ca.crt to extract filebeat installation subdirectory.

method one

Open filebeat profile filebeat.yml, and add the certificate information:

filebeat.yml

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["localhost:9200"]

  # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "123456"
  ssl.certificate_authorities: ["/Users/liuxg/elastic9/filebeat-7.6.0-darwin-x86_64/ca.crt"]
  ssl.verification_mode: none

In the above, you need to fill in your username and password, as well as the need to replace the path above their certification path.

Method Two

We can also produce your own certificate with the following command. We at Elasticsearch installation directory, in the elastic-stack-ca.p12 above premise has been generated, run the following command:

openssl pkcs12 -in elastic-stack-ca.p12 -out newfile.crt.pem -clcerts -nokeys

The above command will generate a file called newfile.crt.pem of. We copy this file to the installation directory filebeat and modify our filebeat.yml as follows:

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["localhost:9200"]

  # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "123456"
  ssl.certificate_authorities: ["/Users/liuxg/elastic9/filebeat-7.6.0-darwin-x86_64/newfile.crt.pem"]
  ssl.verification_mode: none

Run Filebeat

After modifying the above configuration, we start the system modules:

./filebeat modules enable system
./filebeat setup

$ ./filebeat setup
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite:true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
Setting up ML using setup --machine-learning is going to be removed in 8.0.0. Please use the ML app instead.
See more: https://www.elastic.co/guide/en/elastic-stack-overview/current/xpack-ml.html
Loaded machine learning job configurations
Loaded Ingest pipelines

We can run filebeat by the following command: