Beats: Install and configure Metricbeat (1) - 8.x

In my previous article:

• Beats: Getting Started with Beats (1)
• Beats: Getting Started with Beats (2)

I describe in detail how to install and configure Beats on the Elastic Stack 7.x. Where installed, it usually does not come with security and Elasticsearch does not have HTTPS access by default. In the latest Elastic Stack 8.x, Elasticsearch clusters come with secure access by default. This accesses Elasticsearch for Metricbeat it is different from the previous situation. I described how Beats connects to Elasticsearch in my previous article " Elastic Stack 8.0 Installation - Securing Your Elastic Stack Is Now Easier Than Ever ". In today's article, we describe in detail how Beats connects securely to Elasticsearch.

In today's presentation, I will use the latest Elastic Stack 8.9.0 to demonstrate. I will use the following schema:

Install

Elasticsearch 及 Kibana

If you have not installed your own Elasticsearch and Kibana, please refer to my previous article:

• How to install Elasticsearch on Linux, MacOS and Windows

• Kibana: How to install Kibana in the Elastic stack on Linux, MacOS and Windows

When installing, please choose to use the Elastic Stack 8.x installation guide to install. During installation, we need to record the password and fingerprint information of its elastic superuser for later configuration:

✅ Elasticsearch security features have been automatically configured!
✅ Authentication is enabled and cluster connections are encrypted.

ℹ️  Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`):
  p1k6cT4a4bF+pFYf37Xx

ℹ️  HTTP CA certificate SHA-256 fingerprint:
  633bf7f6e4bf264e6a05d488af3c686b858fa63592dc83999a0d77f7e9fe5940

ℹ️  Configure Kibana to use this cluster:
• Run Kibana and click the configuration link in the terminal when Kibana starts.
• Copy the following enrollment token and paste it into Kibana in your browser (valid for the next 30 minutes):
  eyJ2ZXIiOiI4LjkuMCIsImFkciI6WyIxOTIuMTY4LjAuMzo5MjAwIl0sImZnciI6IjYzM2JmN2Y2ZTRiZjI2NGU2YTA1ZDQ4OGFmM2M2ODZiODU4ZmE2MzU5MmRjODM5OTlhMGQ3N2Y3ZTlmZTU5NDAiLCJrZXkiOiJ3WEE3MDRrQkxxWTFWWGY0QWRHbDpCa0VZVXZmaFFidWNPOFUxdXJwXzZnIn0=

ℹ️  Configure other nodes to join this cluster:
• On this node:
  ⁃ Create an enrollment token with `bin/elasticsearch-create-enrollment-token -s node`.
  ⁃ Uncomment the transport.host setting at the end of config/elasticsearch.yml.
  ⁃ Restart Elasticsearch.
• On other nodes:
  ⁃ Start Elasticsearch with `bin/elasticsearch --enrollment-token <token>`, using the enrollment token that you generated.

Note the fingerprint output above. This can be used to configure Beats in our later article.

In order to make our Kibana accessible by other computers, we must make a configuration for the kibana.yml file:

server.host: "0.0.0.0"

After making changes, we have to restart Kibana.

Metricbeat

For the installation of Metricbeat, we can refer to the official document  Metricbeat quick start: installation and configuration | Metricbeat Reference [8.9] | Elastic  to install. For most application scenarios, we can use the warehouse for installation. We can refer to the link  Repositories for APT and YUM | Metricbeat Reference [8.9] | Elastic  to install. First, we create a document as follows:

install.sh

#!/bin/bash
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -;
sudo apt-get -y install apt-transport-https;
echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list;
apt-get update;
apt-get install -y metricbeat;

chmod +x install.sh

We type the above command to turn the above script into an executable file, and type the following command:

sudo ./install.sh

This completes our Metricbeat installation.

Configure Metricbeat

After completing the configuration of Metricbeat, we enter the configuration directory of Metricbeat and configure its configuration file metricbeat.yml:

/etc/metricbeat/metricbeat.yml

parallels@ubuntu2004:/etc/metricbeat$ pwd
/etc/metricbeat
parallels@ubuntu2004:/etc/metricbeat$ ls
fields.yml  http_ca.crt  metricbeat.reference.yml  metricbeat.yml  modules.d

We need to make the following changes to the metricbeat.yml file:

/etc/metricbeat/metricbeat.yml

Set the above flag to true, then every time you modify metricbeat.yml, metricbeat.yml will be loaded automatically without restarting metricbeat.

We need to copy the Elasticsearch certificate to the current Metricbeat configuration directory. We can find this certificate file http_ca.crt file in the Elasticsearch installation directory of the macOS machine:

$ pwd
/Users/liuxg/elastic/elasticsearch-8.9.0/config/certs
$ ls
http.p12      http_ca.crt   transport.p12

We copy the above certificate to the Metricbeat installation directory:

root@ubuntu2004:/etc/metricbeat# pwd
/etc/metricbeat
root@ubuntu2004:/etc/metricbeat# ls
fields.yml  http_ca.crt  metricbeat.reference.yml  metricbeat.yml  modules.d

Let's continue to configure the metricbeat.ym file:

We need to modify the above part manually. Fill in the password of the elastic superuser and configure the certificate.

In order to verify whether our configuration has syntax errors, we can use the following command to detect:

metricbeat test config

root@ubuntu2004:~# metricbeat test config
Config OK

We next test the output. This part is to test that you can connect to Elasticsearch correctly. We use the following command:

metricbeat test output

root@ubuntu2004:~# metricbeat test output
elasticsearch: https://192.168.0.3:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 192.168.0.3
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 8.9.0

Obviously, our test was successful. It was able to connect to Elasticsearch correctly. This way our configuration is completely successful.

In the above, we found that it is a bad habit to write the user name and password into the file with hard coding in metricbeat, because someone can view the file and find the password and other information you use. We can use keystore to protect this information.

root@ubuntu2004:/etc/metricbeat# metricbeat keystore create
Created metricbeat keystore
root@ubuntu2004:/etc/metricbeat# metricbeat keystore add ES_USER
Enter value for ES_USER: 
Successfully updated the keystore
root@ubuntu2004:/etc/metricbeat# metricbeat keystore add ES_PASSWORD
Enter value for ES_PASSWORD: 
Successfully updated the keystore
root@ubuntu2004:/etc/metricbeat# metricbeat keystore list
ES_PASSWORD
ES_USER

With these configurations, we can re-modify the metricbeat.yml file:

After the modification, we use the following command to re-verify the output:

root@ubuntu2004:/etc/metricbeat# metricbeat test output
elasticsearch: https://192.168.0.3:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 192.168.0.3
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 8.9.0

Obviously our output was successful. In this way, our user name and password can not be exposed, even if the configuration file is seen by others.

start module

By default, the system module is started automatically. We can check it with the following command:

metricbeat modules list

root@ubuntu2004:~# metricbeat modules list
Enabled:
system

Disabled:
activemq
aerospike
airflow
apache
aws
awsfargate
azure
beat
beat-xpack
ceph
ceph-mgr
cloudfoundry
cockroachdb
...

 From the above display results, we can see that the system module is activated. We can also see it by looking at the modules.d directory under the current metricbeat:

From the above output, we can see that only the suffix of the system.yml file does not contain disabled. It indicates that this module is activated. In fact, we can directly remove the extension disabled of a certain module in this directory to start the module. We can also disable modules with the following commands, for example:

metricbeat modules disable system

The above command disables the system module. We can check it with the following command:

metricbeat modules list

We can restart the system module with the following command:

metricbeat modules enable system

root@ubuntu2004:/etc/metricbeat# metricbeat modules enable system
Enabled system

Start Metricbeat

Above, we have successfully configured Metricbeat. Next, we run the following setup command to load Dashboards and create corresponding pipelines, index templates, etc.:

metricbeat setup

The above command will run for a while. We can see that:

root@ubuntu2004:~# metricbeat setup
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

For all Metricbeat modules, we only need to run the above command once, even if the current module is not started. After the above command is completed, we can check it in Kibana:

We next run the following command to collect data:

service metricbeat start

 

From the above, we can see that the running status is normal. Let's go to Kibana to view the data:

 We can view it in index management:

 

Obviously this data is collected in the form of data stream. 

Well, today's article will be introduced here first. In a later article, I will continue to introduce how to create a user with less privileges to configure Metricbeat. The current elastic user is used for configuration, which is rarely done. Also, I'll explore using other forms of certificates to configure the output. Please continue reading the article " Beats: Install and configure Metricbeat (2) - 8.x ".