Big data open source tools-components

Upload the offline package to FTP for use

1. Docker installation and use:

(1) Unzip
[root@tingnan-DB3 ~]# tar xzvf docker-18.03.1-ce.tgz

(2) Move all the contents of the docker folder to the /usr/bin/ directory
[root@tingnan-DB3 ~]# sudo cp docker/* /usr/bin/

(3) Start the docker daemon

[root @ view-DB3 ~] # sudo dockerd &

(4) docker version test

[root @ view-DB3 ~] # docker -v

Set the reference website for docker to start automatically after booting: https://www.cnblogs.com/luoSteel/p/10038954.html

2. Docker service operation monitoring tool deployment and installation

[root@tingnan-DB3 ~]# tar xvzf prometheus-2.6.1.linux-amd64.tar.gz
Move to the installation directory:
[root@tingnan-DB3 ~]# mv prometheus-2.6.1.linux-amd64 /usr /local/prometheus

Enter the directory:
[root@tingnan-DB3 ~]# cd /usr/local/prometheus

Modify the monitoring machine at the bottom of the configuration file:
[root@tingnan-DB3 prometheus]# vim prometheus.yml

scrape_configs:

  • job_name: ‘prometheus’
    static_configs:

    Monitor local and port

    • targets: [‘xiangsikai:9090’]

Configuration file annotation

Note: Prometheus pulls sampling point data from the target machine through http. It can also pull its own service data and monitor its own health status.

Note: Of course, the Prometheus service pulls its own service sampling data, which is not very useful, but it is a good DEMO.
global:
####### By default, the target sampling point data is pulled every 15s.
scrape_interval: 15s
####### We can attach some designated tags to the sampling point measurement tag list for communication with third-party systems, including: federation, remote storage, Alertmanager
external_labels:
####### The following is to pull the data configuration
monitor of its own service sampling point :'codelab-monitor'
scrape_configs:
####### The job name will be added to all the sampling points pulled, and there is also a host of the instance target service: The port label will also be added to the sample point

  • job_name:'prometheus'
    ####### Override global sampling points, pull time interval 5s
    scrape_interval: 5s
    static_configs:
    • targets: [‘localhost:9090’]

Start service

./prometheus --config.file=prometheus.yml

Specify the configuration file

–Config.file=“prometheus.yml”
###Specify the listening address port
–web.listen-address=“0.0.0.0:9090”

Maximum number of connections

–web.max-connections=512

The directory where tsdb data is stored, the default current data/

–storage.tsdb.path=“data/”

The time for premetheus to store data, the default is 15 days

–storage.tsdb.retention=15d

Startup options to understand: ./prometheus --help

Other big data tools

node.js -----node-red

Grafana -----Visual monitoring indicator display tool

dataX --------Data extraction tool

DolphinScheduler -------- Scheduling tool

Guess you like

Origin blog.csdn.net/tandelin/article/details/106562953