Loki builds a log collection system

Loki

What is Loki

Loki is a horizontally scalable and highly available multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost-effective and easy to operate. It does not index log content, but sets a set of tags for each log stream

Loki document URL: https://grafana.com/docs/loki/latest/

Download URL: https://github.com/grafana/loki/releases

Install loki

• Get the software package
  
• Unzip the package

unzip loki-linux-amd64.zip

[root@localhost loki]# ll
-rwxr-xr-x 1 root root 88166400 May 20  2020 loki-linux-amd64
-rw-r--r-- 1 root root 26432293 Feb 22 11:26 loki-linux-amd64.zip

• Write configuration file

vim loki.yaml
---
auth_enabled: false

server:
  http_listen_port: 3100

ingester:
  lifecycler:
    address: 192.168.0.181
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 5m
  chunk_retain_period: 30s
  max_transfer_retries: 0

schema_config:
  configs:
    - from: 2020-07-01
      store: boltdb
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 168h

storage_config:
  boltdb:
    directory: /loki/index

  filesystem:
    directory: /loki/chunks

limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h
  ingestion_rate_mb: 15

chunk_store_config:
  max_look_back_period: 0s

table_manager:
  retention_deletes_enabled: false
  retention_period: 0s

• Start loki

[root@localhost loki]# nohup ./loki-linux-amd64 -config.file=./loki.yaml >./loki.log 2>&1 &
[1] 108017
[root@localhost loki]# ps -fe|grep loki
root     108017  79258  9 16:00 pts/0    00:00:00 ./loki-linux-amd64 -config.file=./loki.yaml

Install promtail

Promtail is responsible for log collection, similar to logstash, filebeat, etc. in elk. If you collect docker container logs, you can use the docker plugin to collect container logs.

• Obtain the package and unzip
  
• Configure pormtail configuration file

vim promtail.yaml
---
server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: ./positions.yaml

clients:
  - url: http://192.168.0.181:3100/loki/api/v1/push

scrape_configs:
 - job_name: zzz
   static_configs:
   - labels:
      app: zzz
      host: 192.168.0.181
      env: prod
      __path__: /var/log/messages

• Start promtail

nohup ./promtail-linux-amd64 -config.file=./promtail.yaml > ./promtail.log 2>&1 &
[root@localhost loki]# ps -fe|grep promtail
root     113788  79258  1 17:00 pts/0    00:00:00 ./promtail-linux-amd64 -config.file=./promtail.yaml

Configure grafana to display logs

Collect docker container logs

• Confirm docker version

[root@localhost local]# docker --version
Docker version 18.09.7, build 2d0083d

Need to change docker's log-driver to loki 13 version, the change will fail

• Install plugin

docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions
[root@localhost local]# docker plugin list
ID                  NAME                DESCRIPTION           ENABLED
5df430512cf6        loki:latest         Loki Logging Driver   true

• Start the container test

docker run -itd -p 80:80 --log-driver=loki --log-opt loki-url="http://192.168.0.181:3100/loki/api/v1/push" --log-opt max-size=50m --log-opt max-file=10 nginx

Access nginx through the curl command, and then view it in grafana