logstash be considered together with the project

When using Node.js development project, we used  log4js  module for logging, you can configure log4js of  Appenders  will log output to different places Console, File and GELF and so on.

logstash

logstash is  elastic  technology stack in which one often used to collect and parse logs. It is a relatively common practice in project engineering in the logs are written to the log file, and then read and parse the log files logstash. For example, in Node.js project, to logging to a file, just to make the following configuration can be log4js:

const log4js = require('log4js');
log4js.configure({
  appenders: [{
    type: 'file', filename: './example.log' }] }); const logger = log4js.getLogger(); logger.info('hello');

This will be recorded in the log  example.log file, and then follows the configuration of the input logstash:

file {
    path => "YourLogPath/example.log"
    start_position => "beginning"
  }
}

Logstash so you can read the log files. But this always felt a bit with a file as a transit trouble, if the log can log4js generated directly to logstash better.

logstashUDP

log4js logstashUDP built directly output to log into logstash. Configuration is as follows:

log4js.configure({
  appenders: [{
    type: "logstashUDP",
    host: "localhost", port: 12345 }] });

Logstash then arranged into the following:

input {
    udp {
    host => "127.0.0.1" port => 12345 } }

Ah, a very simple thing! Now you can log log4js generated directly to logstash, while no longer need to use a file as a transit. But when I found that when using a problem that if logstash service hung up, this time log4js still going to generate log data, this time the first thought is to logstash restart it, but after the restart only to find logstash did not to get data when logstash hang log4js generated, that is, if logstash hung up, then log4js data generated will be lost and will not be processed.
This time with an agent to think log4js temporary data generated, log4js outputs the data to the agent, logstash agents that read data from, logstash a data read, the agent will discard the piece of data. Yes, that is the queue. So that no data loss problem.

log4js-logstash-repeat

log4js-logstash-redis  in order to solve the above problems of the Appender a log4js.
His principle is: the resulting log output log4js to redis, then let logstash read data from redis. But direct call log4js-redis just fine Well, why is it called log4js-logstash-redis it? This is because the log format for logstash done some more friendly customization. ?

installation

npm install log4js-logstash-redis --save

use

log4js.configure({
  appenders: [{
    type: "log4js-logstash-redis",
    key: "test", redis: { db: 1 } }] });

redis optional configuration, the connection is not, then redis default value. logstash is as follows:

redis {
        data_type => "list"
        key => "test"
        codec => json
}

Wherein the value of data_type must be a list.

Original Address: using redis generated to the log log4js logstash