1.conf结尾的文件即可，如下图所示：
首先了解一下，logstash的过程，如下图所示：
从配置文件来表述过程如下所示：
然后进入bin目录下，进行执行conf文件加载进去bat文件中去。
然后进入bin目录下执行该文件如下，并执行成功；
logstash的架构设计：
根据打开文件的quene可知：
如果有多个不同的输入，由quene作为一个缓冲队列来分发到不同的pipelines,pipelines根据quene来获取input中的数据，
每个pipelines可以理解为一个线程，线程和线程之间是独立的，
看一下pipelines的结构以及自定的一些文件；
首先看一下pipelines的路径：
查看一下pipelines里面的定义格式和路径：
- pipeline.id: test 线程id
# pipeline.workers: 1 pipelines线程数以及filter_output的处理线程数，默认是CPU的核数
# pipeline.batch.size: 1 Batcher一次批量获取的待处理文档数，默认是125
# config.string: "input { generator {} } filter { sleep { time => 1 } } output { stdout { codec => dots } }"
这里是配置文件的输入，输出格式
# - pipeline.id: another_test
另一个进程数
# queue.type: persisted
队列的持久化操作；
(1)可以持久化到磁盘，如persisted，也可以持久化到内存memory。(内存调度过后即删除)
# path.config: "D:\ELK\logstash\logstash-6.4.0\config\logstash.conf"
//logstash进入文件的读取路径
#
# Available options:
#
# # name of the pipeline
# pipeline.id: mylogs
#
# # The configuration string to be used by this pipeline
# config.string: "input { generator {} } filter { sleep { time => 1 } } output { stdout { codec => dots } }"
#
# # The path from where to read the configuration text
# path.config: "/etc/conf.d/logstash/myconfig.cfg"
#
# # How many worker threads execute the Filters+Outputs stage of the pipeline
# pipeline.workers: 1 (actually defaults to number of CPUs)
#
# # How many events to retrieve from inputs before sending to filters+workers
# pipeline.batch.size: 125
#
# # How long to wait in milliseconds while polling for the next event
# # before dispatching an undersized batch to filters+outputs
# pipeline.batch.delay: 50
#
# # How many workers should be used per output plugin instance
# pipeline.output.workers: 1
#
# # Internal queuing model, "memory" for legacy in-memory based queuing and
persisted" for disk-based acked queueing. Defaults is memory
queue.type: memory
#
# # If using queue.type: persisted, the page data files size. The queue data consists of
# # append-only data files separated into pages. Default is 64mb
queue.page_capacity: 64mb //队列的容量，在logstash的内部，quene是用来做缓冲池的。
#
# # If using queue.type: persisted, the maximum number of unread events in the queue.
# # Default is 0 (unlimited)
queue.max_events: 0 //最大可处理的事件数目
#
# # If using queue.type: persisted, the total capacity of the queue in number of bytes.
# # Default is 1024mb or 1gb
queue.max_bytes: 1024mb //队列最大内存数目是1024MB
#
# # If using queue.type: persisted, the maximum number of acked events before forcing a checkpoint
# # Default is 1024, 0 for unlimited
queue.checkpoint.acks: 1024 //事件传输完成过后的验证，也就是ack进行传输完成的验证结果。
#
# # If using queue.type: persisted, the maximum number of written events before forcing a checkpoint
# # Default is 1024, 0 for unlimited
# queue.checkpoint.writes: 1024
#
# # If using queue.type: persisted, the interval in milliseconds when a checkpoint is forced on the head page
# # Default is 1000, 0 for no periodic checkpoint.
# queue.checkpoint.interval: 1000
#
# # Enable Dead Letter Queueing for this pipeline.
# dead_letter_queue.enable: false
#
# If using dead_letter_queue.enable: true, the maximum size of dead letter queue for this pipeline. Entries
# will be dropped if they would increase the size of the dead letter queue beyond this setting.
# Default is 1024mb
# dead_letter_queue.max_bytes: 1024mb
#
# If using dead_letter_queue.enable: true, the directory path where the data files will be stored.
# Default is path.data/dead_letter_queue
#
# path.dead_letter_queue:
启动输入，输出文件配置，如下所示：
D:\ELK\logstash\logstash-6.4.0\bin>logstash -f ..\config\logstash.conf
启动自己书写的配置文件
Sending Logstash logs to D:/ELK/logstash/logstash-6.4.0/logs which is now configured via log4j2.properties
进入log4j,进行记录
[2018-11-13T14:12:08,471][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-11-13T14:12:08,929][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.4.0"}
runner进行调动程序入口，进行启动。
中间还有一个过程也就是quene的过程进行一个多数据源接入的缓冲过程。
[2018-11-13T14:12:10,707][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
//启动pipelines.yml，也就是个配置文件，去启动每一个r任务，pipeline_id是每个任务的id
pipe.workers是线程数：也就是
[2018-11-13T14:12:11,101][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2018-11-13T14:12:11,110][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-11-13T14:12:13,304][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused: connect"}
[2018-11-13T14:12:13,324][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://localhost:9200"]}
The stdin plugin is now waiting for input:
[2018-11-13T14:12:13,730][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x7eef08f3 run>"}
[2018-11-13T14:12:13,794][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-11-13T14:12:14,159][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2018-11-13T14:12:18,321][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-11-13T14:12:20,347][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused: connect"}[2018-11-13T14:12:23,350][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-11-13T14:12:25,358][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused: connect"}
[2018-11-13T14:12:28,360][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-11-13T14:12:30,388][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused: connect"}
[2018-11-13T14:12:33,392][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-11-13T14:12:35,408][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused: connect"}
[2018-11-13T14:12:38,411][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
'