ELK解决日志换行问题

我们在部署elk完后，在kibana上发现很多ERROR日志换行情况，结果会在kibana上看到多行的error日志

我们常出现的ERROR日志如下

[INFO][2020-03-22 22:37:05,064][org.apache.commons.httpclient.HttpMethodDirector]Retrying request
java.net.ConnectException: 拒绝连接 (Connection refused)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at sun.reflect.GeneratedMethodAccessor178.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:140)
	at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:125)
	at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
	at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
	at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
	at hry.utils.HttpClientWithJson.sendPostWithJson(HttpClientWithJson.java:39)
	at hry.api.moac.service.MoacService.getBlockNumber(MoacService.java:193)
	at hry.api.moac.MoacUtil.moacProductionTx(MoacUtil.java:313)
	at hry.quart.CoinQuart.moacProductionTx(CoinQuart.java:434)
	at sun.reflect.GeneratedMethodAccessor211.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hry.core.quartz.TaskUtils.invokMethod(tb:4)
	at hry.core.quartz.QuartzJob.execute(jc:191)
	at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
java.lang.RuntimeException: 接口连接失败！
	at hry.utils.HttpClientWithJson.sendPostWithJson(HttpClientWithJson.java:55)
	at hry.api.moac.service.MoacService.getBlockNumber(MoacService.java:193)
	at hry.api.moac.MoacUtil.moacProductionTx(MoacUtil.java:313)
	at hry.quart.CoinQuart.moacProductionTx(CoinQuart.java:434)
	at sun.reflect.GeneratedMethodAccessor211.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hry.core.quartz.TaskUtils.invokMethod(tb:4)
	at hry.core.quartz.QuartzJob.execute(jc:191)
	at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
java.lang.reflect.InvocationTargetException
	at sun.reflect.GeneratedMethodAccessor211.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hry.core.quartz.TaskUtils.invokMethod(tb:4)
	at hry.core.quartz.QuartzJob.execute(jc:191)
	at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
Caused by: java.lang.NullPointerException

为了让我们的error日志不换行，我们就需要引入multiline这个中间键

因为logstash本身不带，所以需要我们安装，在我们的logstash的bin目录下执行 ./logstash-plugin install  logstash-filter-multiline
如下图，出现successful则表示安装成功

[root@manage-host bin]# ./logstash-plugin install  logstash-filter-multiline
Validating logstash-filter-multiline
Installing logstash-filter-multiline
Installation successful

其次我们就需要重新配置logstash的配置文件

配置文件的filter部分如下

filter {
       multiline {
           pattern => "^\["       # 正则匹配以[开头的
           negate => true         # true:表示不匹配正则表达式时，false:匹配正则表达式时（negate相反的）
           what => "previous"     # 设置未匹配的内容是向前合并还是先后合并,previous向前合并，next向后合并
       }
}

至此就能看到我们的ERROR日志不换行的结果了