Architecture design of the system log

Use the log
  • Daily Archive: available for download in different service module name.
  • Real-time analysis: Provides multi-dimensional analysis, troubleshooting and alarm monitoring.
  • Calculation of polymerization: on-demand data results.
  • Offline Analysis: complex data model analysis, regular reporting.
A common way to collect logs
  • Script: The log file regular cut or quantitative reporting.
  • Logstash
  • Heka Agent
  • Nginx
  • nsqd
What are unhealthy log
  • Slower rate request
  • The rear end node retry request
  • The number of bytes sent and file length Content-Length inconsistency
  • Abnormal response
Calculated on log aggregation
  • Total amount of collected metrics CDN edge request, real time reporting, the polymerization is calculated, the query
  • Support for multiple time dimension data show
  • Support domain name, service name, user name query
  • Support complex queries (Top N, list)

Mark。

Published 130 original articles · won praise 5 · views 20000 +

Guess you like

Origin blog.csdn.net/LU_ZHAO/article/details/104843195