What can be optimized for Flume in the process of data collection

  • Flume's double-layer collection (first-tier data collection, second-tier data aggregation)
  • Channel can use Kafka channel, which has the advantages of File Channel and Memory Channel; it can also have both offline and real-time computing.
  • Flume is replaced by a lightweight log collection component logstash (L in the ELK suite), or a lighter set of filebeat
  • Flume interceptor, you can take extra data into consideration and only keep the Json string

Guess you like

Origin blog.csdn.net/abc5254065/article/details/112853537