1 Overview
This article is just an accumulation
Reprint: Real-time log data is written to Clickhouse
2. Background
Real-time querying of tens of billions of log data every day is a challenge. Kafka + Flink + Clickhouse + Redash is used in the architecture design to realize real-time analysis of massive data. At the computing layer, we have developed a real-time data platform based on the Flink computing engine to simplify the development process. The data is configured to achieve dynamic Schema generation, and the underlying data is unified. There is no need to repeat the wheel. The entire data link, from data collection and conversion, Storage, visualization, no need to write a line of code, configuration is complete. This article mainly introduces the practice of writing real-time log data to Clickhouse.
Flink Clickhouse Sink
<dependency>