Posted in Tencent Cloud Technology Salon Subscribe

Full of dry goods, Tencent Cloud + Community Technology Salon Kafka Meetup Shenzhen Station ended successfully

Cloud + Introduction: On April 22, the Tencent Cloud + Community Technology Salon Kafka Meetup Shenzhen, hosted by Tencent Cloud and Kafka Community and co-organized by Open Source China, was held in Tencent Building. This event was full of dry goods, not only the past and present of Kafka And outlook, there is also an in-depth analysis of Kafka's technical architecture, as well as related development and operation and maintenance practices of Kafka-based application platforms. The event was extremely popular. Nearly 300 people participated in the exchange. Many participants said that the quality of the event was very high and it was a worthwhile trip.

live audience

The past, present and future of Apache Kafka

In 2010, LinkedIn started to develop Kafka, and by 2011, Kafka entered the Apache incubator and became the world's top open source project, and was quickly adopted by various enterprises. According to analysis, more than 70% of the world's top 500 companies are using Kafka. Rao Jun, the main author of Kafka, the chairman of the project committee and the co-founder of Confluent, first brought you the sharing of "The Past, Present and Future of Apache Kafka", and introduced the key nodes and prospects of Kafka along the way. He believes that "Kafka's future is not just a real-time data collection and transmission system. With the next development, it is more inclined to become a real-time data stream processing, exchange and sharing platform."

Jun Rao: The Past, Present and Future of Apache Kafka

Kafka cluster automation operation practice

Yang Yuan, senior engineer of Tencent Infrastructure Department, shared "Kafka Cluster Automation Operation Practice". He introduced the large-scale Kafka cluster fault node removal, new node addition, and inter-node data that the Tencent Cloud CKafka system needs to face during automatic operation. Tilting, dynamic upgrade and allocation of Kafka instances required by users, and unbalanced utilization of Borker resources, etc., and Tencent's corresponding solutions are also given.

Yang Yuan: Kafka cluster automation operation practice

Know the design and implementation of Kubernetes-based Kafka platform

The next guest to share, Bai Yuqing, is the person in charge of Zhihu Technology Center. He brought the "Design and Implementation of Zhihu's Kubernetes-based Kafka Platform", sharing the application of Kafka in Zhihu and why it is necessary to To build a Kafka platform based on K8S, how to build such a platform. According to Bai Yuqing, Zhihu has applied Kafka in log, data transmission and message queue services. Kafka has become an important component of Zhihu's technology platform and has been running smoothly for two years.

Bai Yuqing: Knowing Design and Implementation of Kubernetes-based Kafka Platform

After the coffee break, the participants asked questions around the sharing guests. The atmosphere was active and the technical atmosphere was strong.

coffee break questions

Application of CKafka in Face Recognition PaaS

Back to the second half of the sharing, first of all, Chen Xinyu, the head of data flow research and development of Shentong Shentong Cloud, brought "The Application of CKafka in Face Recognition PaaS". The large-scale image face recognition service of Geling Shentong uses CKafka optimized by Tencent Cloud, which is applied in various links such as business data import, caching, output, monitoring and log collection, etc. Face feature extraction, comparison and analysis provide guarantees and are the core of the real-time face recognition platform in image streams. Chen Xinyu introduced the position and importance of Kafka in the deep learning face recognition platform architecture from the application level, and introduced the application and problems of some Kafka ecological components such as connect.

Chen Xinyu: Application of CKafka in Face Recognition PaaS

Kafka high-performance decryption and optimization

Finally, Yan Yanfei, senior engineer of Tencent Cloud Infrastructure Department, shared "Kafka High Performance Decryption and Optimization", deeply analyzed Kafka's technical architecture, and shared the reasons for Kafka's high performance from the aspects of storage and message format, and also optimized its performance. Some insights are offered.

Yan Yanfei: Kafka high-performance decryption and optimization

Tencent Cloud + Community Technology Salon Kafka Meetup is a Kafka offline technology sharing event hosted by Tencent Cloud + Community and Kafka Community. This is the first time that Kafka Meetup is held in Shenzhen, co-organized by Open Source China. The event not only expanded the influence of the Kafka open source community, but also triggered developers' further thinking on the application scenarios and future trends of Kafka.