Preface
In the era of rapid development of big data, distributed technology is an indispensable technology for Internet companies, and kafka is a leader in distributed technology. Almost all Internet companies you can think of are using kafka. Kafka is a high Throughput distributed messaging system, because it can process a large amount of information extremely fast, it has always been favored by first-line Internet companies.
As a first-class Internet company in China, Tencent's "kafka hardcore notes", which was internally rated as a "star", is of great weight. After I stayed up late to eat it, I really feel that learning Kafka is enough to have it.
Friends who want this kafka note can receive my password "star" by private message! (There is a surprise at the end of the article!)
Let’s take a look at how this "kafka notes" that made me stay up late gnaws over me:
1. Know kafka from scratch
1.1 Basic concepts of Kafka
1.2 Configuration and installation
1.3 Consumption and production
1.4 How to configure server parameters
2. Producer
1. Client development
2. Principle analysis
Three, consumers
1. Consumers and consumer groups
2. Client development
- 2.1 Necessary parameter configuration
- 2.2 Subscription topic and partition
- 2.3 Deserialization
- 2.4 Message consumption
- 2.5 Displacement submission
2.6 Control switch or turn off consumption
2.7 Designated displacement consumption
2.9 Rebalance
2.10 Consumer interception
2.11 Multi-threaded implementation
2.12 Important consumer parameters
Fourth, the main body and partition
1. Subject management
2. First encounter with kafkaAdminClient
2.1 Basic usage
2.2 Subject legality verification
3. Partition management
4. How to choose the right number of partitions
Five, log storage
1. File directory layout
2. Evolution of log format
2.1 v0 version
2.2 v1 version
2.3 Message compression
2.4 Variable length fields
2.5 v2 version
3. Log Index
4. Log cleanup
5. Disk Management
Six, go deep into the server
1. Controller
2. Parameter decryption
Seven, in-depth client
1. Partition allocation strategy
2. Consumer Coordinator and Group Coordinator
3._Consumer_offsets analysis
4. Affairs
8. Reliability research
1. Copy analysis
2. Log synchronization mechanism
3. Reliability analysis
Nine, kafka application
1. Command line tools
2.kafaka Connect
3.Kafka Mirror Maker
4.Kafka Streams
10. Kafka monitoring
1. Source of monitoring data
1.1 OneMinuteRate
1.2 Obtaining monitoring indicators
2. Consumption lags behind
3. Synchronize failed partitions
4. Description of monitoring indicators
5. Monitoring module
11. Advanced applications
1. Message Broker
2. Selection of message middleware
12. Integration of Kafka and Spark
Friends who need this Tencent star "kafka note" can privately write my password "star" or add WeChat mxj94670 to receive it for free!
Surprise! Kafka star notes are the same set of "kafka source code analysis + actual combat collection"!
I believe that most people find that they have not fully grasped the meaning of the meaning in the process of studying and interviewing. They still have a little understanding of common problems. These are mainly some problems in the process of reading the source code:
1. Too many source files, do not know the point
2. The amount of source code is too large, countless times start to give up
3. The method is wrong, you will be confused when you encounter "technical" coding
4. Not systematic enough to categorize source files for reading
This "kafka source code analysis + actual combat book" of the same level can easily solve the above problems from in-depth source analysis to actual combat!
to sum up
There is always ups and downs on the way to improve yourself, but after reaching a certain level, you can always find a learning method that suits you. People who persist to the end will always be one step closer to success.
Friends who see here must be very eager to learn, I hope my article can help you, this "kafka hardcore notes" and the same set of "kafka source code analysis + actual combat book" private letter my password "star" or add WeChat mxj94670 Share with everyone for free!
Remember Sanlian! thank!