Kafka--kafka basic concept-ISR detailed explanation

Basic concept of kafka-ISR detailed explanation

Detailed ISR-ISR

It mainly talks about the problems in master-slave synchronization

When the green P1 receives the written data, it needs to be synchronized to the purple P1S1 and P1S2

How to ensure consistency?

Using In Sync Replicas is the ISR concept

Why inconsistent?

Because P1S1 sync data may take 50ms P1S2 may take 60ms

After synchronization, enter the ISR collection,

The synchronization time can be set to a specified time (tolerance time)

Enter the OSR collection that has not been synchronized within the specified time

The green part mainly supervises whether the purple part has timeout

Detailed explanation of ISR—HW–LEO

HW
: High Watermark, the high water mark, the consumer can only pull the message up to the high water mark
LEO: Log End Offset, the offset (offset) of the last record of the log file.
The ISR set is inseparable from the direct existence of HW and LEO Relationship

---

The entire article (0-9) can be seen aspartition

What consumers can really consume is 0-5, but after 6 and 6, they can not consume

But the log may have written to 9

Why?

This is related to ISR

For example, now the producer wants to write 3 and 4 (as shown in Figure-1), according to the rules, 3 and 4 will be written to the leader first, (as shown in Figure-2)

At this time, follow1 and follow2 may not synchronize 3 and 4, and it may enter the situation in Figure 3

kafka要求必须全部同步了，HW才能升高

At this time, HW cannot be increased