4) Set the boot script automatically executed
···
vi /etc/rc.d/rc.local # editor, add a line at the end
sh /usr/local/kafka/kafkastart.sh & # script is set to start automatically in the background
···
3-1 kafka basic concepts
2018-11-26
-
Physical Concepts: physical isolation level, such as a database, a server device, etc.
-
Logical concepts: the concept of the code / policy logic level
-
Producer: progress message and producer data, send messages to a topic kafka's / Code / Services
-
Consumer: Consumer news and data, subscription data (Topic) and process messages issued by its processes / Code / Services
-
Consumer Group: logical concept, for the same Topic, broadcast to a different group, a group, only one consumer can consume the message
-
broker: physical concepts, kafka kafka each node in the cluster
-
topic: logical concepts, categories kafka message, data distinction, isolated
-
partition: physical concepts, the basic unit of data storage kafka. A Topic data, will be dispersed into a plurality of memory partition, each partition is ordered
-
Replication: the same Partition may have multiple Replica, Replica of data between multiple is the same as (the equivalent of a partition has multiple copies)
-
Replication Leader: multiple Replica of a Partition, you need a Leader responsible for Partition Producer and Consumer interact with (and only one, the other is responsible for synchronizing data, a good spare tire)
-
ReplicaManager: information on the current broker is responsible for managing all partitions and replicas, processing some requests kafkaController initiated, a copy of the state of the switch, add / read messages, etc.
Partition
1. Each is cut into a plurality of Partitions Topic
partition belong to the basic unit of information storage. topic like classification, there are multiple units (partition) a lower classification.
2. The number of consumer data is less than or equal to Partition
3.Broker Group Broker stored in each of a plurality of Partitions or Topic
// broker is a storage node kafaka of. A plurality of nodes Broker Group
4.Consumer Group Consumer in only one or a plurality of read Topic parttition, and is the only Consumer. A group scheduling manner topic
Time Complexity: it is operating with several changes and changes
Space complexity: actual data row is calculated
rw random write random access
sw sequenue write sequential read and write
Send message storage partitions kafka
有 key hash
No key poll