Kafka configuration 5-Kafka cluster + SASL + SSL under Windows

According to the first 4 articles, we can build Kafka environment from scratch, including cluster, SASL, and SSL certificate configuration. This article will not explain in detail. For detailed instructions, please see the following 4 articles. This article is mainly This complete set of configuration is shown to everyone in the simplest way of explanation.

The complete configuration process is as follows

1. Zookeeper
1.1, modify zoo.cfg configuration

authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000

# 存放数据
dataDir=D:/Net_Program/Net_Zookeeper/data-file
# 存放日志
dataLogDir=D:/Net_Program/Net_Zookeeper/data-log
# 监听端口
clientPort=2181
# 配置集群服务
server.1=192.168.2.200:2881:3881
server.2=192.168.2.200:2882:3882
server.3=192.168.2.200:2883:3883

The above is the setting for one machine, if it is the second, third..., modify the three parameters dataDir, dataLogDir, clientPort in turn .

   1.2, create myid file
       in D: \ Net_Program \ Net_Zookeeper \ data -file in a new name for the myid files (no extension), the content is 1
       in D: \ Net_Program \ Net_Zookeeper2 \ data -file in a new name for the myid file ( No suffix), the content is 2
       Create a new file named myid in D:\Net_Program\Net_Zookeeper3\data-file (without suffix), the content is 3

If there are more Zookeeper configurations, create them one by one according to the above rules

1.3. Create a new zk_server_jaas.conf
in the D:\Net_Program\Net_KafkaConfig folder (used to store the account and password configuration files of Zookeeper and Kafka) and create a new zk_server_jaas.conf file with the following content:

KafkaServer {
	org.apache.kafka.common.security.plain.PlainLoginModule required
	username="admin"
	password="admin123456"
	user_admin="admin123456"
	user_quber="quber123456"
	user_quber1="quber123456"
	user_quber2="quber123456"
	user_scyjTestUser1="scyj&cdkx"
	user_scyjTestUser2="scyj&cdkx"
	user_scyjTestUser3="scyj&cdkx";
};

1.4. Modify zkEnv.cmd
in the Zookeeper installation directory bin, open zkEnv.cmd for editing, in the file set ZOO_LOG4J_PROP=INFO, add the following configuration to the next line of CONSOLE:

set SERVER_JVMFLAGS=-Djava.security.auth.login.config=D:/Net_Program/Net_KafkaConfig/zk_server_jaas.conf

Similarly, if there is a second, third..., you need to modify the above configuration

   1.5. Import the relevant jars
       . Find the following jars in the Kafka installation directory libs respectively, copy them to the lib in the Zookeeper installation directory to
           kafka-clients-2.4.0.jar
           lz4-java-1.6.0.jar
           slf4j-api-1.7 .28.jar
           slf4j-log4j12-1.7.28.jar
           snappy-java-1.1.7.3.jar

2. Kafka
2.1, modify server.properties configuration

# kafka消息存放的路径
log.dirs=D:/Net_Program/Net_Kafka/kafka-data

listeners=SASL_SSL://192.168.2.200:9092
advertised.listeners=SASL_SSL://192.168.2.200:9092
# 使用的认证协议
security.inter.broker.protocol=SASL_SSL
# SASL机制
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
# 完成身份验证的类
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
# 如果没有找到ACL（访问控制列表）配置，则允许任何操作。
#allow.everyone.if.no.acl.found=true
super.users=User:admin

# 唯一标识
broker.id=0
host.name=192.168.2.200
# 监听端口
port=9092
# 对应着3台Zookeeper的IP地址和端口
zookeeper.connect=192.168.2.200:2181,192.168.2.200:2182,192.168.2.200:2183

ssl.keystore.location=D:/Net_Program/Net_KafkaSsl/kafka1.keystore.jks
ssl.keystore.password=qubernet
ssl.key.password=qubernet
ssl.truststore.location=D:/Net_Program/Net_KafkaSsl/ca.truststore.jks
ssl.truststore.password=qubernet
ssl.keystore.type=JKS
ssl.truststore.type=JKS
ssl.client.auth=required
ssl.secure.random.implementation=SHA1PRNG
security.inter.broker.protocol=SASL_SSL

The above is the setting for one machine, if it is the second, third..., modify the five parameters log.dirs, listeners, advertised.listeners, broker.id, port in turn.

2.2. Create a new kafka_server_jaas.conf
in the D:\Net_Program\Net_KafkaConfig folder (used to store the account and password configuration files of Zookeeper and Kafka) to create a new kafka_server_jaas.conf file, the content is as follows:

KafkaServer {
	org.apache.kafka.common.security.plain.PlainLoginModule required
	username="admin"
	password="admin123456"
	user_admin="admin123456"
	user_quber="quber123456"
	user_quber1="quber123456"
	user_quber2="quber123456"
	user_scyjTestUser1="scyj&cdkx"
	user_scyjTestUser2="scyj&cdkx"
	user_scyjTestUser3="scyj&cdkx";
};

2.3. Modify kafka-server-start.bat
in the Kafka installation directory bin\windows, open kafka-server-start.bat for editing, and add the following configuration to the next line of the SetLocal keyword in the file:

set KAFKA_OPTS=-Djava.security.auth.login.config=D:/Net_Program/Net_KafkaConfig/kafka_server_jaas.conf

Similarly, if there is a second, third..., you need to modify the above configuration

Kafka configuration 5-Kafka cluster + SASL + SSL under Windows

Guess you like