kafka add SASL authentication

kafka version information: kafka_2.12-2.3.0

Use kafka comes zookeeper start

bin/zookeeper-server-start.sh config/zookeeper.properties

kafka start:

bin/kafka-server-start.sh config/server.properties

nohup bin / kafka-server-start.sh config / server.properties> logs / server-start.log 2> & 1 &
What is into the background, without the nohup, logs are output to the console above. Which, server-start.log is to write a log file, the original file logs the following is not.

SASL configuration steps:

1. Modify the bin / server.properties file

############################# Socket Server Settings #############################

# The address the socket server listens on. It will get the value returned from
# java.net.InetAddress.getCanonicalHostName() if not configured.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
listeners=SASL_PLAINTEXT://ip:9092

# Hostname and port the broker will advertise to producers and consumers. If not set,
# it uses the value for "listeners" if configured.  Otherwise, it will use the value
# returned from java.net.InetAddress.getCanonicalHostName().
advertised.listeners=SASL_PLAINTEXT://ip:9092

# Maps listener names to security protocols, the default is for them to be the same. See the config documentation for more details
#listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_SSL:SASL_SSL
其他值未做任何改变

2. Create config / kafka_server_jaas.conf

{KafkaServer
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username = "ADMIN"
    password = "ADMIN"
    user_admin = "ADMIN"
    user_alice = "Alice";
};
New config / kafka_client_jaas.conf - This step may not be built, because I want to use the console to consumption data

KafkaClient {
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="admin"
        password="admin";
};

3.修改bin/kafka-server-start.sh

if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G -Djava.security.auth.login.config=/home/zhufei/software/kafka_2.12-2.3.0/config/kafka_server_jaas.conf"
fi

Modify bin / kafka-console-consumer.sh kafka-console-producer.sh

if [ "x$KAFKA_OPTS" ]; then
 export KAFKA_OPTS="-Djava.security.auth.login.config=/home/zhufei/software/kafka_2.12-2.3.0/config/kafka_client_jaas.conf"
fi

Modify config / consumer.properties producer.properties

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN

The red part of the actual modification can not, because I want to use the console consumption data

4. Start the console consumers

bin/kafka-console-consumer.sh --bootstrap-server 192.168.3.8:9092 --topic test --from-beginning --consumer.config config/consumer.properties

5.java application acts as a producer:

public class TestDemo {

   public static void main(String[] args) throws Exception {
      System.setProperty("java.security.auth.login.config", "/home/zhufei/software/kafka_2.12-2.3.0/config/kafka_client_jaas.conf"); // 环境变量添加，需要输入配置文件的路径

 Properties props = new Properties();
 props.put("bootstrap.servers", "ip:9092");
// props.put("acks", "all");
 props.put("retries", 3);
 props.put("batch.size", 16384);
// props.put("linger.ms", 1);
// props.put("buffer.memory", 33554432);
 props.put("key.serializer", StringSerializer.class.getName());
 props.put("value.serializer", StringSerializer.class.getName());
// props.put("partitioner.class", HashPartitioner.class.getName());
// props.put("interceptor.classes", EvenProducerInterceptor.class.getName());

 props.put("security.protocol", "SASL_PLAINTEXT");
 props.put("sasl.mechanism", "PLAIN");

 Producer<String, String> producer = new KafkaProducer<String, String>(props);
// for (int i = 0; i < 10; i++)
 producer.send(new ProducerRecord<String, String>("test", null, "hello world 20190909 fox"));
 producer.close();
 }

}



7. Result: The normal sending, normal consumption 
zhufei @ SilverRiver: ~ / software / kafka_2.12-2.3.0 $ bin / kafka-console-consumer.sh --bootstrap-server ip: 9092 --topic test --from --consumer.config config -beginning / consumer.properties 
the Hello world 20,190,909 Fox