CDH添加KAFKA

 1. 在原有的CDH基础上添加KAFKA服务。系统环境：

centos 7

jdk 1.8.0_101

python 2.7.5

CDH 5.12.2

kerberos 1.15.1-19.el7

  2. kafka parcel包下载，分配和激活

地址：http://archive.cloudera.com/kafka/parcels/2.0.2/

  3.添加kafka服务

按照安装导向安装即可

另 

Destination Broker List（bootstrap.servers）：duan140

Source Broker List（source.bootstrap.servers）：duan139

 

　4.开通相关端口

firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="x.x.x.x" port protocol="tcp" port="9092" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="x.x.x.x" port protocol="tcp" port="9393" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="x.x.x.x" port protocol="tcp" port="9394" accept"
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="x.x.x.x" port protocol="tcp" port="24042" accept"

 

Q&A

1. whitelist must be specified when using new consumer in mirror maker.

配置中搜索whitelist ，添加

Topic Whitelist （whitelist）：duan140

 

   2. Invalid url in bootstrap.servers