Debezium 将MongoDB同步到kafka（待完善验证）

版权声明：本文为博主原创文章,转载请注明出处 https://blog.csdn.net/vkingnew/article/details/89156768

-- 登录mongo查询：
./mongo --host 10.19.166.249 --port 27017
testrs:PRIMARY> rs.status()

配置文件：
#vim mongodb.properties
name=mongo
connector.class=io.debezium.connector.mongodb.MongoDbConnector
mongodb.hosts="testrs/10.19.166.249:27017"
#mongodb.name=""
#mongodb.user=
#mongodb.password=
#mongodb.ssl.enabled=false
#mongodb.ssl.invalid.hostname.allowed=false
database.whitelist=
#database.blacklist=
#collection.whitelist=
#collection.blacklist=
snapshot.mode=initial
#field.blacklist=
#field.renames=
tasks.max=1
initial.sync.max.threads=1
tombstones.on.delete=true
snapshot.delay.ms=1
# Advanced options
max.queue.size=8192
max.batch.size=2048
poll.interval.ms=1000
connect.backoff.initial.delay.ms=1000
connect.backoff.max.delay.ms=1000
connect.max.attempts=16
mongodb.members.auto.discover=false 
#default:true
--报错信息：
java.lang.IllegalArgumentException: Maximum delay must be greater than initial delay

--测试数据：
3.生成测试数据：

for (i = 1; i < 21; i++) {
       db.users.insert({
	   "userid": i, "username": "user" + i, "age": Math.floor(Math.random()*120), "sex": Math.floor(Math.random()*2), "createtime": new Date()	   
	   })
	  }

TiDB中预先定义对应的表名和表结构：

 CREATE TABLE `users` (
  `_id` varchar(32) DEFAULT NULL,
  `userid` int(11) DEFAULT NULL,
  `username` varchar(16) DEFAULT NULL,
  `age` int(11) DEFAULT NULL,
  `sex` int(11) DEFAULT NULL,
  `createtime` varchar(36) DEFAULT NULL
);

4.更新数据：

 db.users.update({"username" : "user20"},{$set:{"username":"MongoDB"}},{multi:true})


5.删除数据：
db.users.remove({"userid":null})
db.users.remove({"userid":2})


参考：
https://debezium.io/docs/configuration/mongodb-event-flattening/
https://debezium.io/docs/connectors/mongodb/
https://github.com/debezium/debezium-examples/tree/master/tutorial#using-mongodb