在网上找了好多，这里只说一下logstash同步数据到elasticsearch
首先：安装logstash（这个比较简单，就不说了）
编写配置文件（主要） 命名为****.conf
内容：
input {
stdin {
}
jdbc {
type => "book"
jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/dataBaseName?/characterEncoding=UTF-8&;useSSL=false"
jdbc_user => "root"
jdbc_password => "root"
jdbc_driver_library =>"D:\ES\logstash-6.2.3\bin\mysql\mysql-connector-java-5.1.46.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
#添加增量类配置
record_last_run => "true"
use_column_value => "true"
tracking_column => "id"
tracking_column_type => numeric
clean_run => "false"
jdbc_page_size => "50000"
statement_filepath => "D:\ES\logstash-6.2.3\bin\mysql\book.sql"
schedule => "* * * * *"
}
jdbc {
type => "users"
jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/dataBaseName?/characterEncoding=UTF-8&;useSSL=false"
jdbc_user => "root"
jdbc_password => "root"
jdbc_driver_library =>"D:\ES\logstash-6.2.3\bin\mysql\mysql-connector-java-5.1.46.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
record_last_run => "true"
use_column_value => "true"
tracking_column => "id"
tracking_column_type => numeric
clean_run => "false"
jdbc_page_size => "50000"
statement_filepath => "D:\ES\logstash-6.2.3\bin\mysql\users.sql"
schedule => "* * * * *"
}
}
filter {
json {
source => "message"
remove_field => ["message"]
}
}
output {
stdout {
codec => json_lines
}
elasticsearch {
hosts => "127.0.0.1:9200"
index => "book"
document_id => "p%{id}"
}
if[type]=="book"{
elasticsearch {
#ESIP地址与端口
hosts => "localhost:9200"
#ES索引名称（自己定义的）
index => "book-increment"
#文档类型
document_type => "book"
#文档类型id
document_id => "%{id}"
}
}
}
if[type]=="users"{
elasticsearch {
hosts => "localhost:9200"
index => "users-increment"
document_type => "users"
document_id => "%{id}"
}
}
}
这个是同步两个表中的信息到es中，如果只需同步一张表那就只写一个jdbc{}就ok了。
其中"D:\ES\logstash-6.2.3\bin\mysql\mysql-connector-java-5.1.46.jar"是mysql-connector-java-5.1.46.jar在本机的存放位置。
"D:\ES\logstash-6.2.3\bin\mysql\users.sql"这个是具体的sql语句存放的位置。"D:\ES\logstash-6.2.3\bin\mysql\users.sql"的内容如下：
select * from book
或者根据自己的情况，只将自己需要的字段同步到es中。
以上全部工作完成后：再logstash的安装目录下的bin目录下执行：logstash -f ****.conf 即可，如果在这一步遇到问题：主类找到到，请点击链接：https://blog.csdn.net/lilamei170607/article/details/81780048