Flink 1.17 Tutorial: Output Operator Output to MySQL (JDBC)

Output to MySQL (JDBC)

The test steps for writing data to MySQL are as follows.
(1) Add dependencies.
Add the MySQL driver:

mysql
mysql-connector-java
8.0.27

. The official dependency of flink-connector-jdbc 1.17.0 has not yet been provided. For the time being, download it from the apache snapshot warehouse. Specify the warehouse path in the pom file:


apache -snapshots
apache snapshots
https://repository.apache.org/content/repositories/snapshots/Add


dependency:

org.apache.flink
flink-connector-jdbc
1.17-SNAPSHOT

If it does not take effect, you also need to modify the local maven configuration file, mirrorOf Add the following red content:

aliyunmaven
*,!apache-snapshots
Alibaba Cloud public warehouse
https://maven.aliyun.com/repository/public

(2) Start MySQL and create table ws under the test library

mysql>     
CREATE TABLE `ws` (
  `id` varchar(100) NOT NULL,
  `ts` bigint(20) DEFAULT NULL,
  `vc` int(11) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8

(3) Write sample code output to MySQL

public class SinkMySQL {
    
    
    public static void main(String[] args) throws Exception {
    
    
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        env.setParallelism(1);


        SingleOutputStreamOperator<WaterSensor> sensorDS = env
                .socketTextStream("hadoop102", 7777)
                .map(new WaterSensorMapFunction());


        /**
         * TODO 写入mysql
         * 1、只能用老的sink写法： addsink
         * 2、JDBCSink的4个参数:
         *    第一个参数： 执行的sql，一般就是 insert into
         *    第二个参数： 预编译sql， 对占位符填充值
         *    第三个参数： 执行选项 ---》 攒批、重试
         *    第四个参数： 连接选项 ---》 url、用户名、密码
         */
        SinkFunction<WaterSensor> jdbcSink = JdbcSink.sink(
                "insert into ws values(?,?,?)",
                new JdbcStatementBuilder<WaterSensor>() {
    
    
                    @Override
                    public void accept(PreparedStatement preparedStatement, WaterSensor waterSensor) throws SQLException {
    
    
                        //每收到一条WaterSensor，如何去填充占位符
                        preparedStatement.setString(1, waterSensor.getId());
                        preparedStatement.setLong(2, waterSensor.getTs());
                        preparedStatement.setInt(3, waterSensor.getVc());
                    }
                },
                JdbcExecutionOptions.builder()
                        .withMaxRetries(3) // 重试次数
                        .withBatchSize(100) // 批次的大小：条数
                        .withBatchIntervalMs(3000) // 批次的时间
                        .build(),
                new JdbcConnectionOptions.JdbcConnectionOptionsBuilder()
                        .withUrl("jdbc:mysql://hadoop102:3306/test?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=UTF-8")
                        .withUsername("root")
                        .withPassword("000000")
                        .withConnectionCheckTimeoutSeconds(60) // 重试的超时时间
                        .build()
        );


        sensorDS.addSink(jdbcSink);


        env.execute();
    }
}

（4） 运行代码，用客户端连接MySQL，查看是否成功写入数据。