structured Streaming与mysql的集成

对于structured Streaming目前版本,没有很好的对mysql等关系型数据库集成的API,所以需要以其它的方式实现

博客地址:https://databricks.com/blog/2017/04/04/real-time-end-to-end-integration-with-apache-kafka-in-apache-sparks-structured-streaming.html(除了DataBase还包括其它的集成)

Databases

Often times we want to be able to write output of streams to external databases such as MySQL. At the time of writing, the Structured Streaming API does not support external databases as sinks; however, when it does, the API option will be as simple as .format("jdbc").start("jdbc:mysql/.."). In the meantime, we can use the foreach sink to accomplish this. Let’s create a custom JDBC Sink that extends ForeachWriter and implements its methods.

We can now use the JDBCSink:

猜你喜欢

转载自blog.csdn.net/weixin_42744795/article/details/82109663