Use sqoop to export the data in hive to mysql. One case is that the mapreduce program is stuck, INFO mapreduce.Job: map 100% reduce 0%
Another situation is: report this error: java.sql.SQLException: Incorrect string value: '\xF0\x9F\x90\xBB' for column
After inspection, it was found that the hive data contains image data. I was also drunk and tossed for a day. The main reason is that the UTF8 encoding in MySQL can only support up to 3 bytes, and the UTF8 encoding used by Emoji emoticon characters, many are is 4 bytes, some are even 6 bytes.
There are two solutions:
1. Use mysql encoding of utf8mb4 to accommodate these characters.
2. Filter out these special emoji characters.
I chose option one
References
1.https://github.com/ZhangRuige/easyDataInsightV1.0/wiki/map-reduce%E6%89%A7%E8%A1%8Csqoop%E6%93%8D%E4%BD%9C%E6%97%B6%E8%B6%85%E6%97%B6%E7%9A%84%E9%97%AE%E9%A2%98
2.http://www.cnblogs.com/lowwen/p/5645242.html
3.http://stackoverflow.com/questions/23670754/exception-java-sql-sqlexception-incorrect-string-value-xf0-x9f-x92-xbc-for