10分钟完成MySQL对emoji的支持

10分钟完成MySQL对emoji的支持 

公司有新要求,ios客户端要上线评论中可以使用emoji表情的功能,在mysql 5.5 之前,UTF-8编码只支持1-3个字节;从MySQL 5.5开始,可以支持4个字节UTF编码 utf8mb4 ,一个字符能够支持更多的字符集,也能够支持更多表情符号。

utf8mb4兼容utf8,且比utf8能表示更多的字符,是utf8字符集的超集。所以现在一些新的业务,比如IOS中的emoji表情,会将MySQL数据库的字符集设置为utf8mb4。

先看问题:

 

Caused by: java.sql.SQLException: Incorrect string value: '\xF6\x9D\x98\x84' for column 'comment' at row 1
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1074)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4096)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4028)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2490)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2651)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2734)
    at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
    at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2458)

 

如果我们将列comment设置为varchar(100),用于存储评论信息,现在上线新功能存储emoji表情,插入emoji表情就会报出上述错误,UTF-8编码有可能是两个、三个、四个字节。Emoji表情是4个字节,而Mysql的utf8编码最多3个字节,所以数据插不进去。utf8mb4兼容utf8,且比utf8能表示更多的字符。

解决方案:将Mysql的编码从utf8转换成utf8mb4。

 

 

网上的文章各执一词,本文就生产环境中真实可用的参数进行记录

 

整体操作流程其实并不难

 

 

一:首先我们修改my.cnf参数

<span style="color:#333333"><span style="color:black"><code class="language-bash"><span style="color:#999999">[</span>client<span style="color:#999999">]</span>
default-character-set<span style="color:#9a6e3a">=</span>utf8mb4
  
  
<span style="color:#999999">[</span>mysql<span style="color:#999999">]</span>
default-character-set<span style="color:#9a6e3a">=</span>utf8mb4
  
  
<span style="color:#999999">[</span>mysqld<span style="color:#999999">]</span>
character-set-server <span style="color:#9a6e3a">=</span> utf8mb4
collation-server <span style="color:#9a6e3a">=</span> utf8mb4_unicode_ci
init_connect <span style="color:#9a6e3a">=</span> <span style="color:#669900">'SET NAMES utf8mb4'</span>
character-set-client-handshake <span style="color:#9a6e3a">=</span> <span style="color:#990055">false</span></code></span></span>

 

 

二:对数据库相关的表进行字符集修改

将数据库转换为utf8mb4

<span style="color:#333333"><span style="color:black"><code class="language-sql">mysql<span style="color:#9a6e3a">></span> <span style="color:#0077aa">ALTER</span> <span style="color:#0077aa">DATABASE</span> erp <span style="color:#0077aa">CHARACTER</span> <span style="color:#0077aa">SET</span> utf8mb4 <span style="color:#0077aa">COLLATE</span> utf8mb4_unicode_ci<span style="color:#999999">;</span></code></span></span>

 

将已经建好的表也转换成utf8mb4 

<span style="color:#333333"><span style="color:black"><code class="language-sql">mysql<span style="color:#9a6e3a">></span><span style="color:#0077aa">ALTER</span> <span style="color:#0077aa">TABLE</span> <span style="color:#999999">`</span>erp_comment<span style="color:#999999">`</span> <span style="color:#0077aa">CONVERT</span> <span style="color:#0077aa">TO</span> <span style="color:#0077aa">CHARACTER</span> <span style="color:#0077aa">SET</span> utf8mb4 <span style="color:#0077aa">COLLATE</span> utf8mb4_unicode_ci<span style="color:#999999">;</span></code></span></span>

将需要使用emoji的字段设置类型为: 

<span style="color:#333333"><span style="color:black"><code class="language-sql">mysql<span style="color:#9a6e3a">></span><span style="color:#0077aa">ALTER</span> <span style="color:#0077aa">TABLE</span> <span style="color:#999999">`</span>erp_comment<span style="color:#999999">`</span> <span style="color:#0077aa">MODIFY</span> <span style="color:#0077aa">COLUMN</span> <span style="color:#999999">`</span><span style="color:#0077aa">comment</span><span style="color:#999999">`</span>  <span style="color:#0077aa">varchar</span><span style="color:#999999">(</span><span style="color:#990055">100</span><span style="color:#999999">)</span> <span style="color:#0077aa">CHARACTER</span> <span style="color:#0077aa">SET</span> utf8mb4 <span style="color:#0077aa">COLLATE</span> utf8mb4_unicode_ci<span style="color:#999999">;</span></code></span></span>

 

三:重启数据库服务器使之生效

<span style="color:#333333"><span style="color:black"><code class="language-bash"><span style="color:#999999">[</span>root@HE3 ~<span style="color:#999999">]</span><span style="color:slategray"># /etc/init.d/mysqld restart</span>
Shutting down MySQL<span style="color:#999999">..</span> SUCCESS<span style="color:#9a6e3a">!</span> 
Starting MySQL<span style="color:#999999">..</span><span style="color:#999999">..</span> SUCCESS<span style="color:#9a6e3a">!</span></code></span></span>

四:登录数据库检查是否如下:

<span style="color:#333333"><span style="color:black"><code class="language-sql">mysql<span style="color:#9a6e3a">></span> <span style="color:#0077aa">SHOW</span> VARIABLES <span style="color:#0077aa">WHERE</span> Variable_name <span style="color:#9a6e3a">LIKE</span> <span style="color:#669900">'character%'</span> <span style="color:#9a6e3a">OR</span> Variable_name <span style="color:#9a6e3a">LIKE</span> <span style="color:#669900">'collation%'</span><span style="color:#999999">;</span>
<span style="color:#9a6e3a">+</span><span style="color:slategray">--------------------------+--------------------+</span>
<span style="color:#9a6e3a">|</span> Variable_name            <span style="color:#9a6e3a">|</span> <span style="color:#0077aa">Value</span>              <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">+</span><span style="color:slategray">--------------------------+--------------------+</span>
<span style="color:#9a6e3a">|</span> character_set_client    <span style="color:#9a6e3a">|</span> utf8mb4            <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> character_set_connection <span style="color:#9a6e3a">|</span> utf8mb4            <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> character_set_database  <span style="color:#9a6e3a">|</span> utf8mb4            <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> character_set_filesystem <span style="color:#9a6e3a">|</span> <span style="color:#0077aa">binary</span>            <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> character_set_results    <span style="color:#9a6e3a">|</span> utf8mb4            <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> character_set_server    <span style="color:#9a6e3a">|</span> utf8mb4            <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> character_set_system    <span style="color:#9a6e3a">|</span> utf8              <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> collation_connection    <span style="color:#9a6e3a">|</span> utf8mb4_unicode_ci <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> collation_database      <span style="color:#9a6e3a">|</span> utf8mb4_unicode_ci <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> collation_server        <span style="color:#9a6e3a">|</span> utf8mb4_unicode_ci <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">+</span><span style="color:slategray">--------------------------+--------------------+</span>
<span style="color:#0077aa">rows</span> <span style="color:#9a6e3a">in</span> <span style="color:#0077aa">set</span> <span style="color:#999999">(</span><span style="color:#990055">0.00</span> sec<span style="color:#999999">)</span></code></span></span>

特别说明下:collation_connection/collation_database/collation_server如果是utf8mb4_general_ci,没有关系。但必须保证character_set_client/character_set_connection/character_set_database/character_set_results/character_set_server为utf8mb4。

 

五:让开发那边的pom配置中,去掉characterEncoding参数,并重新编译一下

如果你用的是java服务器,升级或确保你的mysql connector版本高于5.1.13,否则仍然无法使用utf8mb4

 

最后再让前端应用插入emoji表情,就可以了。

 

一些小知识点:

其中character-set-server 和 collation-server 这些设置为utf8mb4字符集是比较容易理解的,就是将MySQL数据库相关的字符集都设置为utf8mb4;

但为了实现客户端utf8连接到MySQL后,使用的也是utf8mb4字符集,就在 mysqld配置中配置了 init_connect='SET NAMES utf8mb4' 表示初始化连接都设置为utf8mb4字符集,再配置一个 skip-character-set-client-handshake = true 忽略客户端字符集设置,不论客户端是何种字符集,都按照init_connect中的设置进行使用,这样就满足了应用的需求。

参考:Emoji对照表:

http://punchdrunker.github.io/iOSEmoji/table_html/bell.html

 

猜你喜欢

转载自blog.csdn.net/kingmax54212008/article/details/82941332