MySQL character set conversion operation scenarios

In actual scenarios of database upgrade or migration, there may be a need to convert character sets. This article from the technical community "Technology Sharing | MySQL 8.0: Converting Character Sets from utf8 to utf8mb4 " introduces the conversion operation of character sets in MySQL. Learn from it.

The default character set of MySQL 8.0 has changed from latin1 to utf8mb4, so upgrading to MySQL 8.0 will most likely require character set conversion.

Requirement background:

The character set used by some systems is utf8, but utf8 can only store characters with a maximum length of 3 bytes, and cannot store 4-byte uncommon words or emoticons, so it is planned to migrate to utf8mb4.

Migration plan one

1. Prepare a new database instance and modify the following parameters,

[mysqld]
## Character Settings
init_connect='SET NAMES utf8mb4'
#连接建立时执行设置的语句，对super权限用户无效
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci
#设置服务端校验规则，如果字符串需要区分大小写，设置为utf8mb4_bin
skip-character-set-client-handshake
#忽略应用连接自己设置的字符编码，保持与全局设置一致
## Innodb Settings
innodb_file_format = Barracuda
innodb_file_format_max = Barracuda
innodb_file_per_table = 1
innodb_large_prefix = ON
#允许索引的最大字节数为3072（不开启则最大为767字节，对于类似varchar(255)字段的索引会有问题，因为255*4大于767）

2. Stop the application, observe and confirm that no more data is written.

You can observe the GTID or binlog position through show master status. If there is no change, there is no writing.

3. Export data

First export the table structure,

mysqldump -u -p --no-data --default-character-set=utf8mb4 --single-transaction --set-gtid-purged=OFF --databases testdb > /backup/testdb.sql

Then export the data,

mysqldump -u -p --no-create-info --master-data=2 --flush-logs --routines --events --triggers --default-character-set=utf8mb4 --single-transaction --set-gtid-purged=OFF --database testdb > /backup/testdata.sql

4. Modify the table creation statement

Modify the exported table structure file and change utf8 in table and column definitions to utf8mb4.

5. Import data

First import the table structure,

mysql -u -p testdb < /backup/testdb.sql

Then import the data,

mysql -u -p testdb < /backup/testdata.sql

6. Create user

Find out the database user of the old environment and create it in the new database.

7. Modify the new database port and start the application for testing

Close the old database, modify the new database port, restart, and start the application.

Migration plan two

1. Modifying the character encoding of the table will lock the table. It is recommended to stop the application first.

2. Stop mysql and back up the data directory (full backup can also be done in other ways).

3. Modify the configuration file and restart the database.

[mysqld]
## Character Settings
init_connect='SET NAMES utf8mb4'
#连接建立时执行设置的语句，对super权限用户无效
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci
#设置服务端校验规则，如果字符串需要区分大小写，设置为utf8mb4_bin
skip-character-set-client-handshake
#忽略应用连接自己设置的字符编码，保持与全局设置一致
## Innodb Settings
innodb_file_format = Barracuda
innodb_file_format_max = Barracuda
innodb_file_per_table = 1
innodb_large_prefix = ON
#允许索引的最大字节数为3072（不开启则最大为767字节，对于类似varchar(255) 字段的索引会有问题，因为255*4大于767）

4. View all table structures, including fields, modification libraries, and table structures. If the fields have defined character encodings, you also need to modify the field attributes. The SQL statement is as follows:

Modify the character set of the table,

alter table t convert to character set utf8mb4;

Impact: Copying the entire table is slow, will lock, and blocks write operations.

Modify the character set of the field (utf8mb4 occupies 4 bytes per character, pay attention to the relationship between the maximum number of bytes of the field type and the character length),

alter table t modify a char CHARACTER SET utf8mb4;

Impact: Copying the entire table is slow, will lock, and blocks write operations.

Modify the character set of the database,

alter database sbtest CHARACTER SET utf8mb4;

Impact: Only metadata needs to be modified, which is fast.

5. Modify JDBC url characterEncoding=utf-8.

If you think this article is helpful, please feel free to click "Like" and "Reading" at the end of the article, or forward it directly to pyq,

Recently updated articles:

" Financial Knowledge - Secondary Market "

" Introduction to the poweroff command "

" Detailed Scenario Explanation of MySQL 8.0 New Password Policy "

" Troubleshooting and Solution Paths for Several Data Leakage Scenarios "

" Some Problems Encountered Recently "

Recent hot articles:

" Recommend a classic paper on Oracle RAC Cache Fusion "

" The shock that the open source code of the "Red Alert" game brings to us "

Article classification and indexing:

《Classification and indexing of 1,300 articles on public accounts》