Be alert to parameter changes, upgrade MySQL 8.0 to avoid stepping on the pit again

MySQL 8.0 New Features Column Directory

Double password, MySQL 8.0 innovative feature
sql_mode compatibility, MySQL 8.0 upgrade to be
alert to parameter changes, MySQL 8.0 upgrade to avoid stepping on the pit again

---

---

Foreword:

The previous blog sql_mode compatibility, the pit that MySQL 8.0 upgrade stepped on , mentioned the change of the default value of sql_mode in MySQL 8.0 version, which caused business access errors after the upgrade. We know that compared with MySQL 5.7, MySQL 8.0 has added many new features, and made a lot of optimization and adjustment in function and security, which will inevitably modify some parameters or functions. Our existing business data has been upgraded from MySQL 5.7 and earlier to 8.0, and we need to be particularly vigilant about changes in these parameters, which may lead to some unexpected results in business behavior! ! !

During the recent upgrade of MySQL 8.0, I sorted out some important parameter changes that may be used in the business, including some parameters that have been abandoned in MySQL 8.0 and changes in the default values ​​of some common parameters. For some parameter changes at the bottom of innodb, the business is not aware of it, so there is no need to list and discuss it separately.

---

1. Deprecated/Removed Parameters in MySQL 8.0

1.1 Related parameters

1.1.1 Query cache related parameters

The query cache function was removed as a whole in version 8.0.3, and all related parameters were removed.

• query_cache_type
• query_cache_size
• query_cache_limit
• query_cache_min_res_unit
• query_cache_wlock_invalidate

1.1.2 Transaction related parameters

tx_isolation and tx_read_only removed in 8.0.3, use the parameters transaction_isolation and transaction_read_only instead

1.1.3 Log related parameters

expire_logs_days sets the binlog retention days, which has been deprecated since MySQL 8.0.11, and the default value is changed to 0, and the parameter binlog_expire_logs_seconds is used instead. In MySQL 5.7, you need to pay attention to using the expire_logs_days parameter to set the binlog retention time.

1.1.4 Data Type Format

date_format and datetime_formate and time_format time formatting functions, removed in MySQL 8.0.3. Note that these functions are used in business programs, and errors are likely to be reported in 8.0.

1.1.5 Changes in sql_mode

The sql_mode parameter is of course still reserved in MySQL 8.0, however, note that the NO_AUTO_CREATE_USER mode was removed in 8.0.

---

1.2 MySQL 8.0 Upgrade Stepping Pit - Abandoned Parameters

The database version used by the backend of a set of java programs is MySQL 5.7. After the database is upgraded to MySQL 8.0, the business code fails to connect to the database. The error message is "java.sql.SQLException: Cannot create PoolableConnectionFactory (Could not create connection to database server.)"

get connetion from pool error:ConnectionInfo{
    
    jdbcUrl='jdbc:mysql://9.9.9.9:3306/db?useSSL=false&serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8', host='9.9.9.9', port='3306', dbName='db', userName='appuser', userPasswd='appuser'} for 3 times
[WARN] 2022-03-29 09:56:28,389 [4c927ecf695a46d9a78afafa309885c6] com.tenpay.risk.ditto.core.common.data.ExceptionWarper.warper(ExceptionWarper.java:71) --> Exception=============com.tenpay.risk.ditto.core.common.exception.DbException
com.tenpay.risk.ditto.core.common.exception.DbException: java.sql.SQLException: Cannot create PoolableConnectionFactory (Could not create connection to database server.)

Our first reaction, is this an incompatible version of jdbc's MySQL connection driver?

First, check that the MySQL connection driver version used by the JAVA program is 5.1.38. As shown in the figure:

Then, I checked the version compatibility of mysql-connecter-java online, and found that Connector/J version 5.1 is partially compatible with MySQL 8.0.

Next, we tested all minor versions of MySQL Connector/J version 5.1 (from 5.1.1 to 5.1.49) and found that they were indeed partially compatible. Among them, versions 5.1.44 - 5.1.49 support MySQL 8.0. That is to say, the versions after 5.1.44 all support MySQL 8.0, while the versions before 5.1.44 are only compatible with some minor versions, and most of them are not compatible.

At last,We checked the release note of mysql-connector-java-5.1.44 and found that this version fixes an important bug. Use the parameters transaction_isolation and transaction_read_only instead of tx_isolation and tx_read_only.(Refer to the official documentation: https://dev.mysql.com/doc/relnotes/connector-j/5.1/en/news-5-1-44.html)

Important Change: Following the changes in MySQL Server 8.0.3, the system variables tx_isolation and tx_read_only have been replaced with transaction_isolation and transaction_read_only in the code of Connector/J. Users should update Connector/J to this latest release in order to connect to MySQL 8.0.3. They should also make the same adjustments to their own applications if they use the old variables in their codes. (Bug #26440544)

Therefore, the solution to the failure of the java program to connect to MySQL 8.0 is to upgrade the MySQL driver version to version 5.1.44 or later. It is recommended to directly upgrade the driver to version 8.0.

---

2. Parameters with default values ​​changed in MySQL 8.0

2.1 Parameter default value change

The default values ​​of many parameters in MySQL 8.0 have changed compared to 5.7.

Option/Parameter	Old Default	New Default
character_set_server	latin1	utf8mb4
collation_server	latin1_swedish_ci	utf8mb4_0900_ai_ci
explicit_defaults_for_timestamp	OFF	ON
optimizer_trace_max_mem_size	16KB	1MB
validate_password_check_user_name	OFF	ON
back_log	-1 (autosize) changed from : back_log = 50 + (max_connections / 5)	-1 (autosize) changed to : back_log = max_connections
max_allowed_packet	4194304 (4MB)	67108864 (64MB)
max_error_count	64	1024
event_scheduler	OFF	ON
table_open_cache	2000	4000
log_error_verbosity	3 (Notes)	2 (Warning)
innodb_undo_tablespaces	0	2
innodb_undo_log_truncate	OFF	ON
innodb_flush_method	NULL	fsync (Unix), unbuffered (Windows)
innodb_autoinc_lock_mode	1 (consecutive)	2 (interleaved)
innodb_flush_neighbors	1 (enable)	0 (disable)
innodb_max_dirty_pages_pct_lwm	0 (%)	10 (%)
innodb_max_dirty_pages_pct	75 (%)	90 (%)
performance-schema-instrument=‘wait/lock/metadata/sql/%=ON’	OFF	ON
performance-schema-instrument=‘memory/%=COUNTED’	OFF	COUNTED
performance-schema-consumer-events-transactions-current=ON	OFF	ON
performance-schema-consumer-events-transactions-history=ON	OFF	ON
performance-schema-instrument=‘transaction%=ON’	OFF	ON
log_bin	OFF	ON
server_id	0	1
log-slave-updates	OFF	ON
expire_logs_days	0	30
master-info-repository	FILE	TABLE
relay-log-info-repository	FILE	TABLE
transaction-write-set-extraction	OFF	XXHASH64
slave_rows_search_algorithms	INDEX_SCAN, TABLE_SCAN	INDEX_SCAN, HASH_SCAN
slave_pending_jobs_size_max	16M	128M
gtid_executed_compression_period	1000	0
group_replication_autorejoin_tries	0	3
group_replication_exit_state_action	ABORT_SERVER	READ_ONLY
group_replication_member_expel_timeout	0	5

---

2.2 Description of default value changes of some important parameters

1. Before MySQL 5.7, the default character set was latin1, and the corresponding sorting method was latin1_swedish_ci; after MySQL 8.0, the default character set was changed to utf8mb4, and the corresponding sorting method was utf8mb4_0900_ai_ci. The default values ​​for the related parameters character_set_client, character_set_connection, character_set_database, character_set_results, character_set_server, collation_database, and collation_server have all changed.
2. MySQL 8.0之后推出了caching_sha2_password的认证插件，并且作为推荐使用的认证插件，增加了相应的认证插件参数default_authentication_plugin ，默认值为caching_sha2_password。
3. explicit_defaults_for_timestamp参数控制对timestamp列的default和null值的处理，默认值从OFF修改为ON。
4. event_scheduler 时间调度器默认值从OFF修改为ON，建议即使在MySQL 8.0中仍然显式设置为OFF。
5. innodb_autoinc_lock_mode 控制MySQL自增键的生成方式，MySQL 8.0中将默认值从1修改为2以提高部分场景下的并发性能。
6. innodb_flush_neighbors 脏块刷盘方式，当表数据存储在传统 HDD 存储设备上时，与在不同时间刷新单个page页相比，在一次操作中刷新此类相邻page页可减少 I/O 开销（主要用于磁盘寻道操作）。对于存储在 SSD 上的数据，寻道时间不再是影响IO性能的一个重要因素，所以在MySQL 8.0中将默认值设置为0以分散写入操作。
7. local_infile 参数决定能否使用load data local infile命令，在MySQL 5.7之前都是默认放开的，但是在MySQL 8.0之后，处于安全考虑，将默认值从ON修改为OFF。
8. log_error_verbosity 参数控制MySQL日志级别，默认值从3修改为2。
9. log_slave_updates 参数控制从库记录回放日志到binlog，MySQL 8.0中将默认值从OFF修改为ON，这里可能会增加IO负载压力。
10. sql_mode 模式的默认值中去掉了NO_AUTO_CREATE_USER ，MySQL 8.0的默认配置为sql_mode='ONLY_FULL_GROUP_BY, STRICT_TRANS_TABLES, NO_ZERO_IN_DATE, NO_ZERO_DATE, ERROR_FOR_DIVISION_BY_ZERO, NO_ENGINE_SUBSTITUTION'。

2.3 MySQL 8.0升级踩坑，不同版本MySQL的参数默认值改变导致业务报错

2.3.1 After the database is upgraded, the business feedback import data error

When the database version is MySQL 5.7, the business side always uses the LOAD DATA LOCAL INFILE method to import data; after upgrading to MySQL 8.0.23, the feedback derivative command reports an error of ERROR 1148 (42000).

# 导数命令
MYSQL> LOAD DATA LOCAL INFILE 'test.csv' INTO TABLE sbtest1 FIELDS TERMINATED BY ','  LINES TERMINATED BY '\n' (id,name);

# 报错信息
ERROR 1148 (42000): The used command is not allowed with this MySQL version

2.3.2 Problem analysis and solution

LOAD DATA LOCAL INFILE Permission to import data is controlled by the parameter local_infile. In MySQL 5.7, the default value of this parameter is ON, which allows the client to import data using the LOAD DATA LOCAL INFILE method; in MySQL 8.0, for security reasons, the default value of this parameter is set to OFF, that is, the default is It is forbidden to use LOAD DATA LOCAL INFILE to import data.

There are two alternative solutions here:

1. Explicitly set local_infile=ON in MySQL 8.0 to keep the same as the default configuration before MySQL 5.7;

2. The business side uses other methods to import data, such as importing files using pipelines in the shell, or importing sql files using source on the MySQL client.

Obviously, solution 1 is simpler and more direct to solve the problem; but I recommend using solution 2. The use of load data local infile by the client itself has security risks, and it is generally better to disable it.

Summarize

MySQL 8.0 adds many new and useful features, and there are many changes compared to the 5.7 version. When upgrading existing business data from MySQL 5.7 and earlier to MySQL 8.0, you need to be especially alert to changes in some parameters, which may lead to unexpected results in business behavior. At the same time, standardizing the way the database is used is also an effective way to avoid stepping on the pit.