High Performance MySQL Notes

Partition Table

Table partition table is suitable for very large that they can not all fit in memory, or if the table is only part of the scene hotspot data. A table can be up to 1024 partitions, partition expression must return an integer, the partition table can not use foreign key constraints.

grammar

create table xxx(
    -- some columns
) engine=innodb partition by range( xxxx )(
    partition table_1 values less than ( a_value ),
    partition table_2 values less than ( a_value ),
    partition table_3 values less than ( a_value )
)

Several partitioning scheme:

• Range (range)
• Hash (hash)
• Key (key)
• List (predefined list)
• Composite (multi mode)

Under what circumstances will be problems

• null value so that the filtering partition is invalid, a null value is placed in the first partition.
• Partitioning columns and index columns do not match
• Select the partition cost is too high
• Open and lock all the underlying tables can be costly (in the selected partition)

Several principles summary

1. Merge table without, this is a coming out of the technology, there are many defects.
2. If you plan to use the view to enhance the performance of the best to do detailed testing, performance view is difficult to predict. mysql does not support materialized views, create an index does not support the view.
3. Foreign key constraint will consume a great deal of performance, if only to make foreign key constraint, usually achieve better in the application process.
4. Stored procedures, triggers, stored functions, these features events from copying, replication scenario-based statement also prone to problems in the main, these characteristics although they would save network overhead, but still should be used sparingly.
5. Rarely used xa transaction characteristics, it is best not to modify xa configuration.
6. High concurrent queries in the query cache environment can cause performance degradation, it is best not to use, but should use other solutions such as memcached.

mysql character set collation

_cs representatives case-sensitive way to compare strings
_ci not case sensitive
_bin binary

Application layer optimization

1. Add page caching module, caching part of the page
2. Open gzip compression, for now, in terms of cpu such a small price, but you can save most of the traffic.
3. Do not configure the keep-alive to maintain a long connection with the client through a proxy server to connect long distance, does not maintain a long connection between the server and ap proxy server.

Cache Control Strategy

1. TTL survival time, set up a survival time longer than that to clean up data for data change little or no cases of new data.
2. Display failure, if you can not accept the dirty data, while allowing the cache data when updating the original data fail, generally: write - and write failure - Update two ways.
3. When reading fails, when you change the old data, in order to avoid some of the information can be stored dirty data in the cache, when reading the data in the cache can use this information to determine whether the data is invalid. (Object Versioning)

mysql configuration optimization

1. socket and pid file documents not placed in the default location, it is best to explicitly specify the storage location, or an error will occur

   socket = "/var/lib/mysql/mysql.sock"
   basedir = "/var/lib/mysql" 
   pid_file = "mysql.pid"

2. Use innodb engine, you need to configure p

   innodb_data_home_dir = "/var/lib/mysql/data"
   ## 10M 是文档的初始大小，autoextend 是自动扩展
   innodb_data_file_path = ibdata1:10M:autoextend 
   ## 每个表单独存储一个文档
   innodb_file_per_table = 1
   
   ## 内存的 50 - 80 %
   innodb_buffer_pool_size = 2048M
   ## innodb_buffer_pool_size 的 25%
   innodb_log_file_size = 512M
   ## 通常不需要设置的非常大， 1M - 8M 之间就可以
   innodb_log_buffer_size = 8M
   
   ## 0 把日志缓冲写到日志文档，并且每秒钟刷新一次，但是事务提交时不做任何操作。
   ## 1 默认的，并且是最安全的设置，每次事务提交都刷新到持久化存储，保证不会丢失任何已经提交的事务。
   ## 2 每次提交把日志缓冲写到日志文档，但是并不刷新，是 0 与 1 的折中。
   innodb_flush_log_at_trx_commit = 1

3. log

   log_error = /var/lib/mysql/mysql-error.log
   ## 是否开启慢查询日志，1表示开启，0表示关闭
   slow_query_log = 1
   slow_query_log_file=/var/lib/mysql/mysql-slow.log
   ## 慢查询阈值，当查询时间多于设定的阈值时，记录日志
   ## long_query_time
   ## 未使用索引的查询也被记录到慢查询日志中
   ## log_queries_not_using_indexes
   ## 日志存储方式。log_output='FILE'表示将日志存入文档，默认值是'FILE'。
   ## log_output='TABLE'表示将日志存入数据库
   ## log_output：

4. Other configurations

   default_storage_engine = InnoDB 
   ## 现代操作系统句柄开销都很小，这个应该设置的尽可能的大
   open_files_limit = 65535
   ## tmp_table_size max_heap_table_size 是用于设置使用 Memory 引擎的内存临时表能使用多大的内存，超过这个就转而使用磁盘临时表
   tmp_table_size = 32M
   max_heap_table_size = 32M
   ## 禁用查询缓存
   query_cache_type = 0
   query_cache_size = 0
   ## 默认是 100 对大多数应用进程来说应该都是不够用的，根据实际业务调整
   max_connections = <..>
   
   thread_cache = <...>
   open_files_limit = <...>

Original link large column  https://www.dazhuanlan.com/2019/08/15/5d5513c39cda3/