Mysql Optimization and storage engine

1. Mysql performance indicators

mysql performance reference index is TPS, QPS response time and latency.

TPS = (+ number of transactions submitted to roll back the transaction) / server startup time
QPS = (number of queries) / server startup time

waiting for a response time = time to return results - the start time of execution
Here Insert Picture Description

Mysql5.1.x version brought a lot of useful testing tool that can automatically test a highly concurrent, multi-user multi-query performance data, such as MySqlSlap.
Here Insert Picture Description
At the command window, enter the number of operating parameters, parameter values can be tested according to the test data is automatically created, and mimic a multi-user environment under high concurrent queries. After the test results are returned, the query data is created when the test will be automatically deleted.
Here Insert Picture Description

2.MySql fabric logic

MySql architecture

  • The connection layer
  • Service Layer
  • Engine Layer
  • Storage layer

Here Insert Picture Description

  • Connection layer:
    Here Insert Picture Description
    connecting to the database is to support multiple languages, it is common JDBC driver connection


Here Insert Picture Description
Have connection process, verifies that the user name and password is extremely starting operation again not their permissions.


Here Insert Picture Description
MySql is a service designed for multiple users, each user client after the connection is successful, it will allocate a thread, each thread is relatively independent, multiple user threads query is no problem, but if not bound to the database modify the data, it is bound to cause problems synchronizing data.

  • Service Layer :
    This layer is the main function Sql statement parsing, optimization, cache queries.
    Here Insert Picture Description
    Sql Interface: Interface Layer Sql statement
    Parser: parser to convert you write commands to the database itself know
    Optimizer: Optimizer
    Caches & Buffers: buffers, enabled by default after the cache execute SQL optimization, you can also turn on data cache.

Sql process flow
if the query is look
Here Insert Picture Description
first to see if there is the same Sql statement cache, hit the same result is returned directly from the cache. If you do not hit, the first resolves the query (in the order of their own parser creates a parse tree identification, resolve Sql statement syntax and semantics), and then optimize (eg, rewrite the query, choose the required indexing etc.), and then execute the query.


Using show VARIABLES LIKE '% query_cache_type%' open state query cache
        SET GLOBAL query_cache_type = 1 = 0 is turned off
Here Insert Picture Description
show VARIABLES LIKE '% query_cache_size%' queries the size of the cache size
Here Insert Picture Description

If you want to query the data by caching, to speed up the search, then you first need to start the cache, then you need to set the cache size is not 0 (set according to the size of the cache volume size)

Example: The first query (not cached) is about 0.5s
Here Insert Picture Description
second query 0.002s
Here Insert Picture Description

MySql is not recommended to open the cache, error-prone because of dirty data, etc., in the actual development, the cache is more suitable for Redis


It will optimize some SQL statements in the impossible, or SQL statement unreasonable.

For example: First, we look to as an example of a table structure
Here Insert Picture Description
to perform a query parsing, found that the SQL statement is correct
Here Insert Picture Description
we have no syntax errors in a run, but impossible to achieve (the primary key field can not be empty) will be optimized as possible, not queries.
Here Insert Picture Description

  • Engine layers :
    Here Insert Picture Description

Commonly used is MyISAM and InnoDB, you can use SHOW ENGINES; check out how much support the current database engine

Here Insert Picture Description

(1) MyISAM engine

MyISAM tables are stored How?
First, create a MyISAM table engine
Here Insert Picture Description
into the MySQL data storage directory, we find that the MyISAM storage, a table is composed of three format, frm MYD MYI. frm (database table structure documents, any engines have) MYD (database files) MYI (index file)
so MyISAM engine file is also called non-clustered index file
Here Insert Picture Description

Characteristics of the MyISAM engine?

  • The level of concurrency and lock - table-level locking
  • Support for full-text search
  • It supports data compression

Supports data compression is a distinctive feature of MyISAM, how to compress a file it?
There myisampack MYSQL file in the bin directory, and use it to knock compression to compress command
Here Insert Picture Description

MyISAM engine applicable scenarios
(1) non-transactional applications (data warehouse, reports, log files)
      because MyISAM engine is not running a supported transaction

(2) Application of CD-type
reading speed MyISAM is faster than InnoDB

(3) application space class (function of spatial coordinates, GIS applications)
the latitude and longitude of the conversion is not supported InnoDB

(2) InnoDB engine

InnoDB engine is a later version of the default engine MySql5.5, his storage format is divided into two types, one is a system table space, one is a separate table space.

When the value of the variable innodb_file_per_table ON independent tablespace, OFF is a system table space , the default is ON after 5.6
we can also be modified by the SQL statement, the SET innodb_file_per_table the GLOBAL = ON ;
Here Insert Picture Description

Here Insert Picture Description

Storage space is divided into a separate table, and idb two tables. ifm storage configuration table, and storing data IDB index.
Here Insert Picture Description

Table space frm there is only one, its data and indexes are stored in a file system's (ibadateX)
Here Insert Picture Description
Here Insert Picture Description

A separate table space What advantages does?
1. can shrink
the system table space is not a shrink, but a separate table space is shrinking, shrinking finishing similar to the disk, assuming a table of data stored 20 million, accounting for 1000k of memory, you delete this data 20 million, accounting for memory or 1000k table, but you shrink it, it probably 1k, shrink disk functions like finishing

Here Insert Picture Description

Table space because all data tables and indexes are stored in a system table it, it will generate IO bottleneck
each individual table will table its own independent storage space, so a separate table space can refresh the data simultaneously to multiple files

Characteristics of the InnoDB engine

  • It is a transactional storage engine
  • Fully supports transactional ACID properties
  • Redo log和Undo log
  • InnoDB supports row-level locking (higher degree of concurrency)

Applicable scene: suitable for most OLTP (online application) application.

(3) CSV engine

Storing the CSV engine, often the CSM three files (metadata file such as: amount of data and status tables) CSV (document data stored) FRM (a data table structure)
Here Insert Picture Description

CSV engine features

  • Stored in CSV format
  • All columns can not be empty
  • Does not support indexing (not suitable for large table storage is not suitable for online processing)
  • Can directly edit the data files (text retention, and need to flush tables need to knock newline after the data newly added)

(4) Archive Engine

Archive files stored is two, ARZ (save the data), frm (save table structure).
Here Insert Picture Description

Archive engine features

  • Zlib to compress the data table, the disk I / O less, the file data is stored in ARZ
  • Only supports Insert and select operations, there is no deletion, insertion of data can not be deleted successfully
  • Only allowed in an auto-increment indexing

Be used: log storage, and data acquisition

(5) Memory engine

Memory engine files only frm, all data is stored in memory, in addition to the table definition information has a corresponding physical file is stored on disk, but as long as the mysqld service does not exist, then the data in the table are lost.
Here Insert Picture Description
Temporary tables are stored in memory and Memory, Talia difference is that the temporary table is session-level.
Here Insert Picture Description

Memory feature engine

  • Also known as HEAP storage engine, the data stored in memory
  • HASH indexes and index support BTree
  • All fields are fixed length varchar (10) = char (10), so that, varchar, and char storage limited length is the same, and even if there is no maximum length to be stored to calculate the maximum length of the space occupied by
  • Test does not support the BLOG and other large fields, if the construction of the table which contains large field will not be successful
  • Table-level locking
  • Maximum storage size is determined by the parameters max_heap_table_size

(6) .Ferderated 引擎

Ferderated feature

  • Do not store data locally, so the data is stored on a remote server
  • Local retaining only the table structure, and the connection to the remote server information

Ferderated is not supported by default, if you need to open, in the MySQL configuration file my.ini document added federated = 1

Contrast 3.InnoDB and MyISAM engine

Here Insert Picture Description

Published 30 original articles · won praise 4 · views 20000 +

Guess you like