Storage Engine Introduction

I. Introduction Storage Engine

• 1, the file system:
  • A mechanism to organize 1.1 operating system and accessing data.
  • 1.2 is a software file system.
• 2, the file system type: ext2 3 4, xfs data (centos6 ext 4 centos7 xfs)
  • 2.1 No matter what file system, data content will not change
  • The difference is 2.2, the storage space, the size, speed.
• 3, MySQL engine:
  • Can be understood as 3.1, MySQL's "File System", but more powerful.
• 4, MySQL engine features:
  • 4.1 In addition to providing access to basic functions, there's more transaction capabilities, lock, backup and recovery, optimization, and special features

In short, the characteristics of the storage engine is designed to protect the safety and performance of the database structure.

Two .MySQL own storage engine type

MySQL storage engine provides the following:
01) InnoDB
02) MyISAM
03) the MEMORY
04) the ARCHIVE
05) the FEDERATED
06) the EXAMPLE
07) BLACKHOLE
08) MERGE
09) NDBCLUSTER
10) CSV

---

You can also use a third-party storage engine:
01) among MySQL pluggable storage engine type
02) MySQL two branches
03) perconaDB
04) MariaDB

#查看当前MySQL支持的存储引擎类型
mysql> show engines
#查看innodb的表有哪些
mysql> select table_schema,table_name,engine from information_schema.tables where engine='innodb';
#查看myisam的表有哪些
mysql> select table_schema,table_name,engine from information_schema.tables where engine='myisam';

• 1 difference, innodb and the myisam

Physical differences:

#进入mysql目录
[root@db01~l]# cd /application/mysql/data/mysql
#查看所有user的文件
[root@db01 mysql]# ll user.*
-rw-rw---- 1 mysql mysql 10684 Mar  6  2017 user.frm
-rw-rw---- 1 mysql mysql   960 Aug 14 01:15 user.MYD
-rw-rw---- 1 mysql mysql  2048 Aug 14 01:15 user.MYI
#进入word目录
[root@db01 world]# cd /application/mysql/data/world/
#查看所有city的文件
[root@db01 world]# ll city.*
-rw-rw---- 1 mysql mysql   8710 Aug 14 16:23 city.frm
-rw-rw---- 1 mysql mysql 688128 Aug 14 16:23 city.ibd

• About 2.innodb storage engine

After MySQL5.5 version, the default storage engine, provide high reliability and performance.

Benefits:
01) Transaction Security (comply with the ACID)
02) MVCC (Multi-Versioning Concurrency Control, multi-version concurrency control)
03) InnoDB row-level locking
04) Oracle-style consistent non-locking read
05) table data collation to optimize based on the primary key query
06) supports foreign key referential integrity constraints
07) for maximum performance on large data volume
08) will table queries and mix different storage engines
09) after failure fast automatic recovery (csr) appears
10) used in memory buffer pool cache data and indexes

innodb core features

Key:
MVCC
Transaction
row-level locking
hot backup
Crash Safe Recovery (Auto Recovery)

• 3. Review the storage engine

1) Use SELECT to confirm the session storage engine

#查询默认存储引擎
SELECT @@default_storage_engine;

2) Use SHOW sure each table storage engine

#查看表的存储引擎
SHOW CREATE TABLE City\G 
SHOW TABLE STATUS LIKE 'CountryLanguage'\G

3) Use INFORMATION_SCHEMA sure each table storage engine

#查看表的存储引擎
mysql> select table_schema,table_name,engine from information_schema.tables \G
SHOW CREATE TABLE City\G 

• 4. A storage engine is provided

1) Set the server storage engine in the startup configuration file

#在配置文件的[mysqld]标签下添加
[mysqld]
default-storage-engine=<Storage Engine>

2) Use the SET command to set the current client session

#在MySQL命令行中临时设置
SET @@storage_engine=<Storage Engine>

(3) specified in the CREATE TABLE statement

#建表的时候指定存储引擎
CREATE TABLE t (i INT) ENGINE = <Storage Engine>;



III. The real business case

Background of the project:

The company's existing architecture: Display a website, LAMT, MySQL5.1.77 version (MYISAM), 50M amount of data.

Small problems continue:

• 1, table lock: in any row of Table class data modification operation, the entire table will be locked, the operation of the other rows are not simultaneously.
• 2, does not support automatic fault recovery (CSR): There may be a problem of data corruption or loss when power outage occurs.

How to solve:

• 1, suggestions will replace the existing MYISAM engines Innodb, will replace version 5.6.38
  • 1) If you use MYISAM will have a "small problem", performance, safety can not be guaranteed, use innodb can solve this problem.
  • 2) 5.1.77 version to support innodb engine is not perfect, 5.6.38 version of the innodb support very well.
• 2, the implementation process and attentions

Simulation environment

建表
mysql> create table test1(sid int,name varchar(11)) engine=myisam;

mysql> create table test1(sid int,name varchar(11)) engine=myisam;

mysql> create table test3(sid int,name varchar(11)) engine=myisam;

1) production of library data backup (mysqldump)

[root@db01 ~]# mysqldump -uroot -p123 -A --triggers -R --master-data=2 >/tmp/full.sql
#-A系统库 -B程序业务恢复库

2) Preparation of a new database version 5.6.38
3) processing the backup data (engine replaces field)

[root@db01 ~]# sed -i 's#ENGINE=MYISAM#ENGINE=INNODB#g' /tmp/full.sql

4) restore the backup modified to the new library

mysql -uroot -p123 -h 10.0.0.52 </tmp/full.sql

#导出表
mysqldump -uroot -p3308 -S /data/3308/data/mysql.sock --tables test student

5) application testing environment to connect the new library, test all functions
6) stop the application, the new changes after the backup of the production database occurs, compensation to the new library
7) cut to a new database application

Project results:

Solve the "small problem"

Four .Innodb storage engine - table spaces Introduction

Shared table space concept emerged after the 5.5 version

Appearance management table space to store the database easier to expand

The default version 5.6 is a separate table space

• 1, shared table space

  Innodb all data stored in a separate table space inside, and this space can be a number of table files, a file across multiple tables exist, so its size is not restricted file size limit, but the limit of its own, the default maximum limit for the 64tb, including data related to the size of the index table and so all in all

1) View shared table space

#物理查看
[root@db01 ~]# ll /application/mysql/data/
-rw-rw---- 1 mysql mysql 79691776 Aug 14 16:23 ibdata1
#命令行查看
mysql> show variables like '%path%';
innodb_data_file_path =bdata1:12M:autoextend

Version 5.6 default storage:
1. system data
2.undo (transaction logs, 5.7 configuration files can be changed independent)
3. temporary table (query generates temporary tables)

Version 5.7 will undo default and temporary tables independent, version 5.6 can be independent, but need to be configured during initialization

2) Setting method

#编辑配置文件
[root@db01 ~]# vim /etc/my.cnf
[mysqld]
innodb_data_file_path=ibdata1:76M;ibdata2:50M:autoextend

• 2, separate tablespaces

Table for user self-created, will adopt this model, each table is managed by a separate table space

1) See separate table space

#物理查看
[root@db01 ~]# ll /application/mysql/data/world/
-rw-rw---- 1 mysql mysql 688128 Aug 14 16:23 city.ibd
#命令行查看
mysql> show variables like '%per_table%';
innodb_file_per_table=ON

2) separate tablespaces

#物理查看
[root@db01 ~]# ll /application/mysql/data/world/
-rw-rw---- 1 mysql mysql 688128 Aug 14 16:23 city.ibd
#命令行查看
mysql> show variables like '%per_table%';
innodb_file_per_table=ON

Business Case

In the case where there is no backup data, a sudden power failure table corruption, open database.

Simulation environment

db01 db02

Table 1. Packaging

tar tar.world.gz world
scp word  /application/mysql/data/

2. Multi-Instance simulation test

1) copy of the library to the new library catalog

[root@db01 ~]# cp -r /application/mysql/data/world/ /data/3307/data/

2) Start a new database

[root@db01 ~]# mysqld_safe --defaults-file=/data/3307/my.cnf &

3) Log in to view the database

mysql> show databases;

The data 4) look-up table

mysql> select * from city;
ERROR 1146 (42S02): Table 'world.city' doesn't exist

5) find the previous table structure to create a table in the new library

mysql> show create table world.city;
#删掉外键创建语句
CREATE TABLE `city` (
  `ID` int(11) NOT NULL AUTO_INCREMENT,
  `Name` char(35) NOT NULL DEFAULT '',
  `CountryCode` char(3) NOT NULL DEFAULT '',
  `District` char(20) NOT NULL DEFAULT '',
  `Population` int(11) NOT NULL DEFAULT '0',
  PRIMARY KEY (`ID`),
  KEY `CountryCode` (`CountryCode`),
  KEY `idx_city` (`Population`,`CountryCode`),
  CONSTRAINT `city_ibfk_1` FOREIGN KEY (`CountryCode`) REFERENCES `country` (`Code`)
) ENGINE=InnoDB AUTO_INCREMENT=4080 DEFAULT CHARSET=latin1;

6) Delete the table space file

mysql> alter table city_new discard tablespaces;

7) copies of the old table space file

[root@db01 world]# cp -a /data/3307/data/world/city.ibd /data/3307/data/world/city_new.ibd #加-a不需要授权

8) Authorization

[root@db01 world]# chown -R mysql.mysql *

9) into the tablespace

mysql> alter table city_new import tablespace; 

10) physical delete the old table

[root@db02 world]# rm -fr city.*

11) indicate change

mysql> alter table city1 rename city;

12) If you are a new library, application cutover (even modify the code library ip)

Five .Innodb core features - Transaction

• 1. What is a transaction

Mainly for DML statements (update, delete, insert)

A set of data operation step, these steps are considered a unit of work:
1) for grouping a plurality of statements
using 2) may concurrently access the same data table in the plurality of clients

---

All steps are successful or have failed
1) If all the steps correctly, then execute
2) If there are errors or incomplete step is canceled

• 2. Transaction popular understanding

With the emergence of the concept of database "transaction."

What do we understand the "transaction"?
1) exchange things and things of (ancient)
2) Money in cash and in-kind exchange (Modern 1)
3) and the kind of virtual currency exchange (modern 2)
4) virtual currency and virtual barter (modern 3)

---

Database "transaction" What is?
1) how to ensure that the transaction is "trading" and "harmony"?
2) ACID

• 3. The transaction ACID properties

---

Atomic (atomicity)
All statements as a whole unit successfully executed or canceled.

Consistent (consistent)
if the database is in a consistent state at the beginning of the transaction, then the execution. During the transaction will remain consistent state.

Isolated (isolation)
do not affect each other affairs.

Durable (persistence)
after the transaction is completed successfully, any changes made will be accurately recorded in the database. The changes will not be lost.

---

• Example 4. ** ** transaction flow
  

• The transaction control statements

As follows:
the START TRANSACTION (or BEGIN): Explicit start a new transaction
SAVEPOINT: assign a position in the transaction process, for future reference
COMMIT: a permanent record of changes made to the current transaction
ROLLBACK: cancel current transaction changes
ROLLBACK TO SAVEPOINT: cancel the changes executed after the savepoint
the RELEASE SAVEPOINT: savepoint delete identifier
SET AUTOCOMMIT: current connection disable or enable default autocommit mode

The life cycle of a successful transaction

begin;
sql1
sql2
sql3
...
commit;

A failure of the transaction life cycle

begin;
sql1
sql2
sql3
...
rollback;

• 3. Automatic submission

#查看自动提交
mysql> show variables like 'autocommit';
#临时关闭
mysql> set autocommit=0;
#永久关闭,(企业中一般不关闭)
[root@db01 world]# vim /etc/my.cnf
[mysqld]
autocommit=0

• 4. affairs demo

1) successful transactions

mysql> create table stu(id int,name varchar(10),sex enum('f','m'),money int);
mysql> begin;
mysql> insert into stu(id,name,sex,money) values(1,'zhang3','m',100), (2,'zhang4','m',110);
mysql> commit;

2) transaction rollback

mysql> begin;
mysql> update stu set name='zhang3';
mysql> delete from stu;
mysql> rollback;

• 6. Transaction implicit submission

1) version is now in the open transaction, no need to manually begin, as long as you enter a DML statement, the transaction will automatically open.
2) In some cases implicit transaction will be submitted

For example:
during a transaction operation, when performed manually begin automatically submit the transaction
during a transaction operation, adding DDL, DCL action automatically submit the transaction
during the transaction run, execute the lock statement (Lock the Tables, UNLOCK the Tables)
the Load infile the Data
the SELECT Update for (the results of the query sql statement)
in autocommit = 1 when

• 7. The transaction log redo the basic functions

What 1) Redo yes?

redo, as the name suggests "redo log" is a transaction log.

2) What role?

ACID transaction process, to achieve a "D" persistent effect.

Characteristics: WAL (Write Ahead Log) log priority write
REDO: record is that the change process memory data pages

3) REDO work process

#执行步骤
update t1 set num=2 where num=1; 

1) First, load the row num = t1 in Table 1 where the data pages into memory Page Buffer
2) in the instance of the MySQL memory num = 1 data page 2 into num =
. 3) becomes num = 1 num = 2 the process of change will be recorded, redo memory area, which is the redo buffer page in

#提交事务执行步骤
commit; 

1) When the moment Qiaoxia commit command, MySQL will redo buffer page is written to disk log area redo
2) After written success, commit returns ok

• 8.redo instance data recovery process

• 9. undo the transaction log

What 1) undo that?

undo, as the name suggests "rolling log" is a transaction log.

_ 2) What role?

ACID transaction process, to achieve atomic action is "A". Of course CI also features undo and related

• 10.redo and undo storage location

#redo位置
[root@db01 data]# ll /application/mysql/data/
-rw-rw---- 1 mysql mysql 50331648 Aug 15 06:34 ib_logfile0
-rw-rw---- 1 mysql mysql 50331648 Mar  6  2017 ib_logfile1
#undo位置
[root@db01 data]# ll /application/mysql/data/
-rw-rw---- 1 mysql mysql 79691776 Aug 15 06:34 ibdata1
-rw-rw---- 1 mysql mysql 79691776 Aug 15 06:34 ibdata2

In MySQL5.6 version is ibdata undo file, in MySQL5.7 version will be independent.

• 11. The transaction locks

1) What is "lock"?

"Lock" means the name suggests is locked.

2) What is the role of the "lock" is?

ACID properties of the transaction process, the "lock" and "isolation level" work together to achieve the effect of "I" of isolation.

Exclusive lock: to ensure that the multi-transaction operations, the consistency of the data.
Shared locks: to ensure that during a multiple affairs, will not be blocked when the data query.

Pessimistic lock: also modify the data, who should modify who prevail

Optimistic locking: Who is who should submit prevail

• 12. The multi-version concurrency control (MVCC)

1) modify the class only blocking operation, without blocking query class operations (exclusive lock, shared lock)
2) optimistic locking mechanisms (who should submit who prevail)

• 13. The lock granularity
  MyIsam: low concurrency locks (table lock)
  Innodb: high concurrent lock (row-level locks)

  == row-level locking. There must be a clustered index (primary key), or otherwise table-level locking ==

• 14. The transaction isolation level

The four isolation levels:

READ UNCOMMITTED (independent submission) RU level
allows transactions to see uncommitted changes made by other transactions

READ COMMITTED RC level (phantom read prone)
allows transactions to see other firms were committed changes

REPEATABLE READ ****** RR level (phantom reads can be resolved, quit reconnection)
to ensure that each transaction is consistent SELECT output
InnoDB the default level

SERIALIZABLE serialization level (in the process of transaction execution, the block query results)
will be the result of a transaction with other transactions are completely isolated

#查看隔离级别
mysql> show variables like '%iso%';
#修改隔离级别为RU
[mysqld]
transaction_isolation=read-uncommit
mysql> use oldboy
mysql> select * from stu;
mysql> insert into stu(id,name,sex,money) values(2,'li4','f',123);
#修改隔离级别为RC
[mysqld]
transaction_isolation=read-commit

Dirty read phantom reads Repeatable read and identify the cause, and solutions

一个事务读到另外一个事务还没有提交的数据，我们称之为脏读。解决方法：把事务隔离级别调整到READ COMMITTED

一个事务先后读取同一条记录，但两次读取的数据不同，我们称之为不可重复读。解决方法：把事务隔离级别调整到REPEATABLE READ。

一个事务先后读取一个范围的记录，但两次读取的纪录数不同，我们称之为幻象读。解决方法：把事务隔离级别调整到SERIALIZABLE。

1, 脏读

一个事务读到另一个事务，尚未提交的修改，就是脏读。这里所谓的修改，除了Update操作,不要忘了,还包括
Insert和Delete操作。

脏读的后果：如果后一个事务回滚，那么它所做的修改，统统都会被撤销。前一个事务读到的数据，就是垃圾数据。


举个例子：预订房间。
有一张Reservation表，往表中插入一条记录，来订购一个房间。

 事务1：在Reservation表中插入一条记录，用于预订99号房间。

 事务2：查询，尚未预定的房间列表，因为99号房间，已经被事务1预订。所以不在列表中。

 事务1：信用卡付款。由于付款失败，导致整个事务回滚。
        所以插入到Reservation 表中的记录并不置为持久（即它将被删除）。

现在99号房间则为可用。
所以，事务2所用的是一个无效的房间列表，因为99号房间，已经可用。如果它是最后一个没有被预定的房间，那么这将是一个严重的失误。

注：脏读的后果很严重。

 

2，不可重复读。

在同一个事务中，再次读取数据时【就是你的select操作】，所读取的数据，和第1次读取的数据，不一样了。就是不可重复读。

举个例子：
 事务1：查询99号房间是否为双人床房间。结果99号是。

 事务2：将99号房间，改成单人床房间。

 事务1：再次执行查询，99号房间不是双人房了。也就是说， 事务1，可以看到其他事务所做的修改。


在不可重复读，里面，可以看到其他事务所做的修改，而导致2次的查询结果不再一样了。
这里的修改，是提交过的。也可以是没有提交的，这种情况同时也是脏读。

如果，数据库系统的隔离级别。允许，不可重复读。那么你启动一个事务，并做一个select查询操作。
查询到的数据，就有可能，和你第2次，3次...n次，查询到的数据不一样。一般情况下，你只会做一次，select
查询，并以这一次的查询数据，作为后续计算的基础。因为允许出现，不可重复读。那么任何
时候，查询到的数据，都有可能被其他事务更新，查询的结果将是不确定的。


注：如果允许，不可重复读，你的查询结果，将是不确定的。一个不确定的结果，你能容忍吗？


3，幻读
事务1读取指定的where子句所返回的一些行。然后，事务2插入一个新行，这个新行也满足事务1使用的查询
where子句。然后事务1再次使用相同的查询读取行，但是现在它看到了事务2刚插入的行。这个行被称为幻象，
因为对事务1来说，这一行的出现是不可思议的。


举个例子：
事务1：请求没有预定的，双人床房间列表。99号在其中。
事务2：向Reservation表中插入一个新纪录，以预订99号房间，并提交。
事务1：再次请求有双人床的未预定的房间列表，99号房间，不再位于列表中。


注：幻读，针对的是，Insert操作。如果事务2，插入的记录，没有提交。那么同时也是脏读。