MySQL standardized, automated deployment

Job title

1. Why data directories and log directory need to be separated?
2. How to configure a standardized multi-instance? (Example: 3306 and 3307 to deploy two instances on a single physical host)
3. Detailed description of the installation process of compiling MySQL (screenshot mounting step)

1. Why data directories and log directory need to be separated?

Here are separated, my understanding is that the MySQL data directory and log directory were placed in different types of disks.
Suppose there are two types of production environments disk on the server, respectively, and SAS SSD, than SAS SSD faster response time (SSD response time of about 0.1 ms, SAS response time of about 10 milliseconds), in order to better use of disk , will generally active data onto SSD, SAS cold data onto the disk.
Data in the random access data directory is typically thermal data, into the SSD disc have a higher response speed;
logs in the log directory is writable data sequence for cold, into SAS disk write log meet high throughput the amount of demand.

2. How to configure multiple instances standardization?

Standardized configuration multiple instances, each instance primarily standardized directory and memory settings, so that parameter settings for each instance is also very easy to achieve standardization. MYSQL instance standardized configuration to facilitate the implementation of the monitoring and operation and maintenance management.
MySQL instance due to take up a physical memory 64G, so the higher the physical memory on the server, MySQL typically install multiple instances, MySQL examples, are distinguished by the port.

For example: 3306 and 3307 deployed on two MYSQL instances of a physical host 64G

Standardized catalog:

1 2 3 4 5 6 7 8 9 10

Example 1: /data/my3306 /log/my3306 Example 2: /data/my3307 /log/my3307 # / Data directory is mounted onto the SSD # / Log directory is mounted to the SAS disk

标准化内存：

1 2 3 4 5 6 7 8 9 10

实例1： 15G: InnoDB buffer cache 5G : mysql server层 实例2： 15G: InnoDB buffer cache 5G : mysql server层 OS: 24G

标准化参数：

结合标准化的目录及内存设置，设置标准化的参数

3. 详细描述MySQL编译安装的过程

关闭防火墙和SELINUX

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37

service iptables status ------------------------------------------------ Table: filter Chain INPUT (policy ACCEPT) num target prot opt source destination 1 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED 2 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 3 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 4 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22 5 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited Chain FORWARD (policy ACCEPT) num target prot opt source destination 1 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited Chain OUTPUT (policy ACCEPT) num target prot opt source destination ------------------------------------------------ service iptables stop ------------------------------------------------ iptables: Setting chains to policy ACCEPT: filter [ OK ] iptables: Flushing firewall rules: [ OK ] iptables: Unloading modules: [ OK ] ------------------------------------------------ service iptables status ------------------------------------------------ iptables: Firewall is not running. ------------------------------------------------ chkconfig iptables off vi /etc/selinux/config ------------------------------------------------ SELINUX=disabled ------------------------------------------------

配置sysctl.conf

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

# 查看服务器内存 free ------------------------------------------------ total used free shared buffers cached Mem: 8174352 616628 7557724 172 151904 253892 -/+ buffers/cache: 210832 7963520 Swap: 16531452 0 16531452 ------------------------------------------------ vi /etc/sysctl.conf ------------------------------------------------ # 修改 kernel.shmmax = 4398046511104 # 添加 fs.file-max = 6815744 kernel.sem = 250 32000 100 128 kernel.shmmni = 4096 kernel.panic_on_oops = 1 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048576 fs.aio-max-nr = 1048576 net.ipv4.ip_local_port_range = 9000 65500 ------------------------------------------------ # kernel.shmmax算法：修改为物理内容的50%、60% # 8G:kernel.shmmax = (8G*1024*1024*1024*1024)*50% = 4398046511104 # 使配置立即生效 sysctl -p

检查是否已安装MySQL

1 2 3 4 5 6 7

rpm -qa | grep mysql ------------------------------------------------ mysql-libs-5.1.73-7.el6.x86_64 ------------------------------------------------ # 删除mysql-libs-5.1.73-7.el6.x86_64包 rpm -e --nodeps mysql-libs-5.1.73-7.el6.x86_64

下载MySQL源码

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

Download MySQL Community Server https://dev.mysql.com/downloads/mysql/5.6.html#downloads Select Version: 5.6.35 -> Select Platform: Source Code -> 选择【Generic Linux (Architecture Independent), Compressed TAR Archive】下载 # 配置yum源，安装lrzsz(代替ftp上传和下载的工具) mkdir /media/disk mkdir /media/cdrom mount /dev/cdrom /media/cdrom cp -rf /media/cdrom/* /media/disk umount /media/cdrom cp /etc/yum.repos.d/public-yum-ol6.repo /etc/yum.repos.d/public-yum-ol6.repo.bak vi /etc/yum.repos.d/public-yum-ol6.repo ------------------------------------------------ name=Oracle Linux $releasever Latest ($basearch) baseurl=file:///media/disk/Server gpgcheck=0 enabled=1 ------------------------------------------------ yum -y install lrzsz # 上传到服务器的/u01目录下 mkdir /u01 cd /u01 rz # 选择mysql源码包，上传 rz waiting to receive. Starting zmodem transfer. Press Ctrl+C to cancel. Transferring mysql-5.6.35.tar.gz... 100% 31413 KB 15706 KB/sec 00:00:02 0 Errors ll /u01/mysql* -rw-r--r--. 1 root root 32167628 Jan 17 11:16 /u01/mysql-5.6.35.tar.gz

添加MySQL用户和组

1 2 3 4 5 6

groupadd -g 501 mysql useradd -u 501 mysql -g mysql echo "mysql123" | passwd --stdin mysql id mysql uid=501(mysql) gid=501(mysql) groups=501(mysql)

配MySQL环境变量

1 2 3 4

vi /home/mysql/.bash_profile ------------------------------------------------ PATH=$PATH:$HOME/bin:/u01/my3306/bin ------------------------------------------------

创建目录及授权

1 2 3 4 5 6 7 8

mkdir -p /u01/my3306/data mkdir -p /u01/my3306/log/iblog mkdir -p /u01/my3306/log/binlog mkdir -p /u01/my3306/run mkdir -p /u01/my3306/tmp chown -R mysql:mysql /u01/my3306 chmod -R 755 /u01/my3306

解压mysql5.6

1 2	cd /u01 tar xvpf mysql-5.6.35.tar.gz

安装cmake及相关依赖包

1	yum install -y cmake gcc gcc-c++ ncurses-devel bison zlib libxml openssl

编译并安装

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

cd /u01/mysql-5.6.35 cmake -DCMAKE_INSTALL_PREFIX=/u01/my3306 -DINSTALL_DATADIR=/u01/my3306/data -DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATION=utf8_general_ci -DEXTRA_CHARSETS=all -DWITH_SSL=yes -DWITH_EMBEDDED_SERVER=1 -DENABLED_LOCAL_INFILE=1 -DWITH_MYISAM_STORAGE_ENGINE=1 -DWITH_INNOBASE_STORAGE_ENGINE=1 -DWITH_ARCHIVE_STORAGE_ENGINE=1 -DWITH_BLACKHOLE_STORAGE_ENGINE=1 -DWITH_FEDERATED_STORAGE_ENGINE=1 -DWITH_PARTITION_STORAGE_ENGINE=1 -DMYSQL_UNIX_ADDR=/u01/my3306/run/mysql.sock -DMYSQL_TCP_PORT=3306 -DENABLED_LOCAL_INFILE=1 -DSYSCONFDIR=/etc -DWITH_READLINE=on 第一次CMAKE出现错误提示 --------------------------------------------------------------------------------------------- CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: OPENSSL_INCLUDE_DIR used as include directory in directory /u01/mysql-5.6.35/CMakeFiles/CMakeTmp --------------------------------------------------------------------------------------------- 安装openssl-devel包 yum -y install openssl-devel 重新cmake需要删除当前目录下CMakeCache.txt，然后再重新执行 rm -rf CMakeCache.txt 编译并安装 make make install

MySQL参数配置

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96

cd /u01/my3306 vi my. ---------------------------------------------------------- [client] port=3306 socket=/u01/my3306/mysql.sock [mysql] pid_file=/u01/my3306/run/mysqld.pid [mysqld] autocommit=1 general_log=off explicit_defaults_for_timestamp=true # system basedir=/u01/my3306 datadir=/u01/my3306/data max_allowed_packet=1g max_connections=3000 max_user_connections=2800 open_files_limit=65535 pid_file=/u01/my3306/run/mysqld.pid port=3306 server_id=101 skip_name_resolve=ON socket=/u01/my3306/run/mysql.sock tmpdir=/u01/my3306/tmp #binlog log_bin=/u01/my3306/log/binlog/binlog binlog_cache_size=32768 binlog_format=row expire_logs_days=7 log_slave_updates=ON max_binlog_cache_size=2147483648 max_binlog_size=524288000 sync_binlog=100 #logging log_error=/u01/my3306/log/error.log slow_query_log_file=/u01/my3306/log/slow.log log_queries_not_using_indexes=0 slow_query_log=1 log_slave_updates=ON log_slow_admin_statements=1 long_query_time=1 #relay relay_log=/u01/my3306/log/relaylog relay_log_index=/u01/my3306/log/relay.index relay_log_info_file=/u01/my3306/log/relay-log.info #slave slave_load_tmpdir=/u01/my3306/tmp slave_skip_errors=OFF #innodb innodb_data_home_dir=/u01/my3306/log/iblog innodb_log_group_home_dir=/u01/my3306/log/iblog innodb_adaptive_flushing=ON innodb_adaptive_hash_index=ON innodb_autoinc_lock_mode=1 innodb_buffer_pool_instances=8 #default innodb_change_buffering=inserts innodb_checksums=ON innodb_buffer_pool_size= 128M innodb_data_file_path=ibdata1:32M;ibdata2:16M:autoextend innodb_doublewrite=ON innodb_file_format=Barracuda innodb_file_per_table=ON innodb_flush_log_at_trx_commit=1 innodb_flush_method=O_DIRECT innodb_io_capacity=1000 innodb_lock_wait_timeout=10 innodb_log_buffer_size=67108864 innodb_log_file_size=1048576000 innodb_log_files_in_group=4 innodb_max_dirty_pages_pct=60 innodb_open_files=60000 innodb_purge_threads=1 innodb_read_io_threads=4 innodb_stats_on_metadata=OFF innodb_support_xa=ON innodb_use_native_aio=OFF innodb_write_io_threads=10 [mysqld_safe] datadir=/u01/my3306/data ---------------------------------------------------------- # 编译后重新修改目录权限 chown -R mysql:mysql /u01/my3306

初始化MySQL脚本

1 2 3 4

su - mysql cd /u01/my3306/scripts ./mysql_install_db --defaults-file=/u01/my3306/my.cnf --datadir=/u01/my3306/data --basedir=/u01/my3306 --user=mysql

启动MySQL

1	/u01/my3306/bin/mysqld_safe --defaults-file=/u01/my3306/my.cnf --user=mysql &

登录MySQL

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

mysql 或者 mysql -h127.0.0.1 -uroot Welcome to the MySQL monitor. Commands end with ; or g. Your MySQL connection id is 2 Server version: 5.6.35-log Source distribution Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | test | +--------------------+ 4 rows in set (0.01 sec)

Original: Large columns  MySQL standardized, automated deployment