009.MongoDB slice cluster deployment

A preparatory

1.1 Component Description

 
 MongoDB slice cluster contains the following components: 

 
 shard: Each slice is a subset of the data slice. Starting MongoDB 3.6, you must slice deployed for the replica set. 

 
 mongos: mongos act as a query router, provides an interface between client applications and cluster fragmentation. 

 
 config servers: server storage cluster configuration metadata and configuration settings. Starting MongoDB 3.4, you must configure the server to deploy replica set (CSRS). 

 
 Note: mongos do not need to create a replication set, config not need to specify primary and secondary nodes and node arbitration, but to create a replication set. 

1.2 Component Plan

 
 In this study, based on environmental considerations production, assembly planning as follows: 

 
 Config Server 3 will be deployed as a member of the replica set; 

 
 Each Shard will deploy three members of the replica set, to deploy a total of three shard; 

 
 Mongos deploy two routers. 

 
 Tip: mongos deploy multiple routers support high availability and scalability. Mongos common pattern is placed on a per application server can reduce network latency between the applications and routers. 

 
 Mongos router also can be placed on a dedicated host, by a large-scale deployments. Because the number of clients it will separate from the application server mongos number of instances. This allows for better control connection mongod service instance. 

 
 Note: The number mongos routers are not limited. However, due to the mongos often Config Server to communicate with the router, so when increasing the number of router configuration will closely monitor server performance. If you notice a decrease in performance, you can limit the number of appropriate mongos router deployment. 

MongoDB role	A server mongo01	Two server mongo02	Server three mongo03	Four server mongo04	Five server mongo05
mongos				172.24.8.74:27017	172.24.8.75:27017
config server	172.24.8.71:20001 The master node	172.24.8.72:20001 Vice node	172.24.8.73:20001 Arbitration node
shard server 1	172.24.8.71:20002 The master node	172.24.8.72:20002 Vice node	172.24.8.73:20002 仲裁节点
shard server 2	172.24.8.71:20003 仲裁节点	172.24.8.72:20003 主节点	172.24.8.73:20003 副节点
shard server 3	172.24.8.71:20004 副节点	172.24.8.72:20004 仲裁节点	172.24.8.73:20004 主节点

 
 提示：为保证高可用，任何一台物理服务器不应该存在多个主。 

1.3 其他准备

 
 NTP同步； 

 
 关闭SELinux； 

 
 关闭防火墙或放通相应规则； 

 
 添加如下解析至所有节点： 

  1 [root@localhost ~]# vi /etc/hosts		#追加如下解析
  2 172.24.8.71 mongo01
  3 172.24.8.72 mongo02
  4 172.24.8.73 mongo03
  5 172.24.8.74 mongo04
  6 172.24.8.75 mongo05

二 MongoDB安装

2.1 下载tar包

  1 [root@mongo01 ~]# yum -y install libcurl openssl			#安装依赖包
  2 [root@mongo01 ~]# wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-4.0.9.tgz

2.2 解压MongoDB包

  1 [root@mongo01 ~]# tar -zxvf mongodb-linux-x86_64-rhel70-4.0.9.tgz
  2 [root@mongo01 ~]# mv mongodb-linux-x86_64-rhel70-4.0.9 /usr/local/mongodb
  3 [root@mongo01 ~]# useradd mongod -s /bin/false -c mongod
  4 [root@mongo01 ~]# chown -R mongod:mongod /usr/local/mongodb/

2.3 添加PATH路径

  1 [root@mongo01 ~]# echo "export PATH=/usr/local/mongodb/bin::$PATH" >> .bashrc

 
 提示：2.1——2.3需要在所有节点进行操作。 

2.4 配置key

  1 [root@mongo01 ~]# mkdir -p /etc/mongo
  2 [root@mongo01 ~]# openssl rand -base64 100 > /etc/mongo/mongo.key
  3 [root@mongo01 ~]# chown -R mongod:mongod /etc/mongo
  4 [root@mongo01 ~]# chmod 600 /etc/mongo/mongo.key
  5 [root@mongo01 ~]# scp -r /etc/mongo root@mongo02:/etc/
  6 [root@mongo01 ~]# scp -r /etc/mongo root@mongo03:/etc/
  7 [root@mongo01 ~]# scp -r /etc/mongo root@mongo04:/etc/
  8 [root@mongo01 ~]# scp -r /etc/mongo root@mongo05:/etc/
  9 [root@mongo02 ~]# chown -R mongod:mongod /etc/mongo
 10 [root@mongo03 ~]# chown -R mongod:mongod /etc/mongo
 11 [root@mongo04 ~]# chown -R mongod:mongod /etc/mongo
 12 [root@mongo05 ~]# chown -R mongod:mongod /etc/mongo

2.5 创建MongoDB相关目录

  1 [root@mongo01 ~]# mkdir -p /var/log/mongodb		#日志目录
  2 [root@mongo01 ~]# mkdir -p /var/run/mongodb		#pid目录
  3 [root@mongo01 ~]# mkdir -p /var/lib/mongodb/config		#config server存储目录
  4 [root@mongo01 ~]# mkdir -p /var/lib/mongodb/shard{1,2,3}	#shard server存储目录
  5 [root@mongo01 ~]# chown -R mongod:mongod /var/log/mongodb
  6 [root@mongo01 ~]# chown -R mongod:mongod /var/run/mongodb
  7 [root@mongo01 ~]# chown -R mongod:mongod /var/lib/mongodb

 
 注意：通过下载tar包形式安装，未创建默认的MongoDB目录及用户，需要如上所示手动创建并授权即可。 

 
 以上操作需要在mongo01、mongo02、mongo03，即所有config server节点操作。 

三配置config server

3.1 服务器一配置

  1 [root@mongo01 ~]# cat << EOF > /etc/mongo/config.conf
  2 systemLog:
  3   destination: file
  4   logAppend: true
  5   path: /var/log/mongodb/config.log
  6 
  7 storage:
  8   dbPath: /var/lib/mongodb/config
  9   journal:
 10     enabled: true
 11   wiredTiger:
 12     engineConfig:
 13       directoryForIndexes: true
 14 
 15 processManagement:
 16   fork: true
 17   pidFilePath: /var/run/mongodb/config.pid
 18   timeZoneInfo: /usr/share/zoneinfo
 19 
 20 net:
 21   port: 20001
 22   bindIp: 0.0.0.0
 23   maxIncomingConnections: 500
 24 
 25 security:
 26   keyFile: /etc/mongo/mongo.key
 27   authorization: enabled
 28 
 29 replication:
 30   replSetName: configs
 31 
 32 sharding:
 33   clusterRole: configsvr
 34 EOF
 35 [root@mongo01 ~]# scp -r /etc/mongo/config.conf root@mongo02:/etc/mongo/config.conf
 36 [root@mongo01 ~]# scp -r /etc/mongo/config.conf root@mongo03:/etc/mongo/config.conf

3.2 启动mongod

  1 [root@mongo01 ~]# mongod --config /etc/mongo/config.conf
  2 [root@mongo02 ~]# mongod --config /etc/mongo/config.conf
  3 [root@mongo03 ~]# mongod --config /etc/mongo/config.conf

3.3 初始化configsrv副本集群

  1 [root@mongodb01 ~]# mongo --port 20001
  2 > config = { _id: "configs", members: [
  3 {_id: 0, host: "172.24.8.71:20001"},
  4 {_id: 1, host: "172.24.8.72:20001"},
  5 {_id: 2, host: "172.24.8.73:20001"}]
  6 }

 
 参数解释： 

 
 "_id": 副本集的名称 

 
 "members": 副本集的服务器列表 

 
 "_id": 服务器的唯一ID 

 
 "host": 服务器主机 

 
 "priority": 是优先级，默认为1，优先级0为被动节点，不能成为活跃节点。优先级不为0则按照有大到小选出活跃节点。 

 
 "arbiterOnly": 仲裁节点，只参与投票，不接收数据，也不能成为活跃节点。 

3.4 初始化副本集

  1 > rs.initiate(config)			#初始化集群
  2 my_rep1:PRIMARY> rs.status()		#查看集群状态

四配置shard server 1

4.1 服务器一配置

  1 [root@mongo01 ~]# cat << EOF > /etc/mongo/shard1.conf
  2 systemLog:
  3   destination: file
  4   logAppend: true
  5   path: /var/log/mongodb/shard1.log
  6 
  7 storage:
  8   dbPath: /var/lib/mongodb/shard1
  9   journal:
 10     enabled: true
 11   wiredTiger:
 12     engineConfig:
 13       directoryForIndexes: true
 14 
 15 processManagement:
 16   fork: true
 17   pidFilePath: /var/run/mongodb/shard1.pid
 18   timeZoneInfo: /usr/share/zoneinfo
 19 
 20 net:
 21   port: 20002
 22   bindIp: 0.0.0.0
 23   maxIncomingConnections: 500
 24 
 25 security:
 26   keyFile: /etc/mongo/mongo.key
 27   authorization: enabled
 28 
 29 replication:
 30   replSetName: shard1
 31 
 32 sharding:
 33   clusterRole: shardsvr
 34 EOF
 35 [root@mongo01 ~]# scp -r /etc/mongo/shard1.conf root@mongo02:/etc/mongo/shard1.conf
 36 [root@mongo01 ~]# scp -r /etc/mongo/shard1.conf root@mongo03:/etc/mongo/shard1.conf

4.2 启动mongod

  1 [root@mongo01 ~]# mongod --config /etc/mongo/shard1.conf
  2 [root@mongo02 ~]# mongod --config /etc/mongo/shard1.conf
  3 [root@mongo03 ~]# mongod --config /etc/mongo/shard1.conf

4.3 初始化configsrv副本集群

  1 [root@mongodb01 ~]# mongo --port 20002
  2 > config = { _id: "shard1", members: [
  3 {_id: 0, host: "172.24.8.71:20002",priority:2},
  4 {_id: 1, host: "172.24.8.72:20002",priority:1},
  5 {_id: 2, host: "172.24.8.73:20002",arbiterOnly:true}]
  6 }

4.4 初始化副本集

  1 > rs.initiate(config)			#初始化集群
  2 my_rep1:PRIMARY> rs.status()		#查看集群状态

五配置shard server 2

5.1 服务器一配置

  1 [root@mongo01 ~]# cat << EOF > /etc/mongo/shard2.conf
  2 systemLog:
  3   destination: file
  4   logAppend: true
  5   path: /var/log/mongodb/shard2.log
  6 
  7 storage:
  8   dbPath: /var/lib/mongodb/shard2
  9   journal:
 10     enabled: true
 11   wiredTiger:
 12     engineConfig:
 13       directoryForIndexes: true
 14 
 15 processManagement:
 16   fork: true
 17   pidFilePath: /var/run/mongodb/shard2.pid
 18   timeZoneInfo: /usr/share/zoneinfo
 19 
 20 net:
 21   port: 20003
 22   bindIp: 0.0.0.0
 23   maxIncomingConnections: 500
 24 
 25 security:
 26   keyFile: /etc/mongo/mongo.key
 27   authorization: enabled
 28 
 29 replication:
 30   replSetName: shard2
 31 
 32 sharding:
 33   clusterRole: shardsvr
 34 EOF
 35 [root@mongo01 ~]# scp -r /etc/mongo/shard2.conf root@mongo02:/etc/mongo/shard2.conf
 36 [root@mongo01 ~]# scp -r /etc/mongo/shard2.conf root@mongo03:/etc/mongo/shard2.conf

5.2 启动mongod

  1 [root@mongo01 ~]# mongod --config /etc/mongo/shard2.conf
  2 [root@mongo02 ~]# mongod --config /etc/mongo/shard2.conf
  3 [root@mongo03 ~]# mongod --config /etc/mongo/shard2.conf

5.3 初始化configsrv副本集群

  1 [root@mongodb02 ~]# mongo --port 20003
  2 > config = { _id: "shard2", members: [
  3 {_id: 0, host: "172.24.8.71:20003",arbiterOnly:true},
  4 {_id: 1, host: "172.24.8.72:20003",priority:2},
  5 {_id: 2, host: "172.24.8.73:20003",priority:1}]
  6 }

 
 提示：如果选择在预先规划为仲裁节点的服务器上创建复制集会报错，需要在非仲裁节点创建。 

 
 5.4 初始化副本集 

  1 > rs.initiate(config)			#初始化集群
  2 my_rep1:PRIMARY> rs.status()		#查看集群状态

六配置shard server 3

6.1 服务器一配置

  1 [root@mongo01 ~]# cat << EOF > /etc/mongo/shard3.conf
  2 systemLog:
  3   destination: file
  4   logAppend: true
  5   path: /var/log/mongodb/shard3.log
  6 
  7 storage:
  8   dbPath: /var/lib/mongodb/shard3
  9   journal:
 10     enabled: true
 11   wiredTiger:
 12     engineConfig:
 13       directoryForIndexes: true
 14 
 15 processManagement:
 16   fork: true
 17   pidFilePath: /var/run/mongodb/shard3.pid
 18   timeZoneInfo: /usr/share/zoneinfo
 19 
 20 net:
 21   port: 20004
 22   bindIp: 0.0.0.0
 23   maxIncomingConnections: 500
 24 
 25 security:
 26   keyFile: /etc/mongo/mongo.key
 27   authorization: enabled
 28 
 29 replication:
 30   replSetName: shard3
 31 
 32 sharding:
 33   clusterRole: shardsvr
 34 EOF
 35 [root@mongo01 ~]# scp -r /etc/mongo/shard3.conf root@mongo02:/etc/mongo/shard3.conf
 36 [root@mongo01 ~]# scp -r /etc/mongo/shard3.conf root@mongo03:/etc/mongo/shard3.conf

6.2 启动mongod

  1 [root@mongo01 ~]# mongod --config /etc/mongo/shard3.conf
  2 [root@mongo02 ~]# mongod --config /etc/mongo/shard3.conf
  3 [root@mongo03 ~]# mongod --config /etc/mongo/shard3.conf

6.3 初始化configsrv副本集群

  1 [root@mongodb01 ~]# mongo --port 20004
  2 > config = { _id: "shard3", members: [
  3 {_id: 0, host: "172.24.8.71:20004",priority:1},
  4 {_id: 1, host: "172.24.8.72:20004",arbiterOnly:true},
  5 {_id: 2, host: "172.24.8.73:20004",priority:2}]
  6 }

 
 提示：如果选择在预先规划为仲裁节点的服务器上创建复制集会报错，需要在非仲裁节点创建。 

6.4 初始化副本集

  1 > rs.initiate(config)			#初始化集群
  2 my_rep1:PRIMARY> rs.status()		#查看集群状态

七配置mongos路由节点

7.1 创建MongoDB相关目录

  1 [root@mongo04 ~]# mkdir -p /var/log/mongodb		#日志目录
  2 [root@mongo04 ~]# mkdir -p /var/run/mongodb		#pid目录
  3 [root@mongo04 ~]# chown -R mongod:mongod /var/log/mongodb
  4 [root@mongo04 ~]# chown -R mongod:mongod /var/run/mongodb

 
 注意：通过下载tar包形式安装，未创建默认的MongoDB目录及用户，需要如上所示手动创建并授权即可。 

 
 以上操作需要在mongo04、mongo05即所有mongos server节点操作。 

7.2 服务器四配置

  1 [root@mongo04 ~]# cat << EOF > /etc/mongo/mongos.conf
  2 systemLog:
  3   destination: file
  4   logAppend: true
  5   path: /var/log/mongodb/mongos.log
  6 
  7 processManagement:
  8   fork: true
  9   pidFilePath: /var/run/mongodb/mongos.pid
 10   timeZoneInfo: /usr/share/zoneinfo
 11 
 12 net:
 13   port: 27017
 14   bindIp: 0.0.0.0
 15   maxIncomingConnections: 2000
 16 
 17 security:
 18   keyFile: /etc/mongo/mongo.key
 19 #  authorization: enabled
 20 
 21 sharding:
 22   configDB: configs/172.24.8.71:20001,172.24.8.72:20001,172.24.8.73:20001
 23 EOF
 24 [root@mongo04 ~]# scp -r /etc/mongo/mongos.conf root@mongo05:/etc/mongo/mongos.conf

7.3 启动mongos

  1 [root@mongo04 ~]# mongos --config /etc/mongo/mongos.conf
  2 [root@mongo05 ~]# mongos --config /etc/mongo/mongos.conf

7.4 创建管理员用户

  1 [root@mongodb04 ~]# mongo --port 27017
  2 mongos> use admin				#进入admin数据库
  3 mongos> db.createUser({ user: "admin", pwd: "admin", roles: [{ role: "_system", db: "admin" }] })
  4 mongos> db.createUser({"user":"root","pwd":"root","roles":["root"]})
  5 mongos> db.auth("admin", "admin")		#验证创建结果
  6 1

 
 提示：以上为建议项，建议创建一个管理员用于内部管理MongoDB。 

 
 mongodb中的用户是基于身份role的，该管理员账户的 role是 userAdminAnyDatabase。 ‘userAdmin’代表用户管理身份，’AnyDatabase’ 代表可以管理任何数据库。 

 
 db.auth()可以验证 用户。 

  1 mongos> db.getUsers()
  2 mongos> db.system.users.find().pretty()		#查看全局所有账户

 
 提示：mongo shell 可通过.pretty() 对输出进行JSON格式化，以便提高输出的可读性。 

八开启分片功能

8.1 启用分片功能

  1 [root@mongo04 ~]# mongo --port 27017 -u admin -p admin
  2 mongos> use amdin
  3 mongos> sh.addShard("shard1/172.24.8.71:20002,172.24.8.72:20002,172.24.8.73:20002")
  4 mongos> sh.addShard("shard2/172.24.8.71:20003,172.24.8.72:20003,172.24.8.73:20003")

  1 mongos> sh.status()		#查看分片集群状态

8.2 设置分片chunk大小

  1 mongos> use config
  2 mongos> db.settings.save({"_id":"chunksize","value":1})		#设置分片大小为1M便于测试

8.3 模拟数据写入

  1 mongos> use mydb				#创建用于模拟的数据库
  2 mongos> for(i=1;i<=50000;i++){db.user.insert({"id":i,"name":"jack"+i})}
  3 #模拟写入5W条数据到user集合
  4 WriteResult({ "nInserted" : 1 })		#输出

8.4 启用数据库分片

  1 mongos> sh.enableSharding("mydb")

8.5 创建索引

  1 mongos> db.user.createIndex({"id": 1})

8.6 启用表分片

  1 mongos> sh.shardCollection("mydb.user",{"id": 1})

8.7 查看分片情况

  1 mongos> sh.status()

8.8 手动添加第三个分片

  1 mongos> sh.addShard("shard3/172.24.8.71:20004,172.24.8.72:20004,172.24.8.73:20004")

8.9 再次观察分片情况

  1 mongos> sh.status()

 
 提示：当分片有变动时，服务器会对数据进行重新分片（均衡），当你再次移除一个分片服务器，此时又会对数据再次进行分片处理。 

 
 相关参考： 

 
 https://blog.51cto.com/bigboss/2160311 

 
 https://blog.51cto.com/13643643/2148825 

009.MongoDB slice cluster deployment

A preparatory

1.1 Component Description

1.2 Component Plan

1.3 其他准备

二 MongoDB安装

2.1 下载tar包

2.2 解压MongoDB包

2.3 添加PATH路径

2.4 配置key

2.5 创建MongoDB相关目录

三 配置config server

3.1 服务器一配置

3.2 启动mongod

3.3 初始化configsrv副本集群

3.4 初始化副本集

四 配置shard server 1

4.1 服务器一配置

4.2 启动mongod

4.3 初始化configsrv副本集群

4.4 初始化副本集

五 配置shard server 2

5.1 服务器一配置

5.2 启动mongod

5.3 初始化configsrv副本集群

六 配置shard server 3

6.1 服务器一配置

6.2 启动mongod

6.3 初始化configsrv副本集群

6.4 初始化副本集

七 配置mongos路由节点

7.1 创建MongoDB相关目录

7.2 服务器四配置

7.3 启动mongos

7.4 创建管理员用户

八 开启分片功能

8.1 启用分片功能

8.2 设置分片chunk大小

8.3 模拟数据写入

8.4 启用数据库分片

8.5 创建索引

8.6 启用表分片

8.7 查看分片情况

8.8 手动添加第三个分片

8.9 再次观察分片情况

Guess you like

三配置config server

四配置shard server 1

五配置shard server 2

六配置shard server 3

七配置mongos路由节点

八开启分片功能