OS ： CentOS7
MongoDB version: 3.6
Install
Create Mongo's yum source file
vi /etc/yum.repos.d/mongodb-org-3.6.repo
Replace with Aliyuan
[mongodb-org-3.6] name=MongoDB Repository baseurl=http://mirrors.aliyun.com/mongodb/yum/redhat/7Server/mongodb-org/3.6/x86_64/ gpgcheck=1 enabled=1 gpgkey=https://www.mongodb.org/static/pgp/server-3.6.asc
install mongo
sudo yum install -y mongodb-org
Create node data directory
$ mkdir -p /data/rs-{a,b}-{1,2,3}
Start the node mongo service
mongod --shardsvr --replSet shard-a --dbpath /data/rs-a-1 --port 30000 --logpath /data/rs-a-1.log --fork mongod --shardsvr --replSet shard-a --dbpath /data/rs-a-2 --port 30001 --logpath /data/rs-a-2.log --fork mongod --shardsvr --replSet shard-a --dbpath /data/rs-a-3 --port 30002 --logpath /data/rs-a-3.log --fork mongod --shardsvr --replSet shard-b --dbpath /data/rs-b-1 --port 30100 --logpath /data/rs-b-1.log --fork mongod --shardsvr --replSet shard-b --dbpath /data/rs-b-2 --port 30101 --logpath /data/rs-b-2.log --fork mongod --shardsvr --replSet shard-b --dbpath /data/rs-b-3 --port 30102 --logpath /data/rs-b-3.log --fork
Initialize the replica set
mongo localhost:30000
> rs.initiate()
Add a node to a replica set
shard-a:PRIMARY> rs.add("localhost:30001")
shard-a:PRIMARY> rs.addArb("localhost:30002")
mongo localhost:30100
> rs.initiate()
shard-b:OTHER> rs.add("localhost:30101")
shard-b:PRIMARY> rs.addArb("localhost:30102")
Create a configuration service node
Create data directory
mkdir /data/config-{1,2,3}
start configuration node
mongod --configsvr --replSet cfgReplSet --dbpath /data/config-1 --port 27019 --logpath /data/config-1.log --fork
mongod --configsvr --replSet cfgReplSet --dbpath /data/config-2 --port 27020 --logpath /data/config-2.log --fork
mongod --configsvr --replSet cfgReplSet --dbpath /data/config-3 --port 27021 --logpath /data/config-3.log --fork
config server initializes replica set
mongo localhost:27019
>rs.initiate({_id:"cfgReplSet",configsvr:true,members:[{_id:0,host:"localhost:27019"},{_id:1,host:"localhost:27020"},{_id:2,host:"localhost:27021"}]})
Start Mongo routing
mongos --configdb cfgReplSet/localhost:27019,localhost:27020,localhost:27021 --logpath /data/mongos.log --fork --port 40000
Configure the cluster
mongo localhost:40000
Shard settings
sh.addShard("shard-a/localhost:30000,localhost:30001")
sh.addShard("shard-b/localhost:30100,localhost:30101")
Verify sharding results
mongos> db.getSiblingDB("config").shards.find()
{ "_id" : "shard-a", "host" : "shard-a/localhost:30000,localhost:30001", "state" : 1 }
{ "_id" : "shard-b", "host" : "shard-b/localhost:30100,localhost:30101", "state" : 1 }
Authenticate with the admin database
mongos> use admin
switched to db admin
mongos> db.runCommand({listshards:1})
{
"shards" : [
{
"_id" : "shard-a",
"host" : "shard-a/localhost:30000,localhost:30001",
"state" : 1
},
{
"_id" : "shard-b",
"host" : "shard-b/localhost:30100,localhost:30101",
"state" : 1
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1524381570, 2),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1524381570, 2)
}
Database sharding
sh.enableSharding("mydb")
mongos> db.getSiblingDB("config").databases.find()
{ "_id" : "mydb", "primary" : "shard-b", "partitioned" : true }
The collection myCol in the shard mydb uses id and username as the shard key
mongos> sh.shardCollection("mydb.myCol",{username:1,_id:1})
{
"collectionsharded" : "mydb.myCol",
"collectionUUID" : UUID("d2ecc073-ed7e-4940-bee5-9e1a45694f23"),
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1524382128, 14),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1524382128, 14)
}
Cluster Shard Verification
Based on the script modification in "MongoDB Cookbook, Second Edition.pdf", insert data in batches, and modify the script to the version based on MongoDB 3.6
ruby load.rb 100
mongos> sh.status()
{ "_id" : "cloud-docs", "primary" : "shard-a", "partitioned" : true }
cloud-docs.spreadsheets
shard key: { "username" : 1, "_id" : 1 }
unique: false
balancing: true
chunks:
shard-a 3
shard-b 2
{ "username" : { "$minKey" : 1 }, "_id" : { "$minKey" : 1 } } -->> { "username" : "Abbott", "_id" : ObjectId("5adc9b211a5680820f0d28b8") } on : shard-b Timestamp(2, 0)
{ "username" : "Abbott", "_id" : ObjectId("5adc9b211a5680820f0d28b8") } -->> { "username" : "Islas", "_id" : ObjectId("5adc9b381a5680820f0d3161") } on : shard-b Timestamp(3, 0)
{ "username" : "Islas", "_id" : ObjectId("5adc9b381a5680820f0d3161") } -->> { "username" : "Schlabach", "_id" : ObjectId("5adc9b511a5680820f0d3b7c") } on : shard-a Timestamp(3, 1)
{ "username" : "Schlabach", "_id" : ObjectId("5adc9b511a5680820f0d3b7c") } -->> { "username" : "Velasques", "_id" : ObjectId("5adc9b381a5680820f0d31a1") } on : shard-a Timestamp(2, 4)
{ "username" : "Velasques", "_id" : ObjectId("5adc9b381a5680820f0d31a1") } -->> { "username" : { "$maxKey" : 1 }, "_id" : { "$maxKey" : 1 } } on : shard-a Timestamp(1, 3)
The shard-a is the main shard of the cloud-docs database. There are three blocks in shard-a and two in shard-b. At the same time, the result shows which shard the data is located in. As the data increases, the blocks in the shard also change. will continue to increase
ruby load.rb 800
number of blocks in the shard
chunks:
shard-a 9
shard-b 6
Data fragmentation results
{ "username" : { "$minKey" : 1 }, "_id" : { "$minKey" : 1 } } -->> { "username" : "Abbott", "_id" : ObjectId("5adc9b211a5680820f0d28b8") } on : shard-a Timestamp(5, 0)
{ "username" : "Abbott", "_id" : ObjectId("5adc9b211a5680820f0d28b8") } -->> { "username" : "Charles", "_id" : ObjectId("5adc9b911a5680820f0d5198") } on : shard-b Timestamp(5, 1)
{ "username" : "Charles", "_id" : ObjectId("5adc9b911a5680820f0d5198") } -->> { "username" : "Grimmett", "_id" : ObjectId("5adc9ddc1a568089ecb9ac91") } on : shard-b Timestamp(4, 3)
{ "username" : "Grimmett", "_id" : ObjectId("5adc9ddc1a568089ecb9ac91") } -->> { "username" : "Islas", "_id" : ObjectId("5adc9b381a5680820f0d3161") } on : shard-b Timestamp(4, 4)
{ "username" : "Islas", "_id" : ObjectId("5adc9b381a5680820f0d3161") } -->> { "username" : "Lafond", "_id" : ObjectId("5adc9f3d1a568089ecba0a26") } on : shard-b Timestamp(4, 5)
{ "username" : "Lafond", "_id" : ObjectId("5adc9f3d1a568089ecba0a26") } -->> { "username" : "Mccluney", "_id" : ObjectId("5adc9b5b1a5680820f0d3f35") } on : shard-b Timestamp(4, 6)
{ "username" : "Mccluney", "_id" : ObjectId("5adc9b5b1a5680820f0d3f35") } -->> { "username" : "Mendoza", "_id" : ObjectId("5adc9dcf1a568089ecb9a70b") } on : shard-b Timestamp(4, 7)
{ "username" : "Mendoza", "_id" : ObjectId("5adc9dcf1a568089ecb9a70b") } -->> { "username" : "Overman", "_id" : ObjectId("5adca05d1a568089ecba7df0") } on : shard-a Timestamp(5, 5)
{ "username" : "Overman", "_id" : ObjectId("5adca05d1a568089ecba7df0") } -->> { "username" : "Rardin", "_id" : ObjectId("5adc9f031a568089ecb9f30e") } on : shard-a Timestamp(5, 6)
{ "username" : "Rardin", "_id" : ObjectId("5adc9f031a568089ecb9f30e") } -->> { "username" : "Royal", "_id" : ObjectId("5adc9b321a5680820f0d2f4d") } on : shard-a Timestamp(5, 7)
{ "username" : "Royal", "_id" : ObjectId("5adc9b321a5680820f0d2f4d") } -->> { "username" : "Schlabach", "_id" : ObjectId("5adc9b511a5680820f0d3b7c") } on : shard-a Timestamp(3, 4)
{ "username" : "Schlabach", "_id" : ObjectId("5adc9b511a5680820f0d3b7c") } -->> { "username" : "Steffensen", "_id" : ObjectId("5adc9e7b1a568089ecb9e8cb") } on : shard-a Timestamp(5, 2)
{ "username" : "Steffensen", "_id" : ObjectId("5adc9e7b1a568089ecb9e8cb") } -->> { "username" : "Vasconcellos", "_id" : ObjectId("5adca0171a568089ecba6793") } on : shard-a Timestamp(5, 3)
{ "username" : "Vasconcellos", "_id" : ObjectId("5adca0171a568089ecba6793") } -->> { "username" : "Velasques", "_id" : ObjectId("5adc9b381a5680820f0d31a1") } on : shard-a Timestamp(5, 4)
{ "username" : "Velasques", "_id" : ObjectId("5adc9b381a5680820f0d31a1") } -->> { "username" : { "$maxKey" : 1 }, "_id" : { "$maxKey" : 1 } } on : shard-a Timestamp(1, 3)