MongoDB fragment

The solution to the growing data

Vertical expansion: increase CPU, hard disk and other storage resources to expand capacity

Horizontal expansion: the data distributed across multiple servers, i.e. to use slicing

MongoDB what fragmentation that?

Is to split the database, all the data stored in the process of dispersing the different servers

Why should achieve MongoDB fragmentation?

advantage:

1. abstract cluster, so clusters "invisible", MongoDB comes with a proprietary process called routing of mongos

2. solve the problem of disk storage space and improve performance of data processing

3. To improve data security, to ensure that the cluster can always read the MongoDB fragmentation and replication combination, ensure that the data in the slice to multiple servers, but also ensure that each have a corresponding sub-data backup so that you can ensure that there is time to replace the server, the other can take over part of the continuing work immediately broken from the library

4. Make clusters easy to expand

Slice cluster major components:

Routers : data routing, and client dealing module

config server Some configuration information for all deposit, withdraw data, the information of all the nodes shard, fragmentation function of:

Shard : The real data storage location, the data stored in the unit of chunk

Deployment MongoDB cluster fragmentation

As shown in FIG deployment environment

1. Installation MongoDB, 8 Configuration Example

2. Create a directory to store data

mkdir -p /usr/local/mongodb/data/shard11

mkdir -p /usr/local/mongodb/data/shard12

mkdir -p /usr/local/mongodb/data/shard21

mkdir -p /usr/local/mongodb/data/shard22

mkdir -p /usr/local/mongodb/data/config1

mkdir -p /usr/local/mongodb/data/config2

mkdir -p /usr/local/mongodb/data/config3

3. Create a log directory and log files and add permissions

mkdir -p /usr/local/mongodb/logs

touch /usr/local/mongodb/logs/router.log

touch /usr/local/mongodb/logs/shard11.log

touch /usr/local/mongodb/logs/shard12.log

touch /usr/local/mongodb/logs/shard21.log

touch /usr/local/mongodb/logs/shard22.log

touch /usr/local/mongodb/logs/config1.log

touch /usr/local/mongodb/logs/config2.log

touch /usr/local/mongodb/logs/config3.log

chmod -R 777 /usr/local/mongodb/logs/router.log

chmod -R 777 /usr/local/mongodb/logs/shard11.log

chmod -R 777 /usr/local/mongodb/logs/shard12.log

chmod -R 777 /usr/local/mongodb/logs/shard21.log

chmod -R 777 /usr/local/mongodb/logs/shard22.log

chmod -R 777 /usr/local/mongodb/logs/config1.log

chmod -R 777 /usr/local/mongodb/logs/config2.log

chmod -R 777 /usr/local/mongodb/logs/config3.log

4. Edit Profile

shard profile

vim /usr/local/mongodb/bin/shard11.conf

vim /usr/local/mongodb/bin/shard12.conf

Compared shard11.conf modify the port number and dbpath and logpath

vim /usr/local/mongodb/bin/shard21.conf

vim /usr/local/mongodb/bin/shard22.conf

Compared shard21.conf modify the port number and dbpath and logpath

config configuration file

vim /usr/local/mongodb/bin/config1.conf

vim /usr/local/mongodb/bin/config2.conf

vim /usr/local/mongodb/bin/config3.conf

Compared config1.conf modify the port number and dbpath and logpath

router configuration file

vim /usr/local/mongodb/bin/router.conf

The promoter fragment and node configuration node

/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/shard11.conf

/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/shard12.conf

/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/shard21.conf

/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/shard22.conf

/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/config1.conf

/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/config2.conf

/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/config3.conf

6. Log constitute replica set configuration node

/usr/local/mongodb/bin/mongo --port 27111

> cfg={_id:'config',members:[{_id:0,host:'127.0.0.1:27111'},{_id:1,host:'127.0.0.1:27222'},{_id:2,host:'127.0.0.1:27333'}]}

7. Log shard two slices nodes constituting replica set

/usr/local/mongodb/bin/mongo --port 27018

> cfg={_id:'shard1',members:[{_id:0,host:'127.0.0.1:27018'},{_id:1,host:'127.0.0.1:27019'}]}

/usr/local/mongodb/bin/mongo --port 27118

> cfg={_id:'shard1',members:[{_id:0,host:'127.0.0.1:27118'},{_id:1,host:'127.0.0.1:27119'}]}

8. Start node routing, fragmentation and increase

/usr/local/mongodb/bin/mongos -f conf/router.conf

mongos> sh.addShard("shard1/127.0.0.1:27018,127.0.0.1:27019")

mongos> sh.addShard("shard1/127.0.0.1:27118,127.0.0.1:27119")

View database cluster fragmentation status

9. enable fragmentation on the attack to the database

MongoDB fragmentation is based, to a collection of fragmented collection (table) carried out, it is necessary to enable it to support fragmentation in the database

mongos> sh.enableSharding("kgc”)

Collection of fragments

First index

mongos> db.kgc.createIndex({"_id":1})

On the test set in the fragment library kgc

sh.shardCollection("kgc.test",{"id":1})

View Status: mongos> sh.status ()