NoSQL之MongoDB副本集和文档管理

MongoDB副本集--MongoDB复制

多个服务器上存储数据副本，实现数据同步

提高数据高可用性，安全性，方便数据故障恢复

原理

两个或两个节点以上，一个主节点，负责处理客户端请求，其余为从节点，负责复制主节点数据
搭配方式：一主一从，一主多从
主节点记录所有操作oplog，从节点定期轮询主节点获取这些操作，然后对自己的数据副本执行这些操作，保证从与主节点数据一致

实现方式

master-slave主从复制

启动一台加上“-master”参数为主节点，其他加上“-slave”和“-source”为从节点

从节点可以提供数据查询，降低主节点的访问，并且可以执行备份，避免锁定主节点数据

主节点故障时，可以快速切换到从节点，实现高可用

replica sets 副本集--从1.6版本开始支持，优于之前

支持故障自动切换，自动修复成员节点，降低运维成本，此结构类似于高可用集群

配置replica sets

修改配置文件，指定主机所在副本集名称，所有主机副本名称一致

[root@redis51 ~]# vim /usr/local/mongodb/etc/mongodb.conf   //52和53同样操作
logpath=/usr/local/mongodb/log/mongodb.log
logappend=true
dbpath=/usr/local/mongodb/data/db
fork=true
bind_ip=192.168.4.51
port=27051
replSet=rs1                           //指定副本集名称

启动主从节点服务，查看端口号，可以利用pssh同时远程操控51,52,53

注：可以传密钥给要操作的三台主机，免密登录，并且配置域名解析对应主机名，主机名写入文件！

[root@guo ~]# tail -3  /etc/hosts    //解析三台主机
192.168.4.51 host1
192.168.4.52 host2
192.168.4.53 host3
[root@guo ~]# cat host.txt        //写入host文件
root@host1
host2
host3                
[root@guo ~]# ssh-keygen -N  ''   -f /root/.ssh/id_rsa      //非交互生成密钥文件
[root@guo ~]# ssh-copy-id  host1                            //其余两台同样发送密钥

[root@guo ~]# pssh -i -h  host.txt  /usr/local/mongodb/bin/mongod -f  /usr/local/mongodb/etc/mongodb.conf   
[1] 09:26:08 [SUCCESS] host2
about to fork child process, waiting until server is ready for connections.
forked process: 1958
child process started successfully, parent exiting
[2] 09:26:09 [SUCCESS] host3
...
[3] 09:26:09 [SUCCESS] root@host1
...
[root@guo ~]# pssh -i -h host.txt  netstat -pntul | grep 270
tcp      0    0 tcp192.168.4.51:27051    0.0.0.0:*          LISTEN      2006/mongod         
tcp      0    0 192.168.4.53:27053       0.0.0.0:*          LISTEN      1963/mongod         
tcp      0    0 192.168.4.52:27052       0.0.0.0:*          LISTEN      1958/mongod

配置节点信息，在任意一台操作即可，链接mongod服务，执行如下命令

[root@mongo51 ~]# /usr/local/mongodb/bin/mongo --host 192.168.4.51 --port 27051
> config = { _id:"rs1",members:[{_id:0,host:"192.168.4.51:27051"},{_id:1,host:"192.168.4.52:27052"},{_id:2,host:"192.168.4.53:27053"}]}   //显示如下信息
{
	"_id" : "rs1",
	"members" : [
		{
			"_id" : 0,
			"host" : "192.168.4.51:27051"
		},
		{
			"_id" : 1,
			"host" : "192.168.4.52:27052"
		},
		{
			"_id" : 2,
			"host" : "192.168.4.53:27053"
		}
	]
}
> rs.help()                                                        //可以查看帮助信息

出错解决办法

进入配置文件注释掉副本集

关闭服务，重新启动

进入mongod服务，删除local库

然后把注释去掉，重新启动服务，重新配置

初始化副本集 rs.initiate（config）

> rs.initiate(config)
{
	"ok" : 1,                         //显示为1，成功
	"operationTime" : Timestamp(1545789902, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1545789902, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
rs1:SECONDARY>                            //回车
rs1:PRIMARY> show dbs                     //当前主机为主库，主库可以查看数据，从库不可以
admin   0.000GB
config  0.000GB
local   0.000GB
rs1:PRIMARY> use local     
switched to db local
rs1:PRIMARY> show tables            
me                
oplog.rs
replset.election
replset.minvalid
startup_log
system.replset
system.rollback.id

查看副本集信息（在51上为例）

rs1:PRIMARY> rs.status()                    //查看状态信息
{
"set" : "rs1",
	"date" : ISODate("2018-12-26T02:29:25.735Z"),
	"myState" : 1,
...
	},
	"members" : [
		{
			"_id" : 0,
			"name" : "192.168.4.51:27051",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
...
},
		{
			"_id" : 1,
			"name" : "192.168.4.52:27052",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
...
	},
		{
			"_id" : 2,
			"name" : "192.168.4.53:27053",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
...

rs1:PRIMARY> rs.isMaster()                   //查看是否为主库
{
	"hosts" : [
		"192.168.4.51:27051",
		"192.168.4.52:27052",
		"192.168.4.53:27053"
	],
	"setName" : "rs1",
	"setVersion" : 1,
	"ismaster" : true,
	"secondary" : false,
	"primary" : "192.168.4.51:27051",
	"me" : "192.168.4.51:27051",
...

测试数据同步 客户端链接主库，存储文档，从库验证（需开启从库同步数据，db.getMongo().setSlaveOk() )

[root@mongo50 ~]# /usr/local/mongondb/bin/mongo --host 192.168.4.51 --port 27051
rs1:PRIMARY> db.test.save({name:"bob",sex:"m"})          //写入数据
WriteResult({ "nInserted" : 1 })
rs1:PRIMARY> show tables
test
rs1:PRIMARY> db.test.find()
{ "_id" : ObjectId("5c22f411ba77fd8902a24eb9"), "name" : "bob", "sex" : "m" }

rs1:SECONDARY> db.getMongo().setSlaveOk()       //在从库上验证
rs1:SECONDARY> use test
switched to db test
rs1:SECONDARY> show tables
test
rs1:SECONDARY> db.test.find()
{ "_id" : ObjectId("5c22f411ba77fd8902a24eb9"), "name" : "bob", "sex" : "m" }

测试高可用

停止当前的主库51，会在52和53从库中选取新的主库，未被选取为主库的主机自动做新主库的从库，客户端链接新主库存储文档

rs1:PRIMARY> rs.isMaster()              //在53上查看，发现为主库
{
	"hosts" : [
		"192.168.4.51:27051",
		"192.168.4.52:27052",
		"192.168.4.53:27053"
	],
	"setName" : "rs1",
	"setVersion" : 1,
	"ismaster" : true,
	"secondary" : false,
	"primary" : "192.168.4.53:27053",
	"me" : "192.168.4.53:27053",
...
rs1:PRIMARY> db.test.save({name:"rose"})         //在53上写入数据
WriteResult({ "nInserted" : 1 })
rs1:PRIMARY>                         

rs1:SECONDARY> rs.isMaster()               //在52上查看，发现当前为从库
{
	"hosts" : [
		"192.168.4.51:27051",
		"192.168.4.52:27052",
		"192.168.4.53:27053"
	],
	"setName" : "rs1",
	"setVersion" : 1,
	"ismaster" : false,
	"secondary" : true,
	"primary" : "192.168.4.53:27053",
	"me" : "192.168.4.52:27052",
rs1:SECONDARY> db.test.find()
{ "_id" : ObjectId("5c22f411ba77fd8902a24eb9"), "name" : "bob", "sex" : "m" }
{ "_id" : ObjectId("5c22fcebec89744f346d253f"), "name" : "rose" }

[root@mongo51 ~]# /usr/local/mongodb/bin/mongod -f /usr/local/mongodb/etc/mongodb.conf                //重启51 链接数据库
[root@mongo51 ~]# /usr/local/mongodb/bin/mongo --host 192.168.4.51 --port 27051
rs1:SECONDARY> rs.isMaster()
{
	"hosts" : [
		"192.168.4.51:27051",
		"192.168.4.52:27052",
		"192.168.4.53:27053"
	],
	"setName" : "rs1",
	"setVersion" : 1,
	"ismaster" : false,
	"secondary" : true,
	"primary" : "192.168.4.53:27053",
	"me" : "192.168.4.51:27051",
...
rs1:SECONDARY> rs.slaveOk( )
rs1:SECONDARY> db.test.find()                 //发现已经把离线的数据恢复，并自动成为从库
{ "_id" : ObjectId("5c22f411ba77fd8902a24eb9"), "name" : "bob", "sex" : "m" }
{ "_id" : ObjectId("5c22fcebec89744f346d253f"), "name" : "rose" }

恢复单独的mongodb服务

停止服务，注释掉配置文件中的副本集名称，重启服务，删除local库，再次重启！（51上验证，52/53同操作）

[root@mongo51 ~]# /usr/local/mongodb/bin/mongod -f /usr/local/mongodb/etc/mongodb.conf   --shutdown
[root@mongo51 ~]# vim /usr/local/mongodb/etc/mongodb.conf
logpath=/usr/local/mongodb/log/mongodb.log
logappend=true
dbpath=/usr/local/mongodb/data/db
fork=true
bind_ip=192.168.4.51
port=27051
#replSet=rs1                 //注释掉
[root@mongo51 ~]#/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/etc/mongodb.conf 
[root@mongo51 ~]#/usr/local/mongodb/bin/mongo --host 192.168.4.51 --port 27051 
> use local
switched to db local
> db.dropDatabase()
{ "dropped" : "local", "ok" : 1 }
[root@mongo51 ~]# /usr/local/mongodb/bin/mongod -f /usr/local/mongodb/etc/mongodb.conf   --shutdown
[root@mongo51 ~]# /usr/local/mongodb/bin/mongod -f /usr/local/mongodb/etc/mongodb.conf
[root@mongo51~]# /usr/local/mongodb/bin/mongo --host 192.168.4.51 --port 27051 
> show dbs
admin   0.000GB
config  0.000GB
local   0.000GB
test    0.000GB

文档管理

插入文档	db.集合名.save({key:"value",key:"value"})	集合不存在则创建，写入记录，_id字段值存在，修改文档字段值
	db.集合名.insert({key:"value",key:"value"})	集合不存在创建，写入，_id字段值存在，放弃插入，不存在，写入
	db.集合名.insertMany([{key:"value"},{key:"value"}])	插入多条记录
查询文档	db.集合名.find().[count()]	显示所有行，默认20，输入it可显示后面的 count()统计总数
	db.集合名.findOne()	显示第一行
	db.集合名.find({条件},{定义显示的字段}) db.集合名.find({key:"值",keyname:"值"})	指定查询条件并指定显示的字段，字段：0 不显示，字段：1显示不写条件，则显示符合定义字段的所有行
行数显示控制	db.集合名.find().limit(num)	显示前几行
	db.集合名.find().skip(num)	跳过前几行
	db.集合名.find().sort(字段名:1/-1)	1升序显示，-1降序显示
范围比较	db.集合名.find({key:{$in:["值1"，”值2“，”值3“]}})	在...里
	db.集合名.find({key:{$nin:["值1"，”值2“，”值3“]}})	不在...里
	db.集合名.find({$or:[{key:"值1"},{keyname:"值2"}]})	或
正则匹配	db.集合名.find({key:/ 正则规则/})	正则匹配
数值比较	$lt $lte $gt $gte $ne	< <= > >= !=
	db.集合名.find({key:null})	匹配null，也可以匹配没有的字段
更新文档	db.集合名.update({条件},{修改的字段})	默认只修改匹配文件的第一行，且会删除本行中的其余字段，只留下修改的字段
	db.集合名.update({条件},{$set:{修改的字段}},false,true)	默认只更新与条件匹配的第一行，加上false，true可以修改全部与条件匹配的行
	db.集合名.update({条件},{$unset:{key:values}})	删除符合条件的字段
	db.集合名.update({条件},{$inc:{字段名:+/-num}})	调价匹配时，字段值自加或者自减
	db.集合名.update({条件},{$push:{数组名:"值"}})	向数组中添加新元素
	db.集合名.update({条件},{$addToSet:{数组:"值"}})	避免重复添加
	db.集合名.update({条件},{$pop:{数组名:数字}})	从数组头/尾部删除一个元素，1代表尾部，也就是数组列最后列一个，-1 代表从头部！
	db.集合名.update({条件},{$pull:{数组名:"值"}})	删除数组指定元素
删除文档	db.集合名.drop()	删除集合的同时删除索引
删除文档	db.集合名.remove({})/({条件})	删除文档的同时不删除索引，删除所有/符合条件的

插入文档演示

> db.t1.save({_id:1,name:"bob"})
WriteResult({ "nMatched" : 0, "nUpserted" : 1, "nModified" : 0, "_id" : 1 })

> db.t1.insert({_id:2,name:"tom"})
WriteResult({ "nInserted" : 1 })

> db.t1.find()
{ "_id" : 1, "name" : "bob" }
{ "_id" : 2, "name" : "tom" }

> db.t1.save({_id:2,name:"lucy"})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

> db.t1.find()
{ "_id" : 1, "name" : "bob" }
{ "_id" : 2, "name" : "lucy" }

 db.t1.insert({_id:2,name:"jack"})
WriteResult({
	"nInserted" : 0,
	"writeError" : { ....}
> db.t1.find()
{ "_id" : 1, "name" : "bob" }
{ "_id" : 2, "name" : "lucy" }

> db.t1.insertMany([{_id:3,name:"rose"},{_id:2,name:"jack"}])
> db.t1.find()
{ "_id" : 1, "name" : "bob" }
{ "_id" : 2, "name" : "lucy" }
{ "_id" : 3, "name" : "rose" }
> db.t1.insertMany([{_id:4,name:"jack"},{_id:5,name:"kate"}])
{ "acknowledged" : true, "insertedIds" : [ 4, 5 ] }
> db.t1.find()
{ "_id" : 1, "name" : "bob" }
{ "_id" : 2, "name" : "lucy" }
{ "_id" : 3, "name" : "rose" }
{ "_id" : 4, "name" : "jack" }
{ "_id" : 5, "name" : "kate" }

查询文档演示

> use userdb
switched to db userdb
> show tables
user
> db.user.find()
{ "_id" : ObjectId("5c21f814af1ef5d24b56fcd5"), "name" : "root", "pass" : "x", "uid" : 0, "gid" : 0, "comment" : "root", "hmdir" : "/root", "shell" : "/bin/bash" }
...
Type "it" for more

> db.user.findOne()
{
	"_id" : ObjectId("5c21f814af1ef5d24b56fcd5"),
	"name" : "root",
	"pass" : "x",
	"uid" : 0,
	"gid" : 0,
	"comment" : "root",
	"hmdir" : "/root",
	"shell" : "/bin/bash"
}

> db.user.find().pretty()                             //加上pretty() 可以竖行显示
{
	"_id" : ObjectId("5c21f814af1ef5d24b56fcd5"),
	"name" : "root",
	"pass" : "x",
	"uid" : 0,
...
	"comment" : "Saslauthd user",
	"hmdir" : "/run/saslauthd",
	"shell" : "/sbin/nologin"
}
Type "it" for more

> db.user.find({},{_id:0,name:1,shell:1})
{ "name" : "root", "shell" : "/bin/bash" }
{ "name" : "bin", "shell" : "/sbin/nologin" }
...
{ "name" : "colord", "shell" : "/sbin/nologin" }
{ "name" : "saslauth", "shell" : "/sbin/nologin" }
Type "it" for more

> db.user.find().count()
41

> db.user.find({shell:"/sbin/nologin"},{_id:0,name:1,shell:1}).limit(3)
{ "name" : "bin", "shell" : "/sbin/nologin" }
{ "name" : "daemon", "shell" : "/sbin/nologin" }
{ "name" : "adm", "shell" : "/sbin/nologin" }
> db.user.find({shell:"/sbin/nologin"},{_id:0,name:1,shell:1}).skip(2)
{ "name" : "adm", "shell" : "/sbin/nologin" }
{ "name" : "lp", "shell" : "/sbin/nologin" }
{ "name" : "mail", "shell" : "/sbin/nologin" }
...
> db.user.find({shell:"/sbin/nologin"},{_id:0,uid:1,shell:1}).sort({uid:1}).limit(10)
{ "uid" : 1, "shell" : "/sbin/nologin" }
{ "uid" : 2, "shell" : "/sbin/nologin" }
{ "uid" : 3, "shell" : "/sbin/nologin" }
{ "uid" : 4, "shell" : "/sbin/nologin" }
{ "uid" : 8, "shell" : "/sbin/nologin" }
{ "uid" : 11, "shell" : "/sbin/nologin" }
{ "uid" : 12, "shell" : "/sbin/nologin" }
{ "uid" : 14, "shell" : "/sbin/nologin" }
{ "uid" : 29, "shell" : "/sbin/nologin" }
{ "uid" : 32, "shell" : "/sbin/nologin" }
> db.user.find({shell:"/sbin/nologin"},{_id:0,uid:1,shell:1}).sort({uid:-1}).limit(10)
{ "uid" : 65534, "shell" : "/sbin/nologin" }
{ "uid" : 999, "shell" : "/sbin/nologin" }
{ "uid" : 998, "shell" : "/sbin/nologin" }
{ "uid" : 997, "shell" : "/sbin/nologin" }
{ "uid" : 996, "shell" : "/sbin/nologin" }
{ "uid" : 995, "shell" : "/sbin/nologin" }
{ "uid" : 994, "shell" : "/sbin/nologin" }
{ "uid" : 993, "shell" : "/sbin/nologin" }
{ "uid" : 992, "shell" : "/sbin/nologin" }
{ "uid" : 192, "shell" : "/sbin/nologin" }

> db.user.find({name:"root",uid:0},{_id:0,name:1,shell:1})
{ "name" : "root", "shell" : "/bin/bash" }
> db.user.find({name:{$in:["root","adm","nobody"]}},{_id:0,name:1,shell:1})
{ "name" : "root", "shell" : "/bin/bash" }
{ "name" : "adm", "shell" : "/sbin/nologin" }
{ "name" : "nobody", "shell" : "/sbin/nologin" }

> db.user.find({shell:{$nin:["/bin/bash","/sbin/nologin"]}},{_id:0,name:1,shell:1})
{ "name" : "sync", "shell" : "/bin/sync" }
{ "name" : "shutdown", "shell" : "/sbin/shutdown" }
{ "name" : "halt", "shell" : "/sbin/halt" }
{ "name" : "mysql", "shell" : "/bin/false" }

> db.user.find({$or:[{name:"root"},{uid:99}]},{_id:0,name:1,uid:1})
{ "name" : "root", "uid" : 0 }
{ "name" : "nobody", "uid" : 99 }


> db.user.find({uid:{$gte:10,$lte:20}},{_id:0,name:1,uid:1})
{ "name" : "operator", "uid" : 11 }
{ "name" : "games", "uid" : 12 }
{ "name" : "ftp", "uid" : 14 }

> db.user.find({name:/^a/},{_id:0,name:1,uid:1})
{ "name" : "adm", "uid" : 3 }
{ "name" : "abrt", "uid" : 173 }
{ "name" : "avahi", "uid" : 70 }

> db.user.find({$or:[{uid:{$lt:10}},{uid:{$gt:70}}]},{_id:0,name:1,uid:1}).sort({uid:1})
{ "name" : "root", "uid" : 0 }
{ "name" : "bin", "uid" : 1 }
{ "name" : "daemon", "uid" : 2 }
{ "name" : "adm", "uid" : 3 }
{ "name" : "lp", "uid" : 4 }
{ "name" : "sync", "uid" : 5 }
{ "name" : "shutdown", "uid" : 6 }
{ "name" : "halt", "uid" : 7 }
{ "name" : "mail", "uid" : 8 }
{ "name" : "tcpdump", "uid" : 72 }
{ "name" : "sshd", "uid" : 74 }
{ "name" : "radvd", "uid" : 75 }
{ "name" : "dbus", "uid" : 81 }
{ "name" : "postfix", "uid" : 89 }
{ "name" : "nobody", "uid" : 99 }
{ "name" : "qemu", "uid" : 107 }
{ "name" : "usbmuxd", "uid" : 113 }
{ "name" : "pulse", "uid" : 171 }
{ "name" : "rtkit", "uid" : 172 }
{ "name" : "abrt", "uid" : 173 }

更新文档

> db.user.find({uid:{$lte:4}},{_id:0})
{ "name" : "root", "pass" : "x", "uid" : 0, "gid" : 0, "comment" : "root", "hmdir" : "/root", "shell" : "/bin/bash" }
...
{ "name" : "lp", "pass" : "x", "uid" : 4, "gid" : 7, "comment" : "lp", "hmdir" : "/var/spool/lpd", "shell" : "/sbin/nologin" }

> db.user.update({uid:{$lte:4}},{pass:666})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.user.find({uid:{$lte:4}},{_id:0})
{ "name" : "bin", "pass" : "x", "uid" : 1, "gid" : 1, "comment" : "bin", "hmdir" : "/bin", "shell" : "/sbin/nologin" }
...
{ "name" : "lp", "pass" : "x", "uid" : 4, "gid" : 7, "comment" : "lp", "hmdir" : "/var/spool/lpd", "shell" : "/sbin/nologin" }

> db.user.find({pass:666})
{ "_id" : ObjectId("5c21f814af1ef5d24b56fcd5"), "pass" : 666 }
> db.user.find({uid:{$lte:4}},{_id:0}).

> db.user.update({name:"bin"},{$set:{pass:"666"}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.user.find({name:"bin"},{_id:0})
{ "name" : "bin", "pass" : "666", "uid" : 1, "gid" : 1, "comment" : "bin", "hmdir" : "/bin", "shell" : "/sbin/nologin" }

> db.user.update({uid:{$lte:4}},{$unset:{pass:"x"}},false,true)
WriteResult({ "nMatched" : 4, "nUpserted" : 0, "nModified" : 3 })
> db.user.find({uid:{$lte:4}},{_id:0})
{ "name" : "bin", "uid" : 1, "gid" : 1, "comment" : "bin", "hmdir" : "/bin", "shell" : "/sbin/nologin" }
{ "name" : "daemon", "uid" : 2, "gid" : 2, "comment" : "daemon", "hmdir" : "/sbin", "shell" : "/sbin/nologin" }
{ "name" : "adm", "uid" : 3, "gid" : 4, "comment" : "adm", "hmdir" : "/var/adm", "shell" : "/sbin/nologin" }
{ "name" : "lp", "uid" : 4, "gid" : 7, "comment" : "lp", "hmdir" : "/var/spool/lpd", "shell" : "/sbin/nologin" }

删除文档

> db.user.remove({name:"bin"})
WriteResult({ "nRemoved" : 1 })
> db.user.find({uid:{$lt:10}},{_id:0,name:1})       //可以发现已经没有bin用户了，bin的uid前面可以到是1
{ "name" : "daemon" }
{ "name" : "adm" }
{ "name" : "lp" }
{ "name" : "sync" }
{ "name" : "shutdown" }
{ "name" : "halt" }
{ "name" : "mail" }

NoSQL之MongoDB副本集和文档管理

插入

文档

查询

文档

更新

文档

猜你喜欢