Outline
Recent activity mongodb linear rise of the Internet, and we also used the company as a production mongoDB 3.6 important database, we project team to monitor op.log log mongodb of previous knowledge in this sort of learning, learning to use later as a backup.
Definitions and characteristics
MongoDB is a web application designed for Internet infrastructure and database management systems. And is the type of NoSQL database.
1), mongoDB proposed is a document, a collection of concept
It uses the BSON (class JSON) as model data structure, which structure is object-oriented, rather than two-dimensional table, for example:
{
"_id" : ObjectId("5cf60b44e574a35d02c1ca80"),
"host" : "rhel67129230"
}
SQL type of database is normalized, the integrity and uniqueness can be constrained by the primary key or foreign key data guarantee, the database SQL type commonly used in high data integrity system. MongoDB in this regard is not as good as the type of SQL database, MongoDB and no fixed schema, because less of these characteristics mongoDB, database storage structure is more flexible, faster storage.
2), easily scalable, automated failover
Easy telescopic means provides fragmentation capability can be fragmented data set stored pressure data to multiple servers sharing. Automatic failover concept replica set, MongoDB master node can detect whether or not alive, when deactivated automatically lift upgrade from the master node to node, failover capability reached.
3), real-time query capabilities
MongoDB retain the ability to query relational databases instantly, retains the index function (bottom layer is based on B TREE), it draws on the advantages of relational databases.
4), the speed and durability
MongoDB implement a write driver semantic fire and forget, that is transferred through the write drivers, can be obtained immediately returns a successful result (even error), even faster writing speed so, of course, there will be some non security, totally dependent on the network.
MongoDB provides a conceptual Journaling log is actually similar to the mysql bin-log log when the time will first need to insert records written to the log inside, in the completion of the actual data manipulation, so if she can fix the problem.
Database syntax
Find syntax
db.users.find() db.users.count()
Syntax update
db.users.update ({username: "smith"}, {$ the SET: {Country: "Canada"}})
// the user named smith user countries into Canada
db.users.update ({username: "smith"}, {$ the unset: Country {:}}. 1)
// the user of the user name field to remove smith state
db.users.update ({username: "jones" }, {$ set: { Favorites: {Movies: [ "casablance", "Rocky"]}}})
// where mainly multivalued modifications, additions plurality of field values favorties
db.users.update ({ "favorites.movies": " casablance "}, {$ addToSet: {favorites.movies:" at The maltese "}}, false, to true)
// number of updates
Delete grammar
db.foo.remove () // delete all data
db.foo.remove ({favorties.cities: "cheyene" }) // delete according to the conditions
db.drop () // delete the entire set
Index-related syntax
db.numbers.ensureIndex ({NUM:. 1})
// create an ascending index
db.numbers.getIndexes ()
// Get all indexes
The basic syntax Management
dbs Show // query the database for all Show the Collections // display all tables db.stats () // display the database state information db.numbers.stats () // display status information collection table db.shutdownServer () // Stop the database db. help () // get database operations command db.foo.help () // get table operation command tab key // automatically help us fill all orders
SCHEMA design principles
In regard to the schema design should pay attention to some issues such as:
-
You can not create useless index
-
You can not store different types in the same field
-
Can not multi-class entity in the collection can not create a large volume, deep nested documents
-
You can not create too much of a collection, collection, indexing, database name space is limited
-
You can not create a collection can not be fragmented
Watch some of the details inside mongodb
1, the concept of database
And the database is a logical set of packets, mongodb not create a database, and then only when inserted into the collection, database began to build. After creating the database is assigned a set of data in the disk asking about the price, all collections, indexes, and other metadata database will be stored in these files, disk access to the database state can use the command: db.stats ()
2, concerned about the concept of the collection
Collection is similar structural or conceptual container document name set may comprise numbers, letters or symbols, it must start with a letter or number. Defined collection names can not exceed 128 characters.
3, focus on the concept of the document
Followed by key, which is in mongodb utf-8 type. Digital type comprising double, int, lon. UTC format types are date, it is slower than in Beijing inside mongodb 8 hours. The size of the entire document will be limited to less than 16m, because it can limit the type of data to create unsightly, but also can improve the performance of small documents.
4, the concept of index
The index contains: a single indexing, a composite index, a unique index, sparse index.
If the data set is large when building the index will take a lot of time, and the program will affect the process can) to view the index build times by db.currentOp (.
When the implementation of large-scale deletion, use db.values.reIndex () index compression and reconstruction.
5, identify slow queries
Check out the slow query log
grep -E '([0-9]) + ms' mongod.log // Use grep command to identify the command information db.setProfillingLevel (2) // plane solution is used, the read and write to the log record each db.setProfillingLevel (1) // record only slowly (100ms) operation
Slow query analysis
db.values.find({}).sort({close:-1}).limit(1).explain()
-
scanOrder field indicates the index is not used
-
When no index cursor when using a BasicCursor, used when the index is used BtreeCursor
-
n represents the need to return a result set
-
nscanned expressed the need to traverse the document indicates the index number indexBounds border
MONGODB副本集
实际上MongoDB对副本集的操作跟mysql主从操作是差不多的，先看一下mysql的主从数据流动过程
主binlog -> 从relay.log -> 从bin.log -> 从数据库
而MongoDB主要依赖的日志文件是oplog
主oplog -> 从oplog
写操作先被记录下来，添加到主节点的oplog里。与此同时，所有从结点复制oplog。首先，查看自己oplog里最后一条的时间戳；其次，查询主节点oplog里所有大于此时间戳的条目;最后，把那些条目添加到自己的oplog里并应用到自己的库里。从节点使用长轮询立即应用来自主结点oplog的新条目。
当遇到以下情况，从节点会停止复制
-
如果从节点在主节点的oplog里找不到它所同步的点，那么会永久停止复制
-
一旦某个从节点没能 在主节点的oplog里找到它已经同步的点，就无法再保证这个从结点的完美副本
local数据库保存了所有副本集元素据和oplog日志
-
replset.minvalid 包含指定副本集成员的初始化同步信息
-
system.replset 保存在副本集配置文档
-
system.indexes 标准索引说明容器
-
me slaves 主要用于写关注
可以使用以下命令查看复制情况
db.oplog.rs.findOne()
-
ts 保存了该条目的BSON时间戳
-
t 是从纪元开始的描述
-
i是计数器
-
op 表示操作码
-
ns 标明了有关的命名空间
分片
分片是指将数据库进行拆分，将其分散到不同的机器上的过程。将数据分散到不同的服务器上，不需要功能强大的服务器就可以存储更多的数据和处理更大的负载。基本思想就是将集合切成小块，这些快分散到若干的片里面，每个片只负责数据的一部分，最后通过一个均衡器来对各个分片进行均衡，主要是通过一个名为mongos的路由进程进行操作。大部分分片的使用场景都是解决磁盘空间问题，对于写入有可能变差，查询尽量避免分片查询，使用分片的动机是：
1，机器的磁盘不够用，使用分片解决磁盘问题。
2，单个mongod已经不能满足写数据的性能要求，通过分片可以让写的压力分散到各个机器上面。
3，想把大量数据放到内存中，提升性能。
分片的组件
分片：每个分片都是一个副本集。
分片的核心操作
分片的一个集合：分片是根据一个属性的范围进行划分的，mongodb使用所谓的分片键让每个文档在这些范围里面找到自己的位置。
快：是位于一个分片中一段连续的分片建范围，可以理解为若干个快组成分片，分片组成mongodb的所有数据。
拆分和迁移
块的拆分：初始化只有一块，快达到最大尺寸64m或者100000个文档就会触发块的拆分。把原来的范围一分为二，这样就有了2块，每个块都有相同数量的文档。
迁移：当分片的数据大小不一的时候就会产生迁移的操作，分片集群是通过分片中移动块来实现均衡，
sh.help() //查看分片相关帮助
sh.addShard() //添加分片
db,getSiblingDB("config").shards.find() //查看分片列表
sh.status() //分片详情
sh.enableSharding("cloud-docs") //开启一个数据库上的分片
db.getSiblingDB("config").databases,find() //查看数据库列表
sh.shardCollection("cloud-docs.spreadsheets",{username:1,_id:1}) //使用一个分片键定义一个分片集合spreadsheets，根据用户名进行切分
sh.getSiiblingDB("config").collections.findOne() //查看集合列表
db.chunks.count() //查看块的个数
db.chunks.findOne() //查看块的信息
db.changelog.count(}what:"split"|) //查看块切分日志
db.changelog.find({what:"moveChunk.commit"}).count() //查看日志迁移记录