In the previous articles, I shared about MongoDB indexes and common commands. Today, while sharing the actual MongoDB with the team, I will continue to talk about MongoDB's replica set configuration and deployment.
Introduction to replica set
What is a replica set? Replica set is MongoDB's replication system, used to synchronize data to various servers.
In the previous article, I demonstrated basically a single-node mongod server. The single-node is fine in the test environment, but the online production environment is not suitable, because if the single-node fails, it will cause the application to crash.
So at this time, we can use MongoDB's replication function to deploy with multiple nodes. One of the server nodes is down, and the other nodes are running, failover, so that the application will not be affected, thereby making it more highly available.
Speaking of failover, MongoDB initially supported a mode called "master-slave" (master-slave) in which MongoDB does not automatically failover. This mode is now deprecated, so I won't introduce it here.
Since it is very convenient to add or delete nodes in the replica set, it is recommended that you start it as a replica set even if you are a single node at the beginning to facilitate subsequent node additions.
member introduction
The members of the replica set mainly introduce the arbiter arbiter and the delayed backup node.
Speaking of arbitrators, it is necessary to say that the process of electing the master node among the members of a replica set satisfies the "majority" principle, that is, n/2 + 1. The role of the arbitrator is to "participate in the election" without saving data. To solve the problem at the beginning when the amount of our application was very small, we had no resources and did not want to save a copy of three points or more of data.
rs.add({"_id": 3, "host": "server-3:27017", "arbiterOnly": true})
Note here:
Only one arbitrator can be used
Odd nodes do not need an arbiter
Use an odd number of data nodes as much as possible, without using arbiters
After talking about the arbitrators, let’s talk about “delayed backup nodes”. When sharing internally with the team, the operation and maintenance god of our company specifically emphasized this. This is a historical story.
The main function of the delayed backup node is to prevent someone from accidentally deleting the main database, or the application has a serious bug that causes all the data to be corrupted. To avoid this type of problem, set up a delayed backup node. It is strongly recommended to configure online, this is a history of blood and tears.
rs.add( { _id:4, host: "server-4:27020", priority: 0, hidden:true, slaveDelay:7200, votes:0, buildIndexes:true, arbiterOnly:false } )
In fact, this can be regarded as one of the backup methods, a guarantee plan.
Replica set initialization
After the replica set members are started, I will put the configuration here in the last section. Here we directly talk about the initialization operation after the replica set is started.
There are four main steps for member initialization:
Members create their own identifiers in local.me, delete local existing data, and synchronize data
Clone all the recorded data of the synchronization source to the local, this step is the most time-consuming
Record the operation in the first oplog synchronization.
Create index, synchronize all operations during index creation
Through the above 4 steps, the initialization of the replica set is completed.
Replica set backup
A simple summary of the backup method has the following four methods:
1. File system snapshot: The file system snapshot needs the support of the file system itself. Mongod opens the diary system. I will talk about the configuration later.
2. Copy the data directory: copy all the files in the data directory. During the backup, we need to prevent the data files from being changed, otherwise they will be unavailable.
Ensure that the data remains unchanged, you can pass
db.fsyncLock() locks all databases, and subsequent operations will wait in the queue after running.
cp -R /data/db/* /backup/
db.fsyncUnlock() # unlock the database to be able to write again
3. mongodump: The backup and recovery speed is slow, not recommended
Fourth, delay the backup node: In principle, it is not a backup strategy, but it is more important to delay and prevent data misoperation.
Replica deployment
Finally, we have finally talked about the replica set configuration, here we configure the mongod.conf configuration, replaSetName is set to test_rs name, the configuration is as follows:
storage:
dbPath: /test/mongodb/rs1 # 配置路径
journal:
enabled: true # 开启日记系统
systemLog:
destination: file
logAppend: true
path: /test/log/mongodb/rs1.log # 日志
processManagement:
pidFilePath: /var/run/mongodb/rs1.pid
net:
port: 27018
bindIp: 1.2.3.4 # 绑定固定IP
replication:
replSetName: "test_rs" # 选定配置名
security:
authorization: enabled
keyFile: /test/mongodb/key/test_rs.key
In addition to the above configuration, we can configure the init.d startup script. You can go to github and search for it. There are many startup scripts sudo service mongodb.rs1 start that start processing and add to the replica set through multiple nodes.
Since mongo has no password by default, for database configuration, in addition to configuring account passwords, if we are on a Tencent Cloud or Alibaba Cloud server, we need to configure a security group to allow only a few fixed machine IPs and fixed port access in the intranet.
Configuration optimization
In addition to the MongoDB configuration is started, we also need to make some configuration adjustments to the server, mainly as follows:
Prohibit overallocation of memory: overcommit_memory=2
Disable HugePage:
Modify file descriptor> 20000 or unlimited
Turn off regular tasks, such as the automatic update of software packages, which consumes CPU and memory resources and causes abnormal service jitter (similar to Redis asynchronous task hgetall)
Among them, vercommit_memory is set to 1, to satisfy all memory allocation requests (redis deployment), 2 to allocate virtual space not to exceed the sum of swap and a small amount of overallocation, and 0 to let the kernel guess the size of overallocation.
Prohibit large memory: If you can't store all of it in the memory, you can use large memory regardless of the memory capacity, but if you can't store all of it, then large blocks of data will cause more IO, and "dirty data" may land on the hard disk From KB to MB.
So far, our service deployment is over. Due to personal experience, some omissions or even errors are inevitable. Please leave a message to point out, thank you very much.