MongoDB operation and maintenance optimization series (1)

First, set the log level

method 1:

db.adminCommand({"setParameter":1,"logLevel":0});

 Method 2 (startup only):

At startup, this can be done by appending a higher number of "v" to the parameter (i.e. -v, -vv, -vvv, -vvvv or -vvvvv)

    The smaller the number or v, the higher the log priority. Such as: 0 - necessary content, 5 - almost all operations.

 

2. Settings: For operations that query time-consuming Nms, record logs

db.setProfilingLevel(1,N);

 

3. Set up log file segmentation

method 1:

# kill -SIGUSR1 <mongod process id> //Send custom process to mongo process
# find /data/mongodb_data/log/mongodb.log.* -mtime +7 -delete //Only keep the logs of the last 7 days, delete the rest

 Method 2:

use admin;
db.runCommand({logRotate:1});

 WINDOWS system can only use this method.

Method 3: Syslog Log Rotation

# vi /etc/logrotate.d/mongodb
/opt/mongodb/log/mongodb.log {
 daily     
 rotate 7
 compress    
 dateext
 missingok     
 notifempty    
 sharedscripts     
 copytruncate     
 postrotate        
 /bin/kill -SIGUSR1 `cat /data/mongodb_data/mongod.lock 2> /dev/null` 2>/dev/null || true     
endscript }

 

Fourth, set the oplog

Query the size of the oplog and the duration of the saved operation records

repltest: PRIMARY> db.printReplicationInfo ()
configured oplog size:   1024MB
log length start to end: 3705secs (1.03hrs)
oplog first event time:  Thu Oct 10 2013 11:13:29 GMT+0800 (CST)
oplog last event time:   Thu Oct 10 2013 12:15:14 GMT+0800 (CST)
now:                     Fri Oct 11 2013 16:33:42 GMT+0800 (CST)

 

Query the data source list of the slave node, which has data lag time

repltest:PRIMARY> db.printSlaveReplicationInfo()
source:   192.168.1.101:37017
syncedTo: Fri Oct 11 2013 16:38:16 GMT+0800 (CST)
= 1 secs ago (0hrs)
source:   192.168.1.100:37017
no replication info, yet.  State: ARBITER

 

Modify the size of the oplog

method 1:

The oplog exists internally as a capped collection, so you cannot modify its size in the course of normal operations. Another: to change the oplog size, you need to perform maintenance mode on each node. (Official recommendation)
Steps:

1: Restart an instance in stand-alone mode,

Usually before shutting down the server, use rs.stepDown() to force the primary to become the secondary

2: recreate a new size,

The oplog which contains the entry entry for the old oplgo

3: Restart mongod as a member of the replica set

Steps:

1>: Restart a Secondary in Standalone Mode on a Different Port

Shut down the mongod instance:
repset:PRIMARY> use admin
repset:PRIMARY> db.shutdownServer()
Restart the mongod instance in stand-alone mode, modify the port, and do not add the --replSet parameter
#vim /etc/mongo.conf
  dbpath=/var/lib/mongodb
  logpath=/var/log/mongodb/mongo.log
  pidfilepath=/var/run/mongo.pid
  directoryperdb=true
  logappend=true
  #replSet=repset
  bind_ip=192.168.1.100,127.0.0.1
  port=37017
  oplogSize=2000
  fork=true
#mongod -f /etc/mongo.conf
backup oplog
#mongodump --db local --collection 'oplog.rs' --port 37017

 

2>: Recreate the Oplog with a New Size and a Seed Entry

Save the latest time point of the oplog
> use local
> db.temp.save( db.oplog.rs.find( { }, { ts: 1, h: 1 } ).sort( {$natural : -1} ).limit(1).next() )
> db.temp.find()
delete old oplogs
> db.oplog.rs.drop()

 

3> :Create a New Oplog

Create a new Oplog with a size of 2G
> db.runCommand( { create: "oplog.rs", capped: true, size: (2 * 1024 * 1024 * 1024) } )
Insert the record of the time point of the old oplog saved earlier
> db.oplog.rs.save( db.temp.findOne() )
> db.oplog.rs.find()

 

4>:Restart the Member：

Shut down the stand-alone instance:
> use admin
> db.shutdownServer()
Modify back to config # vim /etc/mongo.conf
  dbpath=/var/lib/mongodb
  logpath=/var/log/mongodb/mongo.log
  pidfilepath=/var/run/mongo.pid
  directoryperdb=true
  logappend=true
  replSet=repset
  bind_ip=192.168.1.100,127.0.0.1
  port=37017
  oplogSize=2000
  fork=true
start mongod
#mongod -f /etc/mongo.conf
Repeat the above steps for all nodes that need to be changed.

 

Method 2:

Steps:
1: Stop all replca set nodes.
2: Delete the files in the local library from the master node, and delete all files in the data directory from the node.
3: Modify all node configuration files.
4: Restart all nodes.
5: Reconfigure replca set, the slave node will resynchronize all data (initial sync).
ps: The advantage of this method is simple, but the service needs to be stopped, and if the amount of data is large, the cost of initial synchronization is high

1>: Close the mongod instance (all nodes)

> use admin
> db.shutdownServer()

 

2>: delete all files under the local database (PRIMARY node)

#rm -rf /var/lib/mongodb/local/*
  Delete the mongo data directory (operate on other nodes, do not delete the wrong one, it is recommended to mv all rm operations first, and then delete when there is no problem) # rm -rf /var/lib/mongodb/*

 

3> Modify all node configuration files (oplogsize)

#vim /etc/mongo.conf
  dbpath=/var/lib/mongodb
  logpath=/var/log/mongodb/mongo.log
  pidfilepath=/var/run/mongo.pid
  directoryperdb=true
  logappend=true
  replSet=repset
  bind_ip=192.168.1.100,127.0.0.1
  port=37017
  oplogSize=2000
  fork=true

 

4> Restart all nodes mongod

> mongod -f /etc/mongo.conf

5. Inspection script

#!/bin/bash
pro = `ps -ef | grep mongod | grep -v grep | awk '{print $ 2}'`
echo `ps aux|awk  'BEGIN {cpu=0;mem=0;apc=0;apm=0} {if("USER"!=$1){cpu+=$3;mem+=$4};if('$pro'==$2){apc+=$3;apm+=$4}}END{print ""cpu/100 "\t"mem/100 "\t" apc/100 "\t" apm/100}'`
df -h