Use Haproxy + lua agent Mongodb replica set

In general, the use of mongo client, or other languages ​​mongo client drivers to connect mongodb replica when set, only need to specify the name of the replica set, when high availability can be achieved when the target standby switching mongodb primary replica set.

But there are always some special occasion, the network connection to the client and the replica set replica set where is isolated, accessible only through a proxy network replica set replica set where, for example, mongodb replica set to be deployed to k8s, when outside k8s cluster client wants to access a replica set and can only be accessed through a proxy, such as access to the replica set by haproxy.

Then the question is, the client only knows url or ip address proxy, when the replica set standby switching, by the main client instance replica set name is not connected to the replica set, there is provided a proxy mongo here haproxy + lua way replica set can solve this problem.

And instructions on the use of methods and haproxy lua, please refer to the following links:
https://www.arpalert.org/haproxy-lua.html

First, create a simple mongodb replica set (different from the same host port, no user name and password and other security settings):

# 通过以下shell创建一个简易的mongodb副本集
​
# 杀掉已存在的mongo实例
ps -ef | grep mongod | grep -v grep | awk '{print $2}' | xargs kill -9 ;
​
# 清理并创建目录供副本集使用
rm -rf /data/47017/* &&
rm -rf /data/47018/* &&
rm -rf /data/47019/* &&
rm -f rm -rf /data/47017log &&
rm -f rm -rf /data/47018log &&
rm -f rm -rf /data/47019log &&
mkdir -p /data/47017/ &&
mkdir -p /data/47018/ &&
mkdir -p /data/47019/ &&
​
# 获取本机IP地址
localip=`ifconfig -a|grep inet|grep -v 127.0.0.1|grep -v inet6| grep -v 172|awk '{print $2}'|tr -d "addr:"`
echo "ETH0 IP is ${localip}"
​
# 下面几行注释掉的命令是创建mongodb的用户名和密码，暂时不要
#/home/mongodb/bin/mongod --fork --bind_ip 0.0.0.0 --port 47017 -dbpath "/data/47017" -logpath /data/47017log &&
#/home/mongodb/bin/mongo --host ${localip} --port 47017 --eval "db.getSiblingDB('admin').createUser({user: 'aa', pwd: 'aa', roles: [ { role: 'root', db: 'admin' }]})" &&
#ps -ef | grep mongod | grep -v grep | awk '{print $2}' | xargs kill && sleep 5 &&
​
# 如mongodb的主目录是/home/mongodb的话，直接使用如下命令创建副本集即可
/home/mongodb/bin/mongod --replSet myrep --fork --bind_ip 0.0.0.0 --port 47017 -dbpath "/data/47017" -logpath /data/47017log &&
/home/mongodb/bin/mongod --replSet myrep --fork --bind_ip 0.0.0.0 --port 47018 -dbpath "/data/47018" -logpath /data/47018log &&
/home/mongodb/bin/mongod --replSet myrep --fork --bind_ip 0.0.0.0 --port 47019 -dbpath "/data/47019" -logpath /data/47019log &&
/home/mongodb/bin/mongo --host ${localip} --port 47017 --eval "db.getSiblingDB('admin'); rs.initiate({'_id':'myrep','members':[{'_id':0,'host':'${localip}:47017',priority:3},{'_id':1,'host':'${localip}:47018',priority:2},{'_id':2,'host':'${localip}:47019',priority:0,slaveDelay:86400}]})"

You can then proceed to install haproxy and lua-related environment, because we need to probe mongodb replica set of the primary examples of which node is located, it is also necessary to install mongo-c-driver, this process is relatively time-consuming, it can be used directly docker container to simplify this a process reference:
https://hub.docker.com/r/pengtaoman/haproxy2.1.0-lua-mongodriver
https://github.com/pengtaoman/haproxy2.1.0-lua-mongodriver

There dockerfile source on GitHub, can view the environment of the installation process, and an example of the configuration file, herein briefly described as follows:
haproxy.cfg

global
  log 127.0.0.1 local0 debug
  log 127.0.0.1 local1 notice
  stats timeout 30s
  user haproxy
  group haproxy
  # 这里载入我们做成的lua文件
  lua-load /etc/haproxy/conf/mongo-backend.lua
  daemon
​
defaults
  log global
  mode tcp
  option tcplog
  option dontlognull
  timeout connect 5000
  timeout client 50000
  timeout server 50000
# 通过frontend设置的端口访问副本集
frontend mongo_front
  bind *:37017
  use_backend %[lua.backend_select]
# m1 m2 m3是副本集的三个实例，
# 我们将通过lua程序选择使用哪一个实例，从而实现主备切换时的高可用
# 10.211.55.21即为上一步创建简易副本集的主机的ip
backend m1
  balance roundrobin
  log 127.0.0.1 local0
  server mongorep0 10.211.55.21:47017 check
​
backend m2
  balance roundrobin
  log 127.0.0.1 local0
  server mongorep1 10.211.55.21:47018 check
​
backend m3
  balance roundrobin
  log 127.0.0.1 local0
  server mongorep2 10.211.55.21:47019 check
# 打开haproxy自带监控服务
listen admin_stats
  mode http
  bind 0.0.0.0:8888
  stats uri /stats
  stats realm Global\ statistics

mongo-backend.lua

# 向haproxy注册一个fetch
core.register_fetches("backend_select", function(txn)
    for k,v in pairs(core.backends) do
      local servs = v.servers
      for sk,sv in pairs(servs) do
        core.Debug(sk)
        local svAddr = sv.get_addr(sv)
        local isMaster = checkMongo(svAddr)
        if (isMaster)
        then
      core.Debug("###### Now primary instance is:"..svAddr)
          return k
        end
      end
    end
  end)
​
  checkMongo = function(mongosvr)
    core.Debug("###### checkMongo mongo address ::"..mongosvr)
    local mongo = require 'mongo'
    local client = mongo.Client('mongodb://'..mongosvr)
    local isp = client:command('admin','{ "isMaster": "1" }')
    local bson = mongo.BSON{}
    pcall(function()
        bson:concat(isp)
      end)
    local ispri=bson:find('ismaster')
    if unexpected_condition then error() end
    return ispri

The haproxy.cfg and placed as mongo-backend.lua / home / cfg directory specified under their own, the vessel can be started as follows:

docker run -d -p 37017:37017 -v /home/cfg:/etc/haproxy/conf pengtaoman/haproxy2.1.0-lua-mongodriver:0.0.1

By mongo client to access the proxy:

docker run -it --rm mongo:4.0.14 mongo --host 10.211.55.2 --port 37017

Use docker logs can be printed in our view the log file in lua:

docker logs a1ced56a7444
[NOTICE] 029/131117 (1) : New worker #1 (7) forked
[debug] 029/131228 (7) : mongorep1
[debug] 029/131228 (7) : ###### checkMongo mongo address ::10.211.55.21:47018
[debug] 029/131228 (7) : mongorep0
[debug] 029/131228 (7) : ###### checkMongo mongo address ::10.211.55.21:47017
[debug] 029/131228 (7) : ###### Now primary instance is:10.211.55.21:47017

You can see the current primary instance port is 47017, we have to see what happens after the cessation of examples 47017:

mongo --port 47017
​
use admin
db.shutdownServer();

After using the mongo client connection broker again to see docker logs as follows:

docker logs a1ced56a7444
[NOTICE] 029/131117 (1) : New worker #1 (7) forked
[debug] 029/131228 (7) : mongorep1
[debug] 029/131228 (7) : ###### checkMongo mongo address ::10.211.55.21:47018
[debug] 029/131228 (7) : mongorep0
[debug] 029/131228 (7) : ###### checkMongo mongo address ::10.211.55.21:47017
[debug] 029/131228 (7) : ###### Now primary instance is:10.211.55.21:47017
[WARNING] 029/132201 (7) : Server m1/mongorep0 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 101ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 029/132201 (7) : backend 'm1' has no server available!
[debug] 029/132224 (7) : mongorep1
[debug] 029/132224 (7) : ###### checkMongo mongo address ::10.211.55.21:47018
[debug] 029/132224 (7) : ###### Now primary instance is:10.211.55.21:47018

primary replica set node has switched to port 47018, and the configuration was successful.