Cluster chat server based on muduo

Enterprise 2023-04-09 10:15:27 views: null

cluster chat server

This project is a cluster chat server based on the muduo network library, which mainly realizes login, logout, registration, adding friends, one-on-one chat, group creation, group chat and offline message reception and storage

Project Description

  • Follow the muduo design idea to realize the network library, as the core module of the project network, improve the concurrent network IO, decouple the network and business modules
  • Use json to serialize and deserialize messages as a private communication protocol
  • Configure nginx load balancing based on TCP to realize the cluster chat server function and improve the back-end concurrency capability
  • Redis-based publish-subscribe server middleware to achieve cross-server message communication
  • Use MySQL as the landing storage of project data

Project explanation

database table design

User table
field name Field Type field description constraint
id INT user id PRIMARY KEY、AUTO_INCREMENT
name VARCHAR(50) username NOT NULL、UNIQUE
password VARCHAR(50) user password NOT NULL
state ENUM(‘online’,'offline) current login status DEFAULT ‘offline’
Friend table
field name Field Type field description constraint
userid INT user id NOT NULL, joint primary key
friendif INT friend id NOT NULL, joint primary key
AllGroup table
field name Field Type field description constraint
id INT group id PRIMARY KEY、AUTO_INCREMENT
groupname VARCHAR(50) group name NOT NULL、UNIQUE
groupdesc VARCHAR(200) group introduction DEFAULT ‘’
GroupUser table
field name Field Type field description constraint
groupid INT group id NOT NULL, joint primary key
userid INT user id NOT NULL, joint primary key
grouprole ENUM(‘creator’,‘normal’) role in the group DEFAULT ‘normal’
OfflineMessage table
field name Field Type field description constraint
userid INT user id NOT NULL
message VARCHAR(500) Offline message (store json string) NOT NULL

Network module design

Follow the muduo design idea to realize the network library as the core module of the project network

The thread model of muduo is one loop per thread + threadPool: a thread corresponds to an event loop (EventLoop), and also corresponds to a Reactor model. EventLoop is responsible for the dispatch of IO and timer events

muduo is a master-slave Reactor model, which consists of mainReactor and subReactor. mainReactor receives new connections through Acceptor, and then distributes new connections to subReactor for connection maintenance.
insert image description here

How to decouple network modules and business modules

Assign an id to each request type, save the request id and the corresponding processing function using unordered_map, and then you can get the corresponding processing function according to the id

// 注册消息以及对应的handler回调操作
ChatService::ChatService() {
    msgHandlerMap_.insert({LOGIN_MSG, std::bind(&ChatService::login,this,_1,_2,_3)});
    msgHandlerMap_.insert({REG_MSG,std::bind(&ChatService::reg,this,_1,_2,_3)});
    msgHandlerMap_.insert({ONE_CHAT_MSG,std::bind(&ChatService::oneChat,this,_1,_2,_3)});
    msgHandlerMap_.insert({ADD_FRIEND_MSG,std::bind(&ChatService::addFriend,this,_1,_2,_3)});
    msgHandlerMap_.insert({CREATE_GROUP_MSG,std::bind(&ChatService::createGroup,this,_1,_2,_3)});
    msgHandlerMap_.insert({ADD_GROUP_MSG,std::bind(&ChatService::addGroup,this,_1,_2,_3)});
    msgHandlerMap_.insert({GROUP_CHAT_MSG,std::bind(&ChatService::groupChat,this,_1,_2,_3)});
    msgHandlerMap_.insert({LOGINOUT_MSG,std::bind(&ChatService::loginout,this,_1,_2,_3)});

    // 初始化redis
    if (redis_.connect()) {
        // 设置上报消息回调
        redis_.initNotifyHandler(std::bind(&ChatService::handleRedisSubscribeMessage,this,_1,_2));
    }
}

When a client request arrives, muduo will execute the callback function when the message event arrives. In this callback function, it will read the msgId contained in the request content, and obtain the corresponding processing function according to the msgId, so as to realize the network and business modules. decoupling

void ChatServer::onConnection(const TcpConnectionPtr &conn) {
    // 客户端连接断开
    if (!conn->connected()) {
        // 客户端异常退出处理
        ChatService::instance()->clientCloseException(conn);
        conn->shutdown();
    }
}

Business module design

registration module

Receive data from the network module and locate the registration module according to msgId

According to the passed json object to obtain the user name and user password to generate a User object, call the model layer method to insert the User into the database

If the insertion is successful, the registration is successful, the user Id is obtained and returned to the user, otherwise the insertion fails

void ChatService::reg(const TcpConnectionPtr &conn, json &js, Timestamp time) {
    string name = js["name"];
    string pwd = js["password"];

    User user;
    user.setName(name);
    user.setPwd(pwd);
    bool res = userModel_.insert(user);
    if (res) {
        // 注册成功
        json response;
        response["msgid"] = REG_MSG_ACK;
        response["errno"] = 0;
        response["id"] = user.getId();
        conn->send(response.dump());
    } else {
        // 注册失败
        json response;
        response["msgid"] = REG_MSG_ACK;
        response["errno"] = 1;
        conn->send(response.dump());
    }
}
login module

Query whether the user information matches in the database according to the user ID and password obtained from the json object

If it does not match, tell the user that the login failed, and if it matches, determine whether the user has already logged in

If you have already logged in, return an error message; otherwise, if the login is successful, insert the user id and connection into a hash key-value pair table on the current server, and subscribe to Redis for messages with channel id; finally, modify the user's login status in the database And return relevant information to the user (friend list, group list, offline messages, etc.)

void ChatService::login(const TcpConnectionPtr &conn, json &js, Timestamp time) {
    int id = js["id"].get<int>();
    string pwd = js["password"];
    User user = userModel_.query(id);
    if (user.getId() != -1 && user.getPwd() == pwd) {
        if (user.getState() == "online") {
            // 用户已经登录 不允许重复登录
            json response;
            response["msgid"] = LOGIN_MSG_ACK;
            response["errno"] = 2;
            response["errmsg"] = "this account is using, input another!";
            conn->send(response.dump());
        } else {
            // 登录成功

            // 记录用户连接信息
            {
                lock_guard<mutex> lock(connMutex_);
                userConnMap_.insert({id,conn});
            }

            // 向redis订阅
            redis_.subscribe(id);

            // 更新用户状态信息
            user.setState("online");
            userModel_.updateState(user);

            json response;
            response["msgid"] = LOGIN_MSG_ACK;
            response["errno"] = 0;
            response["id"] = user.getId();
            response["name"] = user.getName();

            // 查询用户是否有离线消息
            vector<string> msgs = offlineMsgModel_.query(id);
            if (!msgs.empty()) {
                response["offlinemsg"] = msgs;
                // 将用户所有离线消息删除
                offlineMsgModel_.remove(id);
            }

            // 查询该用户的好友信息
            vector<User> friendVec = friendModel_.query(id);
            if (!friendVec.empty()) {
                vector<string> friends;
                for (User &f : friendVec) {
                    json js;
                    js["id"] = f.getId();
                    js["name"] = f.getName();
                    js["state"] = f.getState();
                    friends.push_back(js.dump());
                }
                response["friends"] = friends;
            }

            // 查询用户的群组信息
            vector<Group> groupuserVec = groupModel_.queryGroups(id);
            if (!groupuserVec.empty()) {
                vector<string> groupV;
                for (Group &group : groupuserVec) {
                    json grpjson;
                    grpjson["id"] = group.getId();
                    grpjson["groupname"] = group.getName();
                    grpjson["groupdesc"] = group.getDesc();
                    vector<string> userV;
                    for (GroupUser &user : group.getUsers()) {
                        json js;
                        js["id"] = user.getId();
                        js["name"] = user.getName();
                        js["state"] = user.getState();
                        js["role"] = user.getRole();
                        userV.push_back(js.dump());
                    }
                    grpjson["users"] = userV;
                    groupV.push_back(grpjson.dump());
                }
                response["groups"] = groupV;
            }
            conn->send(response.dump());
        }
    } else {
        // 登录失败
        json response;
        response["msgid"] = LOGIN_MSG_ACK;
        response["errno"] = 1;
        response["errmsg"] = "id or password is invalid!";
        conn->send(response.dump());
    }
}
Client Logout Module

Find the corresponding connection in the current server's id and connection collection according to the user id, delete it from the collection, cancel the user's message subscription, and set the user's status to offline

void ChatService::loginout(const TcpConnectionPtr &conn, json &js, Timestamp time) {
    int userid = js["id"].get<int>();
    {
        lock_guard<mutex> lock(connMutex_);
        auto it = userConnMap_.find(userid);
        if (it != userConnMap_.end()) {
            userConnMap_.erase(it);
        }
    }

    // 向redis取消订阅
    redis_.unsubscribe(userid);

    // 更新用户状态信息
    User user(userid,"","","offline");
    userModel_.updateState(user);
}
The client exits the module abnormally

If the connection callback function of muduo detects that the client connection is disconnected, it will execute the corresponding abnormal exit processing and release the corresponding connection resources.

void ChatServer::onConnection(const TcpConnectionPtr &conn) {
    // 客户端连接断开
    if (!conn->connected()) {
        // 客户端异常退出处理
        ChatService::instance()->clientCloseException(conn);
        conn->shutdown();
    }
}

According to the connection, find the user id corresponding to the connection in the current server's id and connection collection, delete it from the map, cancel the user's message subscription, and set the user's status to offline

void ChatService::clientCloseException(const TcpConnectionPtr &conn) {
    User user;
    // 找到客户的对应id
    {
        lock_guard<mutex> lock(connMutex_);
        for (auto it = userConnMap_.begin(); it != userConnMap_.end(); it++) {
            if (it->second == conn) {
                user.setId(it->first);
                // 从map删除用户连接信息
                userConnMap_.erase(it);
                break;
            }
        }
    }

    redis_.unsubscribe(user.getId());

    // 更新用户的状态信息
    if (user.getId() != -1) {
        user.setState("offline");
        userModel_.updateState(user);
    }
}
The server exits the module abnormally

Abnormal exit is generally ctrl+c, by capturing the signal and registering the corresponding signal processing callback function to execute the business of server abnormal exit

Set the status of all online clients to offline

// main.cpp
// 处理服务器ctrl+c结束后重置user的状态信息
void restHandler(int) {
    ChatService::instance()->reset();
    exit(0);
}
int main() {
    signal(SIGINT,restHandler);
}

// ChatService.cpp
void ChatService::reset() {
    // 将onlind状态的用户置为offline
    userModel_.resetState();
}
One-on-one chat module

First check whether the user exists in the id link set of the current server according to the receiver id, and if so, actively push the message

If it does not exist, check whether it is online in the database. If it is online, it means logging in on other servers, and pushes it to the channel subscribed by the user through the distributed subscription method of Redis

If not online, store the offline message in the database and push the offline message to the user when the user goes online next time

void ChatService::oneChat(const TcpConnectionPtr &conn, json &js, Timestamp time) {
    int toId = js["toid"].get<int>();

    // 在同一台服务器上登录
    {
        lock_guard<mutex> lock(connMutex_);
        auto it = userConnMap_.find(toId);
        if (it != userConnMap_.end()) {
            // 在线 服务器主动推送消息给用户
            it->second->send(js.dump());
            return;
        }
    }

    // 在不同服务器上登录
    User user = userModel_.query(toId);
    if (user.getState() == "online") {
        redis_.publish(toId,js.dump());
        return;
    }

    // 存储离线消息
    offlineMsgModel_.insert(toId,js.dump());
}
add friend module

Obtain the user id and the friend id to be added, query whether the friend id exists, and add the corresponding record in the database if it exists

void ChatService::addFriend(const TcpConnectionPtr &conn, json &js, Timestamp time) {
    int userid = js["id"].get<int>();
    int friendid = js["friendid"].get<int>();

    // 存储好友信息
    friendModel_.insert(userid,friendid);
}
Create group module

Get the group name and group introduction according to the passed json object to generate a Group object, call the model layer method to insert the Group into the database

If the insertion is successful, the registration is successful, the group ID is obtained and returned to the user, otherwise the insertion fails

After the insertion is successful, store the relationship between the user and the group

void ChatService::createGroup(const TcpConnectionPtr &conn, json &js, Timestamp time) {
    int userid = js["id"].get<int>();
    string name = js["groupname"];
    string desc = js["groupdesc"];

    Group group(-1,name,desc);
    if (groupModel_.createGroup(group)) {
        groupModel_.addGroup(userid,group.getId(),"creator");
    }
}
Add group business

Get the user id and the group id to be added, query whether the group id exists, and add the corresponding record in the database if it exists

void ChatService::addGroup(const TcpConnectionPtr &conn, json &js, Timestamp time) {
    int userid = js["id"].get<int>();
    int groupid = js["groupid"].get<int>();
    groupModel_.addGroup(userid,groupid,"normal");
}
group chat service

Get the group members according to the group id, and then send the message to all group members, the logic is basically the same as one-to-one chat

void ChatService::groupChat(const TcpConnectionPtr &conn, json &js, Timestamp time) {
    int userid = js["id"].get<int>();
    int groupid = js["groupid"].get<int>();

    vector<int> useridVec = groupModel_.queryGroupUsers(userid,groupid);
    lock_guard<mutex> lock(connMutex_);
    for (int id : useridVec) {
        auto it = userConnMap_.find(id);
        if (it != userConnMap_.end()) {
            it->second->send(js.dump());
        } else {
            // 在其他服务器上登录
            User user = userModel_.query(id);
            if (user.getState() == "online") {
                redis_.publish(id,js.dump());
            } else {
                offlineMsgModel_.insert(id,js.dump());
            }
        }
    }
}

cluster implementation designinsert image description here

Use Nginx load balancing module

When the number of requests is too large, a single server will not be able to handle it. You can increase the number of servers, distribute the requests to each server, and change the original request from a single server to request distribution to multiple servers.

Reasons for using Nginx's Tcp load balancing module

  • Ability to distribute client requests to different servers according to the load balancing algorithm
  • Ability to maintain a heartbeat mechanism with the server to detect server failures
  • Ability to discover newly added servers to facilitate expansion of the number of servers

insert image description here

Redis distribution-subscription mechanism solves cross-server communication problems

In stand-alone mode, if the receiver is online, find the receiver's connection directly in the connection collection and actively push the message to the receiver

But in the case of multiple machines, the receiver may not be on the same server as the sender, so there is no way to find the corresponding connection to push

Let each server directly establish a TCP connection to synchronize the connection information saved by each server

  • It will lead to high coupling between servers, which is not conducive to system shrinkage and expansion
  • It will take up a lot of socket resources in the system and needs to be synchronized

Introduce middleware message queue

Decouple each server, make the whole system loosely coupled, improve the responsiveness of the server, and save the bandwidth resources of the server

Set the corresponding channel in Redis to call back when a message arrives

// 注册消息以及对应的handler回调操作

// 服务器的ChatService在创建时会与Redis建立连接
// 开启一个线程监听通道上的事件，有消息给业务层进行上报
bool Redis::connect()
{
    // 负责publish发布消息的上下文连接
    publishContext_ = redisConnect("127.0.0.1", 6379);
    if (nullptr == publishContext_)
    {
        cerr << "connect redis failed!" << endl;
        return false;
    }

    // 负责subscribe订阅消息的上下文连接
    subcribeContext_ = redisConnect("127.0.0.1", 6379);
    if (nullptr == subcribeContext_)
    {
        cerr << "connect redis failed!" << endl;
        return false;
    }

    seqContext_ = redisConnect("127.0.0.1", 6379);
    if (nullptr == seqContext_)
    {
        cerr << "connect redis failed!" << endl;
        return false;
    }

    // 在单独的线程中，监听通道上的事件，有消息给业务层进行上报
    thread t([&]() {
        observerChannelMessage();
    });
    t.detach();

    cout << "connect redis-server success!" << endl;

    return true;
}

// 在独立线程中接收订阅通道中的消息
void Redis::observerChannelMessage ()
{
    redisReply *reply = nullptr;
    while (REDIS_OK == redisGetReply(this->subcribeContext_, (void **)&reply))
    {
        // 订阅收到的消息是一个带三元素的数组
        if (reply != nullptr && reply->element[2] != nullptr && reply->element[2]->str != nullptr)
        {
            // 给业务层上报通道上发生的消息
            notifyMessageHandler_(atoi(reply->element[1]->str) , reply->element[2]->str);
        }

        freeReplyObject(reply);
    }

    cerr << ">>>>>>>>>>>>> observer_channel_message quit <<<<<<<<<<<<<" << endl;
}

void Redis::initNotifyHandler (function<void(int,string)> fn)
{
    this->notifyMessageHandler_ = fn;
}

ChatService::ChatService() {
    // 初始化redis
    if (redis_.connect()) {
        // 设置上报消息回调
  redis_.initNotifyHandler(std::bind(&ChatService::handleRedisSubscribeMessage,this,_1,_2));
    }
}

User a logs in on server A, user b logs in on server B, and a wants to send a message to b

  • When b logs in, the B server will subscribe to b's information in Redis
  • When a wants to send a message to b, it will check whether there is a connection of b on the A server, and if there is, it will be pushed directly
  • If not, check whether b is online in the database
    • Store offline messages if not online
  • If online, publish the message to the channel of b through the publish-subscribe mechanism of Redis
  • The Redis thread of the B server will monitor that there is a message in the channel, and execute the corresponding callback function
    • Get the receiver id and message, find the connection of b on the B server and push the message to b
void ChatService::handleRedisSubscribeMessage(int userid, string msg) {
    lock_guard<mutex> lock(connMutex_);
    auto it = userConnMap_.find(userid);
    if (it != userConnMap_.end()) {
        it->second->send(msg);
        return;
    }

    // 存储该用户的离线消息
    offlineMsgModel_.insert(userid,msg);
}

在这里插入代码片

User a logs in on server A, user b logs in on server B, and a wants to send a message to b

  • When b logs in, the B server will subscribe to b's information in Redis
  • When a wants to send a message to b, it will check whether there is a connection of b on the A server, and if there is, it will be pushed directly
  • If not, check whether b is online in the database
    • Store offline messages if not online
  • If online, publish the message to the channel of b through the publish-subscribe mechanism of Redis
  • The Redis thread of the B server will monitor that there is a message in the channel, and execute the corresponding callback function
    • Get the receiver id and message, find the connection of b on the B server and push the message to b
void ChatService::handleRedisSubscribeMessage(int userid, string msg) {
    lock_guard<mutex> lock(connMutex_);
    auto it = userConnMap_.find(userid);
    if (it != userConnMap_.end()) {
        it->second->send(msg);
        return;
    }

    // 存储该用户的离线消息
    offlineMsgModel_.insert(userid,msg);
}

Guess you like

Origin blog.csdn.net/blll0/article/details/127218138
Recommended
Ranking
A quick primer on numpy basics
Two sister tease python project
Java | Multiple keywords in the replacement content are returned to the front end in red style
C ++: references
The string leetcood 1018 is a binary prefix divisible by 5
Flutter 布局组件
Java combat spingboot-redis
PMP pre-exam sprint and wrong questions sorting
ActiveMq C# Message Features: Delayed and Timed Message Delivery
DSP28377S_ copy a program from the RAM to FLASH portion DETAILS
Daily
More
2024-05-16(23)
2024-05-15(5)
2024-05-14(9)
2024-05-13(8)
2024-05-12(28)
2024-05-11(32)
2024-05-10(34)
2024-05-09(32)
2024-05-08(18)
2024-05-07(34)

Code World

© 2018 codetd.com