Talking about MMORPG server architecture
foreword
Although this article is about the MMO server architecture, it may just be my own thoughts. This is a review and summary of the few months since I entered the company. Let me talk about the project first. Our project is an old MMO client game with a history of more than ten years. There are a series of problems such as the old structure and bloated code. There is still a lot to learn.
As a novice game programmer, I can get in touch with more complex MMO servers. However, our project has been online for many years, and many things have been developed. Compared with developing projects from scratch, we will lack some experience from design to implementation.
Game Server Features
Because there is no common solution for game servers, there may be a large gap between the usage solutions of different game types.
I personally divide game servers into three categories
1. Big DUA casual game
2. Room type game server
3. MMO game server
The room type game server is special because it mainly depends on the game type in the room.
Big DUA Casual Games
Big DUA casual game is a type of game with relatively simple game logic. It is characterized by relatively simple logic and large number of visits.
As far as I know, such a casual game will be a global server, and generally adopts a more Internet-like architecture. Maybe nginx will be used as a gateway, and the previous redis cluster will be used as a cache. Later, mysql will be divided into databases and tables, and the internal logic of the game will be given to customers. When the terminal is lost, the logic of the server will be less than that of other game types.
room type game
Compared with casual games, room-type games have more logic and are likely to involve game synchronization issues. However, casual games may also be room-type games. The two are intersecting, and the distinction is not strictly made here.
Room-type games include MOBA like League of Legends, racing like QQ Speed, or FPS like Chicken Eater. There are many types of games, and the game type of room-type games depends on the content in the room. The difficulty also depends on the content. For example, League of Legends and Youhua are both room-opening games, but the difficulty is different for the naked eye.
However, compared with MMO games, room-type games have an advantage that rooms are naturally isolated from each other, and it has always been difficult for MMO game servers to achieve seamless maps.
MMO game server
MMO game servers are quite different from the former two. Personally, I think the biggest difference is the number of players on a logical server. There is a high probability that there is no such thing as a logical server in casual games, so casual servers will not be discussed. Generally speaking, ten players in a room are enough for room-type game servers, and each room is naturally isolated from each other, so it is easy to add machines to expand capacity to solve the problem.
On the MMO game server, there may be hundreds or even thousands of people in one scene, and different scenes can be separated, but it is not easy to cut a single scene (but kbengine has already realized it), and there are many problems involved. Therefore, MMO generally uses one process or one thread to be responsible for one scene to cut unused scenes, which is easy to handle and does not involve concurrency issues.
But whether it is a single thread or a single process, the network will become a pain point. To give an extreme example, there are 100 players on the same screen in a certain scene (both belong to the same AOI discovery range, and assume that each packet reaches the MTU size), then The required network bandwidth is 100 X 100 X 1500byte, which is about 140Mbps, and there may be multiple such situations in a single scene, but this is quite extreme.
MMORPG architecture
Now let me share, a client game MMO server architecture of my current project, but although this architecture is relatively old, it is still very valuable to learn.
First, let's explain the meaning of each picture. First of all, this architecture is a multi-process and single-thread architecture. Except for the client in the figure, each box represents a process, and each process has only one thread.
GateWay list server
GateWay List Service is the server responsible for assigning GateWay server addresses to customers. Usually the client communicates with the server for the first time, the first is to go to the GateWay server IP address obtained from the GateWay list server. Disconnect after getting it.
The main function of the GateWay list server in my current project is to dynamically allocate the low-load GateWay to the client, and play a load balancing role.
GateWay server
The GateWay server is a server similar to a gateway. The GateWay server does not process any data content, but only forwards and encrypts and decrypts. Data will be sent to GMS server.
GMS server
The GMS server manages player information, but is not responsible for the logic operations in the game. The logic operations in the game are all handled by the scene server, so the functions of GMS are roughly divided into two parts. One is responsible for all the logic before the player enters the scene (such as login) , The second is to save the player's context information (such as account role information, which scene server, etc.), so that the player can switch between scenes.
GMS will play an important role when the player enters the scene, because different scene servers will be responsible for different scenes. When the player enters the scene, the GMS server will choose which scene server to forward the message to. But there is a problem here. One GMS server will correspond to multiple other servers, and there is only one GMS server globally. When I first saw it, I considered whether it would become a single point problem, but I thought about it later. Because if the GMS server reaches the bottleneck, the scene server may collapse very early, because the logical server is computationally intensive, and a single process in a single scene may only be able to carry thousands of people. After all, it is far from the stand-alone limit. Unless a single scene is made into a multi-threaded form, that is, a seamless large map form, it is necessary to make the GMS distributed to avoid single points.
scene server
The scene server is responsible for all logical operations in the game, such as movement, attack, copying, etc. Arguably the most important part of a game server. In this architecture, scene servers exist in the form of groups, and each scene server is responsible for its own scene. There is no direct data transmission between the scene server and the scene service area (although there is no scene, there may be scenes in extreme environments. The server communicates directly with the scene server).
But there is one exception, that is the global scene server. As the name suggests, the global scene server is responsible for solving some problems that may arise in multiple scenes. For example, the global guild server, the operation logic related to guilds will be processed on this server. Because some operations of the guild may occur in any scene server, but it is necessary to avoid the problem of multi-thread contention, a global service server is directly allocated to handle functions such as the guild.
shared memory server
The shared memory server can be regarded as a DB cache. If you replace this with redis, then everyone will understand, but one thing to note is that the scene server does not have to pull data from the shared memory server for every operation. A logical frame is only 40ms, which is too long, and the main data is still directly operated in memory. So basically when there are more important write operations, it is thrown to the shared server for writing, and then the shared server saves it to the DB regularly.
DB
DB is also a database, here is a special explanation, the above picture is a bit wrong, DB not only communicates with the logic server, but also is responsible for regularly writing the data of the shared memory server, but only the particularly important data logic server directly goes to the DB ( Such as krypton gold mall consumption, etc.), ordinary data is still stored in shared memory, and then stored regularly.
In my project, the DB uses the fixed storage method, so the presence of the DB will be a little lower, unlike the Internet that must save the DB every time (although the Internet king directly loses mq now), the query is also to check the shared memory first, check If you don’t check in the DB, basically the common DBs used by game servers are mysql, mongo, and even Berkeley DB, basically whichever you use. However, I feel that it should be possible to directly replace the combination of shared memory + DB with a redis group.
advantage
This kind of multi-process architecture is easier to expand than the multi-threaded one, because multi-process and multi-machine deployment is simpler, just use sockets in the middle.
The combination of shared memory + db guarantees a certain degree of disaster recovery, but if both of them are down, it is still a big problem.
shortcoming
Unable to solve single scene segmentation problem.
If the shared memory and DB hang up at the same time, it depends on the log rollback.
Architecture Improvements and Ideas
Redis
Because the project team had an operation accident before, the power supply of the computer room was completely cut off, and then all the servers were hung up. This shows that shared memory + DB is not necessarily reliable for two servers. My own idea is to switch to redis cluster+DB+mq. In this way, it is not a big problem even if a few units are hung up. The master-slave of redis can be switched quickly. Even if all of them are down, you can go to mq to get the data directly, and you don’t need to analyze the logs.
Expand GMS
Although the GMS bottleneck is still difficult to achieve, anyway, just talk about it casually, probably to expand GMS into a GMS group. When a user accesses, use redis to record which GMS and Gateway the user is using. This is The same result as a hashmap.
However, it will be easier to do cross-server. As far as I know, Tianlong cross-server is the way to do this. They manage n Tservers (similar to GMS) to realize a tree-shaped server structure. Players can easily play across servers, and cross-server data can be easily returned to the original server.