Reprinted from the article: http://www.pythonheidong.com/blog/article/4052/
The main function of the development of the recent busy, and sometimes feel nothing worth sharing. Data is stored on this topic simply to talk, the way he is thinking about the reason, I hope to do it smoothly back a little, we now project data storage simply is not done yet, simply put each individual player data stored in a file on your hard disk: -)
About Performance
Online game data storage has its particularity. The game is relatively strong interaction product, the system response time requirements are particularly high, if the IO to database synchronization when the processing logic is unacceptable. Also during the game player data changes very dramatic, to say the frequency of writing data will be particularly high, if poorly designed, the data storage system is very easy to become a bottleneck.
We do wish to take a more BBS system. When we visit BBS browsing most of the time, compared to posting behavior is a small probability event. Users spend 20 minutes to finish a post click the Publish button, indicating successfully sent are perfectly acceptable after 3 seconds or 10 seconds, or even prompt asks the user to re-send failed Zaifayici not a big problem. Therefore, the above-described system BBS, when the user submits a request to read and write the database synchronization is feasible. The online games have to do is to support thousands of people at intervals of a few seconds keep posting, to ensure that the response time control within a few milliseconds, but also guarantee issued to quote other people can see immediately.
Common practice in the industry is the function of the database as simple as possible, the final plan and only database for data backup repository to use. That is loaded at startup from the database server processes all the global data, load all the data players from the database when the players on the line, after all logic operations are performed in memory, thus avoiding the course of the game to bring the database to read and write response is too slow. Data changes at regular intervals (typically several minutes) asynchronous write to the database, thus easing the pressure on the database writing.
Obviously, such a database to the required function is so simple, just to have two interfaces GET and SET completely sufficient. So choose the right NoSQL database to improve data access performance also granted, when necessary, adding memcache to improve the speed of reading data is also logical thing. This is also the reason we did not just save the database file can be normal development features - after all files read and write files is GET and SET operations Well, when a change to the interface on the line.
Of course this approach is from the traditional MMORPG evolved. Now a lot of hand travel interactivity is relatively weak, not strong real-time, read and write directly to the database when the user is often totally decent amount of hours of processing logic. However, in the system design stage to put out the hidden data stored to avoid or good, then it could fire a game of it? There are clouds on the wind's blog can refer to specific cases: talk about unfamiliar street hegemony stepped in databases pit (chart papers) .
About Disaster Recovery
Lost file server may be a game programmer is always a nightmare, though they do not personally experienced, I believe that misfortune trauma operations game lost files and processed data recovery heart programmer should have an indelible ......
Like planning often do not understand why a 100% write program does not crash so hard, software programmers often do not understand why would shoot down the host server, the hard disk will be written directly to waste. In fact, hardware engineers, it seems, a variety of computer hardware and constantly question is normal now. Ah I want to say is that we should should consider a good disaster recovery system designed to minimize losses caused by system failures.
Process crashes
The foregoing discussion, in order to quickly respond to client requests, and to reduce the pressure of writing the database, the data does not change immediately into the database, but to save a certain time interval. In fact, this practice is very tricky question: complete data memory after we tell the client to modify the operation is complete, but the data did not succeed this time landing, if the time course of the game will result in the collapse of abnormal retreated, retreated the maximum duration for the save interval (minutes).
Then the game process can not fail to collapse it? Is possible, but more difficult, because the game is often complex and iterative process is crazy, is not 100% guaranteed to collapse, I think have to see specific languages and frameworks. Again, we try to put off the risk aversion in the design phase.
The main idea is to use a stable process to share the risk. The aforementioned process can be added to improve the performance of memcache read data, it is natural that we can use memcache stable process for temporarily storing data. Program is this: each has both start a process on the host memcache processes running game and has been run, starting memcache read when reading data, if the read fails then the database is read, when a player changes the data written synchronously memcache . Even though the game process crashes after the restart will first read out the correct data from memcache.
Write data to memcache is usually via HTTP or socket, the client will request a response rate slightly affected, because the two processes on the same host, is generally acceptable. If the game is particularly sensitive to the speed of response, you can use shared memory inter-process communication way, a few years ago to do the tour MMORPG that is used in this way, but the shared memory is not easy to achieve, high complexity procedures. Both ways no right or wrong, just need to weigh according to the needs of the game.
Host crash
According to previous designs, processes and memcache games are on the same host, once the host crashes will inevitably result in lost files. This is still a need to weigh the issue.
- If the small probability of lost files can be tolerated for a short time, then it is so, do performance testing will save time as short as possible point on the line. After all, linux is more stable system, the collapse of the host is not the norm. Really a problem with players such as Charles compensation log back like when complaints were made.
- If the game is not sensitive to the speed of response, then memcache to another host. The time being that the probability of the collapse of the two hosts at the same time is negligible.
- If you want best of both worlds, you can consider retaining the original memcache on the basis of a new memcache on another host. Of course, this way the system is more complex, I personally do not recommend.
Database corruption
Game database is often hard to use RAID technology, theoretically almost impossible damaged. But the fear of natural disasters hit, the engine room of natural disasters or, programmers misuse or, if you do not do a backup once a problem is absolutely fatal, directly destroy a company is not impossible.
Technically a database backup in fact nothing to say, nothing is separate from the main backup to read and write. Here I want to share that experience before.
On the company's database is used Tokyo Cabinet , this is a very simple KV database. The problem is that seemingly few people use, documents are also relatively scarce, then we do not know the actual deployment configuration bug or not, the master will always be some of sync from the database. After a while we just toss the idea of change, it does not rely on master-slave synchronization mechanisms, and directly to the two databases are save when the save. Because the archiving process is asynchronous, system bottlenecks are not here, so keep once or twice saved just do not care. If you are not familiar with the operation and maintenance of the database and a headache, then you can consider this idea :-)
Imagine proxycache
In fact, relatively closed the game server field, above NoSQL involved, memcache components are developed from the Web, to some extent, it is not particularly suitable for game server. The above game design process took on too much data storage features, such as changes in the player's hand to synchronize data stored in memcache, on the one hand have to pinch and press timed intervals stored in the database table; when another example must first read data memcache to query and found a cache miss reading Shihai to go to the database.
I think that can store data part is completely detached from the process out of the game, available only to the game process GET and SET interface, so you can focus on the game process the processing logic. Pulled out of the process that is imagined proxycache, and can be used to replace the previous design memcache.
proxycache is a combination of two roles.
First, it is the agent game as a proxy process to access the database. After whereabouts own database request is received GET request is forwarded back to the game process, received written to the database at regular intervals when the SET request. Can be added to the delay parameter in the SET parameters, such insignificant data changes can accept memory disk 5 minutes, may require significant changes in the data written to the database immediately. According to internal proxycache request parameter lined up in order of priority queue after storage, such abnormal downtime caused by loss can be minimized.
Second, it can cache as cache data. Read data from the database back to the game and the process of writing the incoming data can be cached, not only to achieve memcache functionality also simplifies the process logic game.
Of course now only on paper only, pumping empty hope to achieve out after :-)
Select database
A major reason our project database totally did not start doing what is not selected database. Currently I expect the database is this: The first is simple enough NoSQL database, the best is the simplest Key-Value, according to our design Key only string, Value Blob just completely enough; then hope It runs as a separate process; then there are operation and maintenance to be convenient, if only one data file that is excellent.
First looked Redis. Frankly, I think Redis is not suitable for online games. Redis was originally designed as a memory database, in terms of data handled rough landing, fork way bgsave have, you must set aside enough memory, how you feel wasted. And after a period of online game operators are often the most cold data in the database, Redis indiscriminate white will fall into memory is a bit unacceptable. It may still be used as a cache might be the right path now.
Then there MongoDB. But the powerful complexity far beyond our expectations, a problem is intractable fear.
Now two main ideas:
- More familiar with Tokyo Cabinet , but this project has not been updated in many years, there is no community or something. Now we use the Go language, to write a driver is certainly the inevitable.
- rocksDB or levelDB package. The options are LedisDB , SSDB , there are many similar, can not really judge is not reliable ToT
You have a right to know it? Seeking recommendations!
Reprinted from the article: http://www.pythonheidong.com/blog/article/4052/