MYSQL quickly synchronizes data to Redis

Example scenario: store the task data of game players, and synchronize the data of players in mysql to redis when the game server starts.

Import data from MySQL into Redis Hash structure. Of course, the most direct way is to traverse the MySQL data and write it into Redis one by one. There is nothing wrong with this, but it will be very slow. If you can make the query output data of MySQL directly match the input data protocol of the Redis command line, you can save a lot of consumption and shorten the time.

Mysql database name is: GAME_DB, table structure example:

CREATE TABLE TABLE_MISSION (
    playerId int(11) unsigned NOT NULL,
    missionList varchar(255) NOT NULL,
    PRIMARY KEY (playerId)
);

Data structures in Redis use hash tables:

The key KEY is mission, the hash field is the corresponding playerId in mysql, and the hash value is the corresponding missionList in mysql. Data are as follows:

[root@iZ23zcsdouzZ ~]# redis-cli
127.0.0.1:6379> hget missions 36598
"{\"10001\":{\"status\":1,\"progress\":0},\"10002\":{\"status\":1,\"progress\":0},\"10003\":{\"status\":1,\"progress\":0},\"10004\":{\"status\":1,\"progress\":0}}"

Quick sync method:

Create a new suffix .sql file: mysql2redis_mission.sql

The content is as follows:

SELECT CONCAT(
  "*4\r\n",
  '$', LENGTH(redis_cmd), '\r\n',
  redis_cmd, '\r\n',
  '$', LENGTH(redis_key), '\r\n',
  redis_key, '\r\n',
  '$', LENGTH(hkey), '\r\n',
  hkey, '\r\n',
  '$', LENGTH(hval), '\r\n',
  hval, '\r'
)
FROM (
  SELECT
  'HSET' as redis_cmd,
  'missions' AS redis_key,
  playerId AS hkey,
  missionList AS hval
  FROM TABLE_MISSION
) AS t

Create shell script mysql2redis_mission.sh
content:

mysql GAME_DB --skip-column-names --raw < mission.sql | redis-cli --pipe

The Linux system terminal executes the shell script or directly runs the system command to synchronize the TABLE_MISSION data of the mysql database GAME_DB table to the key missions in redis. The mysql2redis_mission.sql file matches the output data format of mysql data with the input data format protocol of redis, thereby greatly shortening the synchronization time.

After testing, it takes 5 minutes for the same piece of data to be synchronously written to redis by taking out and modifying the data format in a single piece. Using the above sql file and shell command, it only takes about 3 seconds to synchronize the data.