Article directory

I. Introduction
- 1.1 Reasons for needing to lock trading orders
- 1.2 Locking scheme
2. Go + Redis implements distributed locks

I. Introduction

1.1 Reasons for needing to lock trading orders

Before starting to share this article, let's briefly describe the project. The project is a centralized wallet. After java receives the user's Ethereum transfer request, it calls the transfer interface of the backend golang service to send the transaction to the chain.

If the golang service normally returns the transaction hash to java after processing the transaction, it means that the transaction has been sent to the chain, and then check whether the transaction hash is on the chain. If java does not receive the transaction hash returned by golang due to network and other reasons, it is considered that there is a problem with the transaction, java should put the transaction in a pending state, java should not continue to send the order transaction, but wait for manual intervention to check specific reason. This prevents users from double spending.

After communicating the above rules with colleagues in the java layer, the java layer believes that: after the transaction fails, it will not retry, but if the code error causes a bug or the transaction is concurrent (the queue should be used when sending the transaction, otherwise it will definitely appear Transaction concurrency), multiple transactions of the same order may be sent, so it needs to be locked in the backend golang for the last interception.

1.2 Locking scheme

levelDB or mysql persistent storage

Because golang supports the use of levelDB for key value storage, java transaction orders can be persistently stored using levelDB. The transaction order that has been received will not be processed again. If the transaction needs to be re-initiated, the transaction order will be changed by java and then resent.

Because the golang service needs to deploy multi-node load later, the storage of multi-node levelDB can use shared storage, but the feature of levelDB is that only one process is allowed to access a specific database at a time. So it cannot be used as distributed storage.

say again

Use redis to store key values and implement simple distributed locks. The disadvantages of using this method are:

The key value has an expiration time. If the java layer still retries the transaction after the key value is invalid, the double-spending phenomenon will still occur. Therefore, it is necessary to manually intervene in the troubleshooting before the key value becomes invalid.
If the redis library is Flush, the key value does not exist, and the problem transaction will be released
The redis service is down, the key value cannot be obtained, and the transaction double-spending problem will also occur.

2. Go + Redis implements distributed locks

2.1 Why do we need distributed locks?

User places an order

Lock the uid to prevent repeated orders.

Inventory deduction

Lock up inventory to prevent oversold.

Balance deduction

Lock the account to prevent concurrent operations.

When sharing the same resource in a distributed system, distributed locks are often required to ensure the consistency of changing resources.

2.2 Distributed locks need to have characteristics

exclusivity

Basic property of a lock, and can only be held by the first holder.

Anti-deadlock

In high concurrency scenarios, once deadlock occurs on critical resources, it is very difficult to troubleshoot. Usually, it can be avoided by setting the timeout period to automatically release the lock.

3. Reentrant

The lock holder supports reentrancy, preventing the lock from being released by timeout when the lock holder re-entries again.

4. High performance and high availability

The lock is the key pre-node for the code to run. Once it is unavailable, the business will report a failure directly. In high concurrency scenarios, high performance and high availability are the basic requirements.

2.3 What knowledge points should be mastered before implementing Redis lock

set command

SET key value [EX seconds] [PX milliseconds] [NX|XX]

EXsecond : Set the expiration time of the key to second seconds. SET key value EX second has the same effect as SETEX key second value .
PXmillisecond : Set the key's expiration time to millisecond milliseconds. SET key value PX millisecond has the same effect as PSETEX key millisecond value .
NX: Set the key only when the key does not exist. SET key value NX has the same effect as SETNX key value .
XX: Set the key only when the key already exists.

First use setnx to grab the lock. If it is grabbed, then use expire to set an expiration time for the lock to prevent the lock from forgetting to release.

setnx method

Let's take a look at the setnx method first:

unc (c *cmdable) SetNX(key string, value interface{}, expiration time.Duration) *BoolCmd {
	var cmd *BoolCmd
	if expiration == 0 {
		// Use old `SETNX` to support old Redis versions.
		cmd = NewBoolCmd("setnx", key, value)
	} else {
		if usePrecise(expiration) {
			cmd = NewBoolCmd("set", key, value, "px", formatMs(expiration), "nx")
		} else {
			cmd = NewBoolCmd("set", key, value, "ex", formatSec(expiration), "nx")
		}
	}
	c.process(cmd)
	return cmd
}

The meaning of setnx is SET if Not Exists, the method is atomic. If the key does not exist, setting the current key as value succeeds and returns 1; if the current key already exists, setting the current key fails and returns 0.

expire(key, seconds)
expire sets the expiration time. It should be noted that the setnx command cannot set the timeout time of the key, and can only be set for the key through expire().

2.4 golang connect redis

download reids package

go get github.com/go-redis/redis

golang connect redis

package redis

import (
	"github.com/go-redis/redis"
	"wallet/config"
	"wallet/pkg/logger"
)

// RedisDB Redis的DB对象
var RedisDB *redis.Client

func NewRedis() {  //创建redis连接
	RedisDB = redis.NewClient(&redis.Options{
		Addr:     config.Conf.Redis.Host,   //redis连接地址
		Password: config.Conf.Redis.Password,  //redis连接密码
		DB:       config.Conf.Redis.Database,  //redis连接库
	})

	defer func() {
		if r := recover(); r != nil {
			logger.Error("Redis connection error,", r)
		}
	}()
	_, err := RedisDB.Ping().Result()
	if err != nil {
		panic(err)
	}
	logger.Info("Redis connection successful!")
}

When the project starts, a redis connection should be created first

func main() {
	//连接redis
	redis.NewRedis()
}

2.5 golang + redis to realize distributed lock

Use the Set method of redis to store keys, and simply implement a locking method

//判断当前订单是否已进行处理
isExist := redis.RedisDB.Exists(order)
//判断是否获取到key值，若获取到，则说明该交易订单已请求，向调用者返回报错
if isExist.Val() != 0 {
	apicommon.ReturnErrorResponse(ctx, 1, "Order transaction already exists, please check transaction", "")
	return
} else { //若未获取到，则说明暂未处理此笔交易订单，向redis中set此订单
	redis.RedisDB.Set(order, order, 86400*time.Second)
}

Using the setnx method of redis to realize distributed locking

//判断当前订单是否已进行处理
bool := redis.RedisDB.SetNX(order, order, 24*time.Hour)
if bool.Val() { //SetNX只进行一次操作，若返回为true，则之前未处理该订单，此次已set该key
	logger.Info("The transaction order key value has been saved")
} else { //若返回false，则说明该交易订单已请求，向调用者返回报错
	logger.Error("The transaction order key value has been saved")
	apicommon.ReturnErrorResponse(ctx, 1, "Order transaction already exists, please check transaction", "")
	return
}

2.6 Summary

From the code and execution results, we can see that our remote call to setnx is actually very similar to the stand-alone trylock. If the lock acquisition fails, the related task logic should not continue to be executed.

setnx is very suitable for competing for some "unique" resources in high concurrency scenarios. For example, in the transaction matching system, a seller initiates an order, and multiple buyers will compete for it concurrently. In this scenario, we have no way to rely on the specific time to judge the sequence, because whether it is the time of the user equipment or the time of each machine in a distributed scenario, there is no way to ensure the correct timing after the merger. Even if it is a cluster in the same computer room, the system time of different machines may be slightly different.

Therefore, we need to rely on the order in which these requests arrive at the redis node to do the correct lock grab operation. If the user's network environment is relatively poor, it can only be self-sufficient.

refer to:

Golang operates redis official documentation: https://pkg.go.dev/github.com/go-redis/redis#Client.Expire
golang distributed lock: https://books.studygolang.com/advanced-go-programming-book/ch6-cloud/ch6-01-lock.html

Go + Redis implements distributed locks