Common CP systems include etcd and consul, and the opposite of common is a dedicated system. So in some cases there is such a demand.
However, the usability of etcd embed is extremely poor. There will be various problems when running on Windows, and the protocol cannot be customized. You must use the protocol defined by etcd and the client to communicate with the etcd cluster. So the choice at this time:
1. endure
2. Implement a raft algorithm library by yourself, and apply it on it
There is a certain possibility, at least MIT 6.824 can be made, but there is still a big gap with industrial applications
3. Find an industrial-grade raft library, and then apply it on it
At this time, go to the Raft Consensus Algorithm and find several optional Raft algorithm libraries, such as braft, hashicorp/raft, lni/dragonboat.
However, C++ code is more difficult to write, so the braft is passed. Only consul raft and dragonboat are left.
This article uses consul raft to do a simple KeyValue service.
First of all, the gin used in the front-end provides put/get/inc/delete interfaces, and all three interfaces use the raft state machine.Because it supports multiple nodes, the internal non-leader nodes need to forward the request to the leader node.
The front-end code looks like this:
func (this *ApiService) Start() error {
//转发请求给leader节点
this.router.Use(this.proxyHandler())
this.router.POST("/get", this.Get)
this.router.POST("/put", this.Put)
this.router.POST("/delete", this.Delete)
this.router.POST("/inc", this.Inc)
address := fmt.Sprintf(":%d", this.port)
return this.router.Run(address)
}
The request is very simple, that is, directly plug the command or the primitive provided by the service into the Raft state machine and wait for the Raft state to apply, and then get the result (future/promise mode), such as the put command:
func (this *ApiService) Put(ctx *gin.Context) {
req := &Request{}
if err := ctx.ShouldBindJSON(req); err != nil {
ctx.JSON(http.StatusBadRequest, Response{
Error: err.Error(),
})
return
}
result, err := this.raft.ApplyCommand(raft.CommandPut, req.Key, req.Value)
if err != nil {
ctx.JSON(http.StatusInternalServerError, Response{
Error: err.Error(),
})
return
}
ctx.JSON(http.StatusOK, Response{
Value: result.Value,
})
}
There is also an interceptor on the front end that forwards the request to the leader node (? It should be called this name, in fact, it is a kind of pipeline mode)
func (this *ApiService) proxyHandler() gin.HandlerFunc {
return func(context *gin.Context) {
if this.raft.IsLeader() {
context.Next()
} else {
leaderServiceAddress := this.raft.GetLeaderServiceAddress()
if this.leaderServiceAddress != leaderServiceAddress {
Director := func(req *http.Request) {
req.URL.Scheme = "http"
req.URL.Host = leaderServiceAddress
}
this.leaderProxy = &httputil.ReverseProxy{
Director: Director,
}
this.leaderServiceAddress = leaderServiceAddress
}
this.leaderProxy.ServeHTTP(context.Writer, context.Request)
context.Abort()
}
}
}
The following is the treatment of the agreement :
func (this *FSM) Apply(log *raft.Log) interface{} {
result := &FSMApplyResult{
Success: false,
}
t, cmd, err := raftLogToCommand(log)
if err != nil {
result.Error = err
return result
}
binary.LittleEndian.PutUint64(keyCache, uint64(cmd.Key))
binary.LittleEndian.PutUint64(valueCache, uint64(cmd.Value))
switch t {
case CommandPut:
result.Success, result.Error = this.add(keyCache, valueCache)
case CommandDelete:
result.Success, result.Error = this.delete(keyCache)
case CommandGet:
result.Value, result.Error = this.get(keyCache)
case CommandInc:
result.Value, result.Error = this.inc(keyCache, cmd.Value)
}
return result
}
The commands input to the Raft state are actually serialized. The Raft state machine will save the commands to the Storage (memory, or disk/DB, etc.). So when applying the command, first perform the raft log Decode, then switch to process.
Let's take a look at the processing of for example inc:
func (this *FSM) inc(key []byte, add int64) (int64, error) {
var value int64 = 0
err := this.db.Update(func(tx *bbolt.Tx) error {
b, err := tx.CreateBucketIfNotExists(BBoltBucket)
if err != nil {
return err
}
valueBytes := b.Get(key)
if len(valueBytes) != 8 {
logging.Errorf("FSM.inc, key:%d, value length:%d, Reset",
int64(binary.LittleEndian.Uint64(key)), len(valueBytes))
valueBytes = make([]byte, 8)
}
value = int64(binary.LittleEndian.Uint64(valueBytes))
value += add
binary.LittleEndian.PutUint64(valueBytes, uint64(value))
err = b.Put(key, valueBytes)
return err
})
if err != nil {
return -1, err
}
return value, err
}
This instruction is a bit more complicated. You need to find it in the db first. If you find it, add an N, store it, and return the new value. Because the raft state machine applies log sequentially, so there is no need to lock it. Yes, inc itself is atomic.
So far, a simple distributed KeyValue service has been implemented, and it is also a CP system.
Of course, this is just a demo, the actual application is far more complicated than this, this article just provides an idea.
You don’t have to tie yourself to Etcd, all roads lead to Rome . If your system only needs to provide limited operating principles, then you can consider Consul Raft or DragonBoat to create a custom protocol CP service. Ant’s SOFARaft also You can do this kind of thing.
Finally, I prepared some Java architecture learning materials for everyone. The learning technology content includes: Spring, Dubbo, MyBatis, RPC, source code analysis, high concurrency, high performance, distributed, performance optimization, advanced microservices Architecture development, etc., click here to receive directly.