NEO PDFT consensus algorithm
The dbft algorithm, which is confirmed by multiple network requests, finally obtains a majority consensus. The disadvantage is that the network overhead is high. If there is a problem with the network or the performance of the bookkeeper is insufficient, the system speed will be slowed down. If there are too many bookkeepers, it will also lead to the expansion of network communication, making it difficult to quickly reach an agreement. Not suitable for use in public chains. And NEO is positioned as a private chain or a consortium chain. The bookkeeper node is limited, and the machine and network environment can be controlled, so it is suitable for this algorithm. It can not only avoid large computing power overhead but also ensure consistency.
Code structure description
├── Consensus
│ ├── ChangeView.cs //viewchange 消息
│ ├── ConsensusContext.cs //共识上下文
│ ├── ConsensusMessage.cs //共识消息
│ ├── ConsensusMessageType.cs //共识消息类型 ChangeView/PrepareRequest/PrepareResponse
│ ├── ConsensusService.cs //共识核心代码
│ ├── ConsensusState.cs //节点共识状态
│ ├── PrepareRequest.cs //请求消息
│ └── PrepareResponse.cs //签名返回消息
Consensus state change process
- 1: Nodes that open consensus are divided into two categories, non-bookkeepers and bookkeeper nodes, non-bookkeepers do not participate in consensus, and bookkeepers participate in the consensus process
- 2: Select the speaker. The Neo speaker generation mechanism is obtained by MOD operation based on the current block height and the number of bookkeepers. The speaker is actually elected in order.
- 3: Node initialization, the speaker is the primary node, and the councilor is the backup node.
- 4: After meeting the block generation conditions, the speaker sends PrepareRequest
- 5: After the MP receives the request, verify that the PrepareResponse is sent by signing
- 6: After the accounting node receives the PrepareResponse, the node saves the signature information of the other party, and checks that if it exceeds two-thirds, it sends a block
- 7: The node receives the block, and the whole is reinitialized after the PersistCompleted event is triggered.
Consensus context core members
public const uint Version = 0;
public ConsensusState State; //节点当前共识状态
public UInt256 PrevHash;
public uint BlockIndex; //块高度
public byte ViewNumber; //试图状态
public ECPoint[] Validators; //记账人
public int MyIndex; //当前记账人次序
public uint PrimaryIndex; //当前记账的记账人
public uint Timestamp;
public ulong Nonce;
public UInt160 NextConsensus; //共识标识
public UInt256[] TransactionHashes;
public Dictionary<UInt256, Transaction> Transactions;
public byte[][] Signatures; //记账人签名
public byte[] ExpectedView; //记账人试图
public KeyPair KeyPair;
public int M => Validators.Length - (Validators.Length - 1) / 3; //三分之二数量
ExpectedView maintains the view state and is used to initiate a new round of consensus when the speaker fails to work properly. Signatures are used to maintain the confirmation state during the consensus process.
Node consensus status
[Flags]
internal enum ConsensusState : byte
{
Initial = 0x00, // 0
Primary = 0x01, // 1
Backup = 0x02, // 10
RequestSent = 0x04, // 100
RequestReceived = 0x08, // 1000
SignatureSent = 0x10, // 10000
BlockSent = 0x20, // 100000
ViewChanging = 0x40, //1000000
}
Speaker choice
When initializing the consensus state, the PrimaryIndex will be set to learn the current speaker. The principle is a simple MOD operation. There are two cases here. If the node is normal, the block height and the number of bookkeepers can be directly modulated. If there is a situation, it needs to be adjusted according to view_number.
//file /Consensus/ConsensusService.cs InitializeConsensus方法
if (view_number == 0)
context.Reset(wallet);
else
context.ChangeView(view_number);
//file /Consensus/ConsensusContext.cs
public void ChangeView(byte view_number)
{
int p = ((int)BlockIndex - view_number) % Validators.Length;
State &= ConsensusState.SignatureSent;
ViewNumber = view_number;
PrimaryIndex = p >= 0 ? (uint)p : (uint)(p + Validators.Length);//当前记账人
if (State == ConsensusState.Initial)
{
TransactionHashes = null;
Signatures = new byte[Validators.Length][];
}
ExpectedView[MyIndex] = view_number;
_header = null;
}
//file /Consensus/ConsensusContext.cs
public void Reset(Wallet wallet)
{
State = ConsensusState.Initial;
PrevHash = Blockchain.Default.CurrentBlockHash;
BlockIndex = Blockchain.Default.Height + 1;
ViewNumber = 0;
Validators = Blockchain.Default.GetValidators();
MyIndex = -1;
PrimaryIndex = BlockIndex % (uint)Validators.Length; //当前记账人
TransactionHashes = null;
Signatures = new byte[Validators.Length][];
ExpectedView = new byte[Validators.Length];
KeyPair = null;
for (int i = 0; i < Validators.Length; i++)
{
WalletAccount account = wallet.GetAccount(Validators[i]);
if (account?.HasKey == true)
{
MyIndex = i;
KeyPair = account.GetKey();
break;
}
}
_header = null;
}
state initialization
If it is the speaker, the state is marked as ConsensusState.Primary, and the timer is changed to trigger the event, which is triggered 15s after the last block. The parliamentarian sets the state to ConsensusState.Backup, and the time is adjusted to trigger after 30s. If the speaker cannot work normally, this trigger will start to work (the details will be analyzed in detail later).
//file /Consensus/ConsensusContext.cs
private void InitializeConsensus(byte view_number)
{
lock (context)
{
if (view_number == 0)
context.Reset(wallet);
else
context.ChangeView(view_number);
if (context.MyIndex < 0) return;
Log($"initialize: height={context.BlockIndex} view={view_number} index={context.MyIndex} role={(context.MyIndex == context.PrimaryIndex ? ConsensusState.Primary : ConsensusState.Backup)}");
if (context.MyIndex == context.PrimaryIndex)
{
context.State |= ConsensusState.Primary;
if (!context.State.HasFlag(ConsensusState.SignatureSent))
{
FillContext(); //生成mine区块
}
if (context.TransactionHashes.Length > 1)
{ //广播自身的交易
InvPayload invPayload = InvPayload.Create(InventoryType.TX, context.TransactionHashes.Skip(1).ToArray());
foreach (RemoteNode node in localNode.GetRemoteNodes())
node.EnqueueMessage("inv", invPayload);
}
timer_height = context.BlockIndex;
timer_view = view_number;
TimeSpan span = DateTime.Now - block_received_time;
if (span >= Blockchain.TimePerBlock)
timer.Change(0, Timeout.Infinite);
else
timer.Change(Blockchain.TimePerBlock - span, Timeout.InfiniteTimeSpan);
}
else
{
context.State = ConsensusState.Backup;
timer_height = context.BlockIndex;
timer_view = view_number;
timer.Change(TimeSpan.FromSeconds(Blockchain.SecondsPerBlock << (view_number + 1)), Timeout.InfiniteTimeSpan);
}
}
}
Speaker initiates a request
After the speaker arrives at the accounting time, execute the following method, send a MakePrepareRequest request, change its state to RequestSent, and set a timer to repeatedly trigger after 30s (it also takes effect when the speaker works abnormally).
//file /Consensus/ConsensusContext.cs
private void OnTimeout(object state)
{
lock (context)
{
if (timer_height != context.BlockIndex || timer_view != context.ViewNumber) return;
Log($"timeout: height={timer_height} view={timer_view} state={context.State}");
if (context.State.HasFlag(ConsensusState.Primary) && !context.State.HasFlag(ConsensusState.RequestSent))
{
Log($"send perpare request: height={timer_height} view={timer_view}");
context.State |= ConsensusState.RequestSent;
if (!context.State.HasFlag(ConsensusState.SignatureSent))
{
context.Timestamp = Math.Max(DateTime.Now.ToTimestamp(), Blockchain.Default.GetHeader(context.PrevHash).Timestamp + 1);
context.Signatures[context.MyIndex] = context.MakeHeader().Sign(context.KeyPair);
}
SignAndRelay(context.MakePrepareRequest());
timer.Change(TimeSpan.FromSeconds(Blockchain.SecondsPerBlock << (timer_view + 1)), Timeout.InfiniteTimeSpan);
}
else if ((context.State.HasFlag(ConsensusState.Primary) && context.State.HasFlag(ConsensusState.RequestSent)) || context.State.HasFlag(ConsensusState.Backup))
{
RequestChangeView();
}
}
}
}
MPs broadcast trust message
When the MP receives the PrepareRequest, the node information will be verified, and the transaction that does not exist will be synchronized. After the transaction synchronization is completed, the verification will send a PrepareResponse, including its own signature information, indicating that it has passed the node verification. The state transitions to ConsensusState.SignatureSent.
//file /Consensus/ConsensusContext.cs
private void OnPrepareRequestReceived(ConsensusPayload payload, PrepareRequest message)
{
Log($"{nameof(OnPrepareRequestReceived)}: height={payload.BlockIndex} view={message.ViewNumber} index={payload.ValidatorIndex} tx={message.TransactionHashes.Length}");
if (!context.State.HasFlag(ConsensusState.Backup) || context.State.HasFlag(ConsensusState.RequestReceived))
return;
if (payload.ValidatorIndex != context.PrimaryIndex) return;
if (payload.Timestamp <= Blockchain.Default.GetHeader(context.PrevHash).Timestamp || payload.Timestamp > DateTime.Now.AddMinutes(10).ToTimestamp())
{
Log($"Timestamp incorrect: {payload.Timestamp}");
return;
}
context.State |= ConsensusState.RequestReceived;
context.Timestamp = payload.Timestamp;
context.Nonce = message.Nonce;
context.NextConsensus = message.NextConsensus;
context.TransactionHashes = message.TransactionHashes;
context.Transactions = new Dictionary<UInt256, Transaction>();
if (!Crypto.Default.VerifySignature(context.MakeHeader().GetHashData(), message.Signature, context.Validators[payload.ValidatorIndex].EncodePoint(false))) return;
context.Signatures = new byte[context.Validators.Length][];
context.Signatures[payload.ValidatorIndex] = message.Signature;
Dictionary<UInt256, Transaction> mempool = LocalNode.GetMemoryPool().ToDictionary(p => p.Hash);
foreach (UInt256 hash in context.TransactionHashes.Skip(1))
{
if (mempool.TryGetValue(hash, out Transaction tx))
if (!AddTransaction(tx, false))
return;
}
if (!AddTransaction(message.MinerTransaction, true)) return;
if (context.Transactions.Count < context.TransactionHashes.Length)
{
UInt256[] hashes = context.TransactionHashes.Where(i => !context.Transactions.ContainsKey(i)).ToArray();
LocalNode.AllowHashes(hashes);
InvPayload msg = InvPayload.Create(InventoryType.TX, hashes);
foreach (RemoteNode node in localNode.GetRemoteNodes())
node.EnqueueMessage("getdata", msg);
}
}
//file /Consensus/ConsensusContext.cs
private bool AddTransaction(Transaction tx, bool verify)
{
if (Blockchain.Default.ContainsTransaction(tx.Hash) ||
(verify && !tx.Verify(context.Transactions.Values)) ||
!CheckPolicy(tx))
{
Log($"reject tx: {tx.Hash}{Environment.NewLine}{tx.ToArray().ToHexString()}");
RequestChangeView();
return false;
}
context.Transactions[tx.Hash] = tx;
if (context.TransactionHashes.Length == context.Transactions.Count)
{
if (Blockchain.GetConsensusAddress(Blockchain.Default.GetValidators(context.Transactions.Values).ToArray()).Equals(context.NextConsensus))
{
Log($"send perpare response");
context.State |= ConsensusState.SignatureSent;
context.Signatures[context.MyIndex] = context.MakeHeader().Sign(context.KeyPair);
SignAndRelay(context.MakePrepareResponse(context.Signatures[context.MyIndex]));
CheckSignatures();
}
else
{
RequestChangeView();
return false;
}
}
return true;
}
Broadcast blocks after consensus is reached
After receiving the PrepareResponse, other nodes record the signature information of the other party in their own signature list, and then check whether their own signature list has more than two-thirds of the signatures. The state changes to ConsensusState.BlockSent.
private void CheckSignatures()
{
if (context.Signatures.Count(p => p != null) >= context.M && context.TransactionHashes.All(p => context.Transactions.ContainsKey(p)))
{
Contract contract = Contract.CreateMultiSigContract(context.M, context.Validators);
Block block = context.MakeHeader();
ContractParametersContext sc = new ContractParametersContext(block);
for (int i = 0, j = 0; i < context.Validators.Length && j < context.M; i++)
if (context.Signatures[i] != null)
{
sc.AddSignature(contract, context.Validators[i], context.Signatures[i]);
j++;
}
sc.Verifiable.Scripts = sc.GetScripts();
block.Transactions = context.TransactionHashes.Select(p => context.Transactions[p]).ToArray();
Log($"relay block: {block.Hash}");
if (!localNode.Relay(block))
Log($"reject block: {block.Hash}");
context.State |= ConsensusState.BlockSent;
}
}
Restore state after Blockchain_PersistCompleted
After the block is broadcast, the node receives this information, saves the block, triggers the PersistCompleted event after the save is completed, and finally returns to the initial state to start a new round of consensus.
private void Blockchain_PersistCompleted(object sender, Block block)
{
Log($"persist block: {block.Hash}");
block_received_time = DateTime.Now;
InitializeConsensus(0);
}
Error analysis
If the current bookkeeper cannot complete the bookkeeping task
In this case, the speaker still cannot reach a consensus after the time is exceeded. At this time, the parliamentarian will increase his ExpectedView value and broadcast it. If it is checked that two-thirds of the members have added 1 to the viewnumber, then the nodes with the same viewnumber A new round of consensus begins between them, the speaker is re-selected, and a consensus is initiated. If the original speaker returns to normal at this time, it will increase its own viewnumber after the time expires, and slowly catch up with the current view state.
refer to
Source: [ https://github.com/neo-project/neo/tree/master/neo/Consensus )