The Hyperledger Fabric project has attracted the close attention of many companies around the world since its birth. Two major versions have been released, the 0.6 experimental version (September 2016) and the 1.0 official version (July 2017).
At present, the core features of the Hyperledger Fabric architecture mainly include:
- Decouples atomic sorting links and other complex processing links, eliminates network processing bottlenecks, and improves scalability;
- The logical roles of decoupled transaction processing nodes are Endorser and Committer, which can be deployed flexibly according to the load;
- The identity certificate management service has been strengthened to provide more functions as a separate Fabric CA project;
- Support multi-channel features, data between different channels are isolated from each other, improving isolation security;
- Supports pluggable architecture, including consensus, authority management, encryption and decryption, and ledger mechanism modules, and supports multiple types;
- The system chain code is introduced to realize the processing of the blockchain system, supporting programmable and third-party implementation.
The overall architecture of Hyperledger Fabric is shown in the figure below.
Fabric provides gRPC APIs for applications, and SDKs that encapsulate the APIs for application calls. Applications can access various resources in the Fabric network through the SDK, including ledgers, transactions, chain codes, events, and permission management. Application developers only need to deal with these resources, and do not need to care about how to implement them. Among them, the ledger is the core structure, which records application information, and the application records data in the ledger by initiating transactions. The logic of transaction execution is carried by chaincode. Events that occur throughout the operation of the network can be accessed by applications to trigger external processes or even other systems. Rights management is responsible for access control throughout the process. The ledger and transactions further depend on the core blockchain structure, database, consensus mechanism and other technologies; the chain code relies on the container, state machine and other technologies; the authority management uses the existing PKI system, digital certificates, encryption and decryption algorithms and many other security Technology. The bottom layer consists of multiple nodes to form a P2P network, which interacts through the gRPC channel and uses the Gossip protocol for synchronization.
The hierarchical structure improves the extensibility and pluggability of the architecture and facilitates developers to develop in modules.
Based on the functions of different links in the transaction process, Hyperledger Fabric logically decouples node roles into Endorser and Committer, so that different types of nodes can focus on processing different types of workloads. A typical transaction processing process is shown in the figure below.
In the whole transaction process, the functions of each component are mainly:
Client (App): The client application uses the SDK to deal with the Fabric network. First, the client obtains a valid identity certificate from the CA to join the application channel in the network. Before initiating a formal transaction, you need to construct a transaction proposal (Proposal) and submit it to Endorser for endorsement (through the ProcessProposal(ctx context.Context, signedProp *pb.SignedProposal)(*pb.ProposalResponse,error) interface provided by EndorserClient); the client collects After enough endorsement support (determined by the endorsement policy), you can use the endorsement to construct a legal transaction request and send it to the Orderer for sorting (through the Send(env *cb.Envelope)error interface provided by BroadcastClient) for processing. The client can also monitor the messages in the network through the event mechanism to know whether the transaction is successfully received. The main implementation code of the command line client is in the peer/chaincode directory.
Endorser node: mainly provides the ProcessProposal(ctx context.Context,signedProp *pb.SignedProposal)(*pb.ProposalResponse,error) method (the code is in the core/endorser/endorser.go file) for the client to call to complete the endorsement of the transaction proposal (Currently mostly signature) processing. After receiving the transaction proposal from the client, first check the validity and ACL permissions. If the check is passed, the transaction will be simulated and run, and the state changes caused by the transaction (recorded in the form of read and write sets, including the key and version of the read state, and the state changes caused by the transaction) are recorded. state key) to endorse and return the result to the client. Note that only some nodes in the network can assume the role of Endorser. The main code is in the core/endorser directory;
Committer node: responsible for maintaining the blockchain and ledger structure (including status DB, historical DB, index DB, etc.). The node will periodically obtain the sorted batch transaction block structure from the Orderer, and perform final checks on these transactions before placing them in the order (including transaction message structure, signature integrity, duplication, read-write set version matching, etc.). After the check is passed, execute a legal transaction, write the result into the ledger, construct a new block, and update the BlockMetadata[2] (TRANSACTIONS_FILTER) in the block to record whether the transaction is legal or not. The same physical node can run only as the Committer role, or it can act as both Endorser and Committer roles. The main implementation code is in the core/committer directory;
Orderer: Only responsible for sorting. Globally sort all legal transactions in the network, and combine a batch of sorted transactions to generate a block structure. Orderers generally do not need to deal directly with the ledger and transaction content. The main implementation code is in the orderer directory. It mainly provides two RPC methods, Broadcast(srv ab.AtomicBroadcast_BroadcastServer)error and Deliver(srv ab.AtomicBroadcast_DeliverServer)error (the code is in the orderer/server.go file);
CA: Responsible for the management (distribution, revocation, etc.) of all certificates in the network, and implement the standard PKI architecture. The main code is in a separate fabric-ca project. After the CA issues the certificate, it does not participate in the transaction process in the network itself.
Core Concepts and Components
Hyperledger Fabric adopts a modular functional design, and the overall functional module structure is shown in the figure below.
Hyperledger Fabric provides different levels of functions for different developers, which can be divided into three layers from bottom to top:
- Network layer: For system administrators. Implementing a P2P network, providing the basic capabilities of building a blockchain network at the bottom, including nodes and services representing different roles;
- Consensus mechanism and authority management: for managers of alliances and organizations. Based on the connection of the network layer, realize the consensus mechanism and authority management, and provide the basis for distributed ledgers;
- Business layer: for business application developers. Based on distributed ledgers, it supports business-related functional modules such as chain codes and transactions, and provides a higher level of application development support.
The functions and functions of the network layer related components are described below.
Network layer related components
The network layer realizes the connectivity support for the distributed ledger structure through software and hardware devices, including participating roles such as nodes, orderers, and clients, as well as supporting components such as membership management and Gossip protocol.
The concept of a node (Peer) originated from a P2P distributed network, which means a service or software that plays a certain role in the network. Node functions may be peer-to-peer or cooperative. In the Hyperledger Fabric network, Peer means each fabric-peer instance in the network responsible for accepting transaction requests and maintaining a consistent ledger. These instances may be running on bare metal, virtual machines or even containers. Nodes communicate with each other through gRPC messages. According to functional roles, Peers can include three types:
- Endorser (endorser node): responsible for checking and endorsing transaction proposals from clients;
- Committer (confirmation node): responsible for checking transaction requests, executing transactions and maintaining the blockchain and ledger structure;
- Submitter: Responsible for receiving transactions and forwarding them to the orderer. Currently, it does not appear alone.
These roles are functional divisions and are not mutually exclusive. Under normal circumstances, all nodes in the network have the Committer function; some nodes have the Endorser function; the Submitter function is often integrated in the client (SDK) for implementation.
The main data structures related to Peer nodes include PeerEndpoint and endorserClient. The former represents the access endpoint of a Peer node in the network; the latter implements the EndorserClient interface, representing the client handle connected to the Peer node, providing ProcessProposal(ctx context.Context,signedProp *pb.SignedProposal)(* for the Endorser role implementation pb.ProposalResponse, error) method access. As shown below.
Orderers, or ordering nodes, are responsible for the global ordering of incoming transactions in the network. Orderer mainly provides two interfaces, Broadcast(srv ab.AtomicBroadcast_BroadcastServer) error and Deliver(srv ab.AtomicBroadcast_DeliverServer) error. The former represents the client sending data (transactions) to the Orderer, and the latter represents the block structure obtained from the Orderer and constructed after sorting. Clients can access both interfaces using the atomicBroadcastClient structure. The atomicBroadcastClient structure is shown in the figure below, which maintains a two-way gRPC channel.
Orderer can support multiple channels. Different channels are isolated from each other, and the transaction-related information in the channel will only be sent to the peers (also based on gRPC messages) that join the channel, thereby improving privacy and security. In the current design, all transaction information will pass through the Orderer, therefore, the Orderer node must be in a reliable and credible position in the network.
From a functional point of view, the purpose of Orderer is to assign globally unique serial numbers to transactions in the network, and actually does not require specific data content related to transactions. Therefore, in order to further improve privacy, what is sent to the Orderer may not be complete transaction data, but partial information, such as the result of transaction encryption processing, or just the Hash value and Id information of the transaction. These improved designs reduce the need for orderer node reliability and security. The community has already had some similar design discussions (refer to FAB-1151: Side DB-Private Channel Data).
The client is the bridge between users and applications dealing with the blockchain network. The client mainly includes two functions:
- Operate the Fabric network: including updating the network configuration, starting and stopping nodes, etc.;
- Operate the chaincode running in the network: including installation, instantiation, initiating transactions and calling chaincode, etc.
These operations need to deal with Peer nodes and Orderer nodes. In particular, operations involving consensus, such as chain code instantiation and transaction, need to interact with the Orderer. Therefore, the client often needs to have the ability to submit. Nodes such as Peer and Orderer in the network correspondingly provide gRPC remote service access interfaces for clients to call. Currently, Hyperledger Fabric already has SDKs in multiple languages in addition to the command line-based client. These SDKs encapsulate the calls to the underlying gRPC interface and can provide more complete client and development support, including multiple implementations such as Node.Js, Python, Java, and Go.
The CA node (Fabric-CA) is responsible for managing the membership in the Fabric network. The Fabric network currently uses the digital certificate mechanism to realize identity authentication and authority control, while the CA node implements the PKI service, which is mainly responsible for the management of identity certificates, including generation and revocation. It should be noted that the CA node can issue an identity certificate in advance and send it to the corresponding member entities. These entities can access various resources in the network after deploying the certificate. In the subsequent access process, the entity does not need to make a request to the CA node again. Therefore, the processing of CA nodes is completely decoupled from the processing of transactions in the network, and will not cause performance bottlenecks.
The nodes in the Fabric network use the Gossip protocol for state synchronization and data distribution. Gossip protocol is a common protocol in the P2P field, which is used for data distribution or information exchange among multiple nodes in the network. Because of its simple design, easy implementation, and high fault tolerance, it is widely used in many distributed systems. For example, Cassandra uses it to implement cluster failure detection and load balancing. The basic idea of the Gossip protocol is very simple. The data sender randomly selects several nodes from the network and sends the data to the past; the receiver repeats this process (often only selects nodes other than the sender for propagation). This process continues, and all nodes in the network will eventually reach consensus (the time complexity is the logarithm of the total number of nodes). The direction of data transmission can be sent by the sender or pulled by the acquirer.
In the Fabric network, the node will periodically use the Gossip protocol to send the latest data of the ledger it sees, and sign and authenticate the sent message. By using this protocol, the following functions are mainly achieved:
- Detection of members in the channel: Nodes that newly join the channel can learn the information of other nodes and send an Alive message to announce that they are online; offline nodes can be sensed by other nodes after a period of time.
- Data synchronization between nodes: Multiple nodes synchronize data with each other to maintain consistency. In addition, after the leader node pulls block data from the orderer, it can also be propagated to other nodes in the channel through gossip.
Recommended reading:
This article is excerpted from the book "Blockchain Principles, Design and Application", which has been authorized by Huazhang IT of Machinery Industry Press.
About the author:
Yang Baohua, Ph.D., graduated from Tsinghua University. Chairman of Hyperledger Greater China Technical Working Group, Chief Consultant of IBM Greater China Blockchain Technology Community, Senior Researcher. He has presided over the architecture design and R&D implementation of many large-scale system platforms, and is an early researcher and practitioner of blockchain, cloud computing, big data and other technologies. He loves open source technology and has contributed to OpenStack, OpenDaylight and other open source projects. He is the core designer and developer of the Hyperledger Fabric project, and the initiator of the Cello and Fabric-SDK-Py projects. The personal homepage is https://yeasy.github.com .
