【Paper Reading】Performance Modeling of Hyperledger Fabric (Permissioned Blockchain Network)

Summary

Hyperledger Fabric (HLF) is an open-source implementation of a distributed ledger platform for running smart contracts in a modular architecture. In this paper, we propose a performance model for Hyperledger Fabric v1.0+ using Stochastic Reward Network (SRN). From our detailed model, we can calculate throughput, utilization, and average queue length on each node, as well as key processing stages within the node. To validate our model, we set up an HLF network in our lab and run workloads using Hyperledger Caliper. From our analysis results, we find that the time to complete the endorsement process is significantly affected by the number of peers and strategies such as and(). Performance bottlenecks in ordering services and ledger writes can be mitigated with larger block sizes, although latency will increase. For committing peers, the transaction validation check (using Validating System Chaincode (VSCC)) is a time-consuming step, but its performance impact can be easily mitigated because it can be parallelized. However, its performance is crucial, as it absorbs the shock of the exploding blocks as they arrive. We also analyzed various hypothetical scenarios, such as peers processing transactions in a pipeline, and multiple endorsers per organization.

I. Introduction

According to the National Institute of Standards and Technology (NIST) [1], “a blockchain is an immutable digital ledger system implemented in a distributed fashion (i.e., without a central repository), usually without a central authority.” Blockchain Network Enables trusted parties to send transactions peer-to-peer in a verifiable manner without the need for a trusted intermediary. It allows parties to settle transactions more quickly, thereby speeding up the flow of goods and services [2]. Therefore, cryptocurrencies such as Bitcoin are applications that run on a blockchain network.

The Hyperledger Project is an open source collaborative project hosted by the Linux Foundation to advance blockchain technology for the enterprise. In our research, we focus on Hyperledger Fabric version v1.0+ [3] (referred to as Fabric V1 or simply Fabric). It is currently deployed in more than 400 proof-of-concept and production distributed ledger systems across different industries and use cases [4]. As opposed to a public blockchain network (like Bitcoin or Ethereum) where anyone can join the network, HLF is a permissioned blockchain network where participants know and recognize each other but don't fully trust each other. Therefore, organizations can benefit from distributed ledger technology (DLT) without the need for cryptocurrencies [5].

As HLF projects grow and mature, it is necessary to model the complex interactions between nodes performing different functions. Such models provide a quantitative framework to help compare different configurations and make design trade-off decisions. In this paper, we propose a performance model of Fabric V1 that uses a Stochastic Reward Network (SRN) to compute the throughput, utilization, and average queue length of various peers as a function of various system parameters. Just as multiple subsystems of the HLF are "pluggable", we ensure that the corresponding submodels are also pluggable.
With this model in place, we can ask various what-if questions, such as

1. How do the throughput, utilization, and average queue length per node change as the transaction arrival rate increases?
2. Would overall system throughput increase if nodes validated transactions in the pipeline instead of batching them?
3. How much performance gain can we get if each organization has multiple endorsers?

The research contributions of this paper are as follows.

1. A comprehensive performance model of the Fabric V1 blockchain network. For Fabric's unique blockchain network architecture, it captures the key steps performed by each subsystem and the interactions between them.

2. Analysis of key what-if scenarios of concern to system developers and practitioners.

3. Validate the model using a multi-node experimental setup.

Two System Description - HYPERLEDGER FABRIC V1

Fabric V1 introduced the Execution-Order-Verification architecture [3], which is a fundamental shift from the traditional order execution design, followed by other blockchain platforms, including the preview version of HLF (called v0.6) [6 ]. The goal is to separate transaction execution (via smart contracts (SC)) from transaction ordering. Compared to traditional state machine replication approaches [7], the architecture offers better scalability, new trust assumptions for transaction verification, support for non-deterministic SCs, and modular consensus implementations [3], [ 5].

Figure 1. Transaction flow on Hyperledger Fabric V1 with 2 nodes and an ordering service running a single channel

In Fabric V1, nodes execute transactions and maintain distributed ledgers. A sorter sorts all transactions in the network, proposes new blocks and seeks consensus. A collection of subscribers forms a subscription service. By default, all nodes are committers, receiving ordered state updates in transaction blocks from the ordering service and maintaining the ledger. Upon receiving a new block, peers verify transactions, commit changes to their local copy of the ledger and append to the block on the blockchain. Nodes can take on the additional responsibility of endorsing transactions, hence the name endorsers. An endorser simulates a transaction by executing a smart contract (SC) (called chaincode in HLF) and attaching its cryptographic signature (called an endorsement) before sending the result back to the client. Note that a single peer can be both an endorser and a committer.

Fabric maintains a global state for all nodes using a versioned key-value store (KVS) and ledger. KVS shows the latest state of the transaction, which can be read by the chaincode using the get() operation, and state updates are proposed by the transaction using the put() operation. The version number is monotonically increasing. KVS is partitioned through chaincode. The ledger is an ordered chain of hashes of all transaction blocks, thus providing a verifiable history of all state changes. Both the key-value store and the ledger are privately updated by each peer.

In a Fabric network, a subset of nodes that want to conduct business transactions privately can form a channel, maintaining separate partitions for key-value storage and ledgers. In our analysis, we consider single-channel Fabric networks.

To execute a transaction, a client requires signatures from all peers as defined by the endorsement policy. Endorsement policies are reflections of business logic and can be described using arbitrary Boolean logic using AND, OR, and k/n (see Section VII-A for more details).

Let's consider the Fabric network in Figure 1. In this example, each peer represents an organization (for example, Org0 and Org1). The transaction process in Fabric V1 is described as follows

1. The client sends a transaction proposal (TxProposal) to the node defined by the endorsement policy (all running the same smart contract, such as SC1), including clientID, payload and transactionID, and the client's encrypted signature transaction header. The payload contains chaincodeID, action and input parameters.

2. Each peer simulates transaction execution by invoking the corresponding SC (SC1 in this case) with user input for the local key-value state. SCs run in containers isolated from peers. After endorser simulation, read-set is generated to represent the key version number read by SC, and write-set represents the key-value pair updated by SC.

3. Each peer sends an endorsement message to the client that includes the result, read-set, write-set, and metadata, where the metadata includes transactionID, endorserID, and endorser signature. This message is signed by the peer's cryptographic signature.

4. The client waits for the peer's endorsement until it satisfies the transaction's endorsement policy and verifies that the received results are consistent. The client then prepares the transaction containing the payload and endorsement set received from the endorsing peers and sends it to the ordering service. This operation is asynchronous and the client will be notified by its peer when the transaction is successfully committed.

5. After a few seconds (called the block timeout) or a certain number of pending transactions (called the block size), the ordering service creates a block of pending transactions, maintaining order by timestamp. The block is appended with a cryptographic signature based on the block header, which is then broadcast to all peers on the same channel.

6. When a node receives a block of transactions, it evaluates transaction endorsements in parallel (using Validating System Chaincode (VSCC)) according to its endorsement policy. Failed flags are invalid. Next, for each valid transaction, it performs Multiversion Concurrency Control (MVCC) [8], [9] (called read-write check in [3]), which means it serially verifies whether the read-set version Matches the current version on the KVS (assuming previous transactions have committed). The validity of the transaction is captured as a bitmask and appended to the block before it is appended to the local ledger. Finally, all write-sets are written to the local KVS to complete the state transition. The node notifies the client of the success or failure of the transaction.

Therefore, every transaction in Fabric V1 goes through three stages: endorsement, ordering, and verification.

Three performance indicators

In this paper, we model the performance of Fabric at the DLT system level. Hyperledger Performance and Scalability Work Group1 recently published the first version of the Performance Metrics document [10] in an attempt to provide clear and unambiguous performance metrics applicable to different DLT platforms. We borrow and refine definitions for our models and analyses.

Transaction throughput: Transaction throughput is the rate at which a blockchain network submits valid transactions within a defined period of time. It is expressed in transactions per second (tps). For single-channel fabric networks, we consider measurements at a single peer in both our model and experimental analysis. Note that a transaction here corresponds to a chaincode "invocation" rather than a "query". Since only valid transactions are considered, the throughput is the same as the actual throughput.

Transaction Latency: Transaction latency is the time between submitting a transaction and confirming that the transaction has been committed over the network. Unlike lottery-based consensus protocols like Bitcoin or Ethereum, where transaction finality is probabilistic, Fabric's consensus process leads to deterministic finality and is therefore simple to define. In our model and analysis, we examine transaction confirmations by individual nodes. The end-to-end delay includes three delays: endorsement delay, ordering delay, and commit delay [9].

Queue Length: A node's queue length is the number of jobs waiting to be served or being served at that node. In our model, we compute the average queue length for each processing stage of the ordering service, each endorser, and committer.

Utilization: The utilization of a node is the percentage of time the node is busy. In our model, we calculate the utilization of each endorsing node and each processing stage of the submitting node. For multithreaded operations, utilization corresponds to the average utilization of all logical processors.

Four SRN models of HYPERLEDGER FABRIC V1

In this section, we present our Fabric performance model using a modeling form of Stochastic Petri Nets called Stochastic Reward Networks (SRN) [11]. This formalism allows for a concise specification and automatic generation/solution of an underlying (stochastic) process that captures the performance behavior of a blockchain network system. Furthermore, using SRN also allows us to study different scenarios by easily adding or removing system details. SRN has been successfully used to model different computer/communication systems from a performance perspective [12]. We solve the model using the stochastic Petri net package (SPNP) [13].

Figure 2 shows the SRN model for a single-channel Fabric network, which consists of a client, two endorsing nodes (AND()), and a node running validation logic. Transaction requests follow a Poisson arrival process with rate λC. maximum. The pending request limit is no. The number of threads in the workload generator (20 in our case). The client prepares the endorsement request and sends it to the endorsing peers (eg peer 0 and peer 1) (converts TPr, including transmission time). Peers approve transactions (converted to TEn0 and TEn1 respectively). When the client receives responses from both peers, the client sends the endorsed transaction to the ordering service (transforming TTx), indicated by the tokens deposited into the POS. After a block size of pending transactions (denoted by M), a block of transactions is created and delivered to the submitting peer (transition TOS). The submitting peer first executes all transactions in the VSCC verification block in parallel, limited by the number of logical processors (cores/vCPUs) in the peer (CPUmax). It then performs MVCC validation (conversion TMVCC) on all transactions serially. Finally, all transactions in the block are written to the local copy of the ledger (transition TLedger). Both TMVCC and TLedger transitions are captured at the block level. For reasons discussed in Section VII-B, we do not consider block timeouts in this model. Note that we implicitly assume that all transactions have the same complexity and are independent of each other. In summary, the three phases of a Fabric transaction can be seen in the SRN model.

We obtain metrics from our model as follows. The throughput of the transaction phase corresponds to the rate of the corresponding transition, using the function rate() in SPNP [14]. For example, TLedger's conversion rate represents the block throughput of the system (multiply by M to get transaction throughput). The utilization of a transaction phase is calculated by enabling the probability of the corresponding transition in the SRN using the function enabled(). For transformations with a function-dependent marking rate (such as TVSCC), the average utilization of all logical processors is calculated using the reward function. The average queue length of the transaction phase can be obtained by the number of tokens in the corresponding phase through the function mark(). For example, the average number of tokens in a POS indicates the average queue length for an ordering service.

Figure 3. Hyperledger Fabric V1 network setup

Empirical Analysis of Five-Model Parameterization

We parameterize our model using data collected from our lab's Fabric network setup. Unlike the studies summarized in Section IX-B, the goal of this work is not to benchmark performance by stress-loading the system; instead, we measure from a Fabric network affected by real traffic patterns. In this section we provide details of the tools used, the network setup and the methods we used for data collection and analysis.

A. Network settings

The deployed network is shown in Figure 3. Each node is started as a Docker container and then connected to the network using Docker Swarm2. Although nodes and orderers can run locally on physical/virtual machines, Docker container networking is the recommended way to deploy Fabric networking [15]. Each organization (peer and ca) corresponds to a container running on an independent physical node ('Org0', 'Org1'). Peers execute chaincode in a separate container (called CC). All containers corresponding to the ordering service run on a single physical node (the "ordering service"). It includes 1 sorting service node (OSN), 4 Kafka brokers, and 3 ZooKeeper nodes. Hyperledger Caliper is deployed on a single physical node with multiple client threads interacting with the locally installed Fabric Node.js SDK3. We provide detailed step 4 for setting up a Fabric network here. Note that we consider a network where each organization has a peer, so we ignore the impact of the gossip protocol on network performance.

The physical machines corresponding to Org0, Org1, and Caliper have 4 CPUs (1-way 4-core) (Intel Xeon 2.2GHz) and 12GB memory, and the running order service has 16 CPUs (2-way 4-core 2 hyperthreading) Intel Xeon 2.4 GHz and 32GB RAM. Each machine was running Ubuntu 16.04 LTS on a 7200 rpm hard drive with Fabric version v1.15 installed. All physical machines are connected with 1 Gbps switches. All nodes are synchronized using the NTP (Network Time Protocol) service so that transaction latencies can be measured across nodes. Communication between all nodes is configured to use Transport Layer Security (TLS).

To execute the workload, we use Hyperledger Caliper6 which was recently approved as a Hyperledger project. It is a benchmark execution platform that enables users to consistently measure the performance of different DLT platforms. An advantage of using this tool is that it can handle complex workflows performed by clients (using the Fabric SDK), including handling event notifications from peers. To generate traffic following a Poisson arrival process, we implemented a new rate control feature in Caliper. A current limitation of Caliper is that it only supports interaction with one OSN node.

B. Test the application

For our performance tests, we leveraged the simple chaincode provided by the Caliper tool and extended it according to our needs. This application maintains the user's account balance. It can perform two functions. The function "open" checks if the account exists, and if not, creates a new account and assigns it the account balance. Therefore, it performs one read, one write operation to the key-value store. The "Transfer" function allows the transfer of funds from one account to another. So it does two reads, two writes. We run the "open" transaction for a few minutes before running the "transfer" transaction. Load the key-value store. The authors in [9] also adopted a similar approach to differentiate workloads by the number of read and write operations. For both functions, the input account number is chosen at random; thus there is little dependency between successive transactions; thus transactions appear to never fail MVCC validation. This way, we can easily generate a workload of all valid transactions at a high rate7.

c. to measure

We measure the time spent performing key steps in the transaction lifecycle by analyzing caliper output files, peer logs, and ordering logs. Figure 4 shows a transaction sequence diagram with snapshots of where timestamps were captured and log entries in peers/orderers. We transition these important log entries from DEBUG mode to INFO mode. To measure the average queue length for each transaction lifecycle phase, we added additional log entries to capture when new transactions enter and leave that phase. No weighted time average. The number of transactions/block gives the average queue length for that phase [16]. We also increased the timestamp resolution to microseconds. We measured the overhead of additional logging for a setting of block size 500 and λC = 100, and observed only a 0.64% increase in average transaction latency and a 0.48% increase in 75%ile latency. Our Fabric source code changes can be found here8.

D. Measuring motion

We run Caliper with 20 client threads for a test duration of 240 seconds, ensuring that transaction arrivals follow a Poisson arrival process. We trim the first and last 20 seconds. The test run acts as an acceleration and deceleration phase. To validate our model against different configuration settings, we varied three parameters: a) client tx. Arrival rate (λC), b) block size, c) transaction type ('open' and 'transfer'). Due to our hardware limitations, we cannot vary the amount of CPU used to authenticate peers. We plan to pursue it in future work. For a given block size, we keep λC high enough so that most blocks are created due to block size rather than timeouts (see Section VII-B). Restart all docker instances between each test run.

E. Model parameters

We perform test runs on each of the above sets of configuration parameter values ​​and collect log files from which we derive the desired output metrics. Depending on λC, each test run contains 7k to 30k transactions, resulting in 7k to 30k samples of transaction-level parameter values ​​and 150 to 900 block-level samples of parameter values. To ensure consistent parameter values ​​across groups, we performed an analysis of variance (ANOVA) F-test [17] to see if the means were statistically significantly different, followed by a multiple comparison procedure using Tukey's honest significance test [17]], [ 18]. Therefore, we derive parameter values ​​from large datasets.

In our current work, we assume an exponential distribution of trigger times for all transitions. In our future work, we plan to perform distribution analysis and choose the most suitable distribution for each transformation. We are confident in our selection due to the good results of our validation. With this choice, the underlying stochastic process is a continuous-time Markov chain (CTMC), so we can analyze our model using an analytical numerical solution (Section VII).

The parameter values ​​for "open" transactions are summarized in Table I. Due to space limitations, we only show the results of "open" transactions in this article. A detailed empirical analysis is shared in [19]. Transitions TMVCC and TLedger are measured at the block level because the measurement at the transaction level is too small. The time to process a message at the client (TPr) and the time to prepare a block (TOS) also includes the transfer time to the peer. Regarding TOS, although the current Kafka-based sorting service does not have the concept of consensus, when the Byzantine Fault Tolerant (BFT) consensus protocol is implemented in the future, this conversion will also include the consensus time.

Six overall system analysis and model validation

Let us analyze the overall system model (Figure 2) of a network using the AND() endorsement policy with CPUmax = 4 for VSCC verification and various block sizes. Unfortunately, the underlying Markov model has a huge space, and we cannot solve the complete system model using an analytical-numerical solution. Therefore, we adopt a simulation method using SPNP.

To validate our model, we calculated the average queue lengths of the ordering service (OS) and key processing stages within nodes (VSCC validation, MVCC validation, and ledger write (LWrite)) and compared them with empirical results. We chose "Average Queue Length" because it is reliably measured and provides intuitive insight into system performance.

We validate our results for different client arrival rates and different block sizes. From the results in Fig. 5, we observe that all measured results (labeled (m)) are comparable to the model results under different customer arrival rates. The results are similar for other block sizes. Therefore, we consider our model validated. Unfortunately, we cannot reliably measure queue lengths for endorsing peers from peer logs.

However, since the results matched the other four measured points, we utilized the results of our model. From our model results, let us visualize the utilization and average queue length per transaction phase (Fig. 6) as λC increases. We also calculated utilization for client-side processing and transfers to the ordering service. We found that the transmission from the client to the ordering service was the performance bottleneck, and the utilization rate rose sharply, followed by the endorsement nodes. The queue length for the ordering service/ledger writes is expected to be large as it waits for block-sized transactions before generating/committing a block.

In our lab setup, under high client arrival rates, transactions tended to time out (perhaps due to queuing delays) and fail, even though the peer or ordering service was not particularly busy. It also explains that the transfer time of endorsed transactions to the ordering service is a performance bottleneck. Based on our measurements, it is difficult to say whether the delay is due to network transfer time or to the internal queuing of the ordering service. Assuming this delay is not an issue in another setup (replacing TTx with immediate transitions), let's calculate the maximum. The possible throughput in this network. In our model, we continuously increase customer transaction arrival rates until utilization at any stage of the transaction reaches 90%. We also consider the scenario where each organization has multiple endorsers. (Enmax), transform TEn0, TEn0 using label-dependent emissivity. From the results in Figure 7, we see the maximum value. Throughput increases significantly with block size, although average latency increases (not shown). When Enmax = 1, the bottleneck is the endorsement nodes. When Enmax = 4, the bottleneck is ledger writing. Hence the max. If there are multiple endorsers per organization, system throughput increases considerably (especially for larger block sizes like 120).

Seven model analysis

In the next three subsections, we analyze the subsystems corresponding to each transaction phase. All analyzes in this section are done using Analytical Numerical Solution (SPNP).

A. Endorsement Policy

Client nodes are responsible for seeking endorsement of their proposed transactions, thereby satisfying the endorsement policy. Endorsement policies are monotonic logical expressions that evaluate to TRUE or FALSE. For example. , the endorsement policy OR(Org1, Org2) means that an endorsement from any peer of Org1 or Org2 is sufficient. Endorsement policies can be expressed as any combination of AND, OR, and k/n expressions, such as OR(Org1, Org2) AND (2/3 of Org3, Org4, Org5).

Figure 8. Generic SRN model for capturing AND/OR endorsement policies between two nodes

The endorsement part of the model in Figure 2 represents two peers in an AND() strategy; thus, the client waits for a response from both parties before forwarding the transaction to the ordering service. This model can be easily extended to represent AND() strategies for more nodes. To model the OR() and k/n() strategies, we use variable cardinality arcs in the SRN (called viarc()) to refresh tokens from where they were not triggered, as indicated by the Z-symbolized arcs in Figure 8. Show. In addition, a guard function is used to convert Iwait immediately, which is written similarly to an endorsement logic expression. Thus, complex endorsement policies can be easily captured by extending the network shown in Figure 8 (without the dotted line) and inserting the entire system network in Figure 2.

The endorsement process adds significant latency to transactions. First, chaincode executes in separate Docker containers on each node, adding a reasonable performance overhead. Second, the client needs to wait for endorsement responses from multiple nodes to satisfy the endorsement policy. Let us analyze the average time to complete endorsement for different endorsement strategies in Table II. We assume that the endorsement time of each node is exponentially distributed with a rate of 0.308 per millisecond.

In the current version of the Fabric Node SDK, the client waits for replies from all endorsing peers (or times out) before preparing the endorsement message for the ordering service. Therefore, we cannot validate our system-wide model with different endorsement strategies. The upcoming feature 9 will address this limitation.

B. Ordering service


The ordering service receives endorsed transactions from clients, orders them and creates a block of transactions based on block timeout or block size. Given the arrival rate (λE) of endorsed transactions, a block size, and a block timeout, we want to evaluate the probability of a block being produced due to the block size or block timeout.

Let us consider the SRN model in Figure 9. The number of tokens in the POS represents pending transactions. Tokens are held in PSize or PTimeout, depending on whether the block was created due to a size limit or a timeout. Since the block timeout is deterministic, we approximate it using the Erlang distribution [20], [21], which appears as a single network. The TOtrigger is enabled immediately when at least one token is in place for POS. We consider a 5-phase Erlang distribution, where each phase fires at a rate of 5/(blocktimeout).

Figure 10 shows a contour plot of the probability of creating a block due to a timeout condition of timeout = 1.0 seconds. For smaller block sizes, even at low arrival rates (λE), blocks are generated due to size constraints. As the block size increases, a higher endorsement transaction arrival rate (λE) is required to skip the timeout condition in our model. As a result, we can skip the extra logic to account for timeout conditions, significantly reducing the size and complexity of the overall system model (Figure 2).

C. Block Validation and Commitment

Let's analyze the commit node performance using the SRN model in Figure 11. Let us assume that block arrivals (from the ordering service) follow a Poisson arrival process with rate λB, each of size M. To constrain the size of the underlying Markov model, we consider the maximum value. Queue length of two blocks per stage.

Let's consider two scenarios of interest to Fabric designers.

1. Burst arrival of blocks: Let us consider a scenario where submitting peers face a burst of blocks, consisting of periods of high and low average block arrival rates. To capture this, we model the input arrival process using a Markov modulated Poisson process (MMPP) [20], [21]. Let us consider the extended SRN model with block size M in Fig. 11 (shown in dashed box). The average time fraction µstart µstart+µstop for blocks to arrive at a high rate (say 6 blocks per second) and a low rate (say 2.25 blocks per second) when there are tokens in PMMPP. The average boot time for burst mode is eight seconds. ( 1 µstart ) for an average duration of 2 seconds. ( 1 µstop ). To compare the model's results with non-bursty arrival rates, we consider the same average arrival rate. Therefore, λB = λB-high ∗ µstart µstart+µstop + λB-low ∗ µstop µstart+µstop.

We compare the MMPP arrival results starting in burst mode (MMPP(1)) with those of a model with a Poisson arrival rate λB = 3 blocks per second. From Figure 12, we find that the utilization of the Ledger write and VSCC verify stages jumps in the first few seconds as MMPP(1) arrives. There are significant jumps in the average queue length for both the MVCC check and Ledger write phases. Overall, the VSCC verification phase seems to absorb the shock of burst arrivals well. However, utilization and average queue length during the ledger write phase are most affected and thus critical from a performance perspective. Remember that chunks are delivered in order from the ordering service. Therefore, we need to ensure that the input queue is large enough and that submitting nodes can process blocks fast enough. Re-requesting blocks again will slow down the peer significantly.

2) Pipeline model: In order to improve the performance of submitting nodes, the authors proposed a pipeline architecture in [3], [9], where each transaction goes through various stages in the pipeline, rather than the current transaction through the architectural blocks in each stage. We assume that such a system will have only one logical processor for VSCC verification. We assume the same parameter value for VSCC verification, divided by the block size of MVCC, LWrite. We calculate the maximum value. Throughput for this pipeline model for various block sizes (Figure 13). Surprisingly, max. Throughput is comparable to the regular model with CPUmax = 1. However, the average queue length is slightly larger for MVCC validations and smaller for ledger writes (not shown). The rest of the indicators are comparable.
Let us use a modified version of the model in Figures 11 and 13 to compute the transaction block latency by removing the transition Tblock-arr and considering the M initial tokens in PVSCC and the output location Done from the transition TLedger. When M tokens are deposited into Done, it signifies the completion of a block. We calculated the average block latency for the regular model and the pipeline model for different block sizes (Fig. 14). As expected, there is a slight improvement in average block latency in the pipelined model compared to the regular model with CPUmax=1. Latency decreased further as we added more CPUs to the regular model.

We perform a sensitivity analysis of the pipeline model against the average block latency of the VSCC verification rate. We found the max. Throughput results are comparable to VSCC validated regular models for CPUmax = 2, 4, etc. Overall, pipelining improves block latency slightly, but not max. Throughput and other performance metrics compared to traditional architectures.

eight discussion

A. The Enormity of Stochastic Models

We use a high-level formalism, SRN, to model the complex interactions of Fabric networks. The downside of this approach is that the resulting underlying stochastic model is very large. We cannot generate the underlying state space for the full-system model because it is infinite. We therefore adopted a simulation approach (see Section VI). For the commit-to-peer model (see Section VII-C), we consider a maximum value. Queue length of two blocks per stage. This truncates the state space of the underlying model, which we address using an analytical numerical solution. Interestingly, we found that the model state space size increases linearly with the block size.

Scale issues limit our ability to answer questions about the scalability of Fabric networks. In our future work, we will consider employing layered approaches and fixed-point iterative solution techniques to alleviate this problem [12].

B. Limitations of our model

A limitation of our model is that we cannot calculate latency for a specific transaction throughput. From our model, we can estimate the average latency of transaction blocks (see Section VII-C2). However, this does not take into account any queuing delays at individual nodes while the system "loads". Along these lines, the model also fails to capture transaction failures due to timeouts (mainly due to queuing delays). In our future work, we plan to compute the response time distribution of transactions using a state-space modeling approach, as described in [22].

C. Threats to effectiveness

One threat to the validity of our results is that we did not use a custom VSCC. Custom VSCCs allow users to define additional (business) logic to validate transactions. Since this is in addition to the endorsement policy signature verification done during VSCC verification, we think a higher mean parameter could easily capture this. Note that the author in [3] used a custom VSCC instead of the author in [9]. Another threat is that our nodes are connected in a LAN setup rather than a wide area network (WAN) as expected in the real world. In our future work, we plan to replicate our results by running Fabric on cloud services such as Amazon Web Services (AWS).

Nine related work

A. Performance Modeling of Blockchain Networks

Decker and Wattenhofer [23] proposed a simple model to calculate the stale block generation rate in the Bitcoin network, which considers Bitcoin's block generation and block propagation processes. Gervais et al. [24] model and analyze the security and performance aspects of proof-of-work (PoW)-based blockchain networks. For performance modeling, they developed a simulator that simulates the block mining and propagation process as a function of block interval, block size, and block propagation mechanism. Its output metric is the stale block rate. Papadis et al. [25] developed a stochastic model for PoW-based blockchain networks to calculate the block growth rate and stale block rate as a function of propagation delay and computing nodes' hash power. In our previous work, we modeled and analyzed the Practical Byzantine Fault Tolerant (PBFT) consensus process for Fabric v0.6 [26]. Since the architecture of Fabric V1 has evolved significantly compared to version v0.6 [5], the models discussed in this paper do not apply to version v0.6.

B. Performance evaluation of Hyperledger Fabric

Thakkar et al. extensively studied the performance of Hyperledger Fabric v1.0 in [9], where they varied five tunable parameters: block size, endorsement policy, number of channels, number of vCPUs for peers, and key-value database ( GoLevelDB and CouchDB). Their research led to three optimizations for Fabric: parallelization of VSCC validation, caching for Membership Service Providers (MSPs), and batch read/write for CouchDB, all included in the v1.1 release at [3] and our work. [3] presents Fabric V1 in great detail. They tested Fabric v1.1 with an application called Fabcoin, using a data model similar to Bitcoin-style UTXO10. Unlike [3], [9] who run nodes on virtual machines in cloud data centers, we run nodes on physical machines. Sousa et al. [27] integrated BFTSMaRt [28] as an ordering service for Fabric v1.0 and performed performance analysis only for the ordering service they deployed in a geographically distributed setup in AWS data centers.

Auh Dinh et al. [29] performed performance tests on Fabric v0.6 (along with Ethereum and Parity) on a 48-node cluster using macro benchmarks based on YCSB and Smallbank benchmarks. They also developed separate micro-benchmarks to gain insight into the performance characteristics of various layers such as consensus, data model, and execution engine.

10 Conclusions and future work

In this paper, we develop stochastic models for a popular distributed ledger platform called Hyperledger Fabric. Every transaction in the Fabric network goes through three stages: endorsement, ordering, and verification. We analyzed the entire system and the subsystems corresponding to each transaction stage in detail. We collected data from a Fabric setup running real workloads to parameterize and validate our models. Our model provides a quantitative framework that helps system architects evaluate performance against different system configurations and make design trade-off decisions.

For incoming blocks, due to the commit node verifying transactions in parallel (VSCC), there is a significant performance gain if the commit node is deployed on a system with a lot of CPU. It can significantly reduce the queue length during VSCC validation and allow the system to absorb the shock of bursty block arrivals. However, this is not a performance bottleneck for submitting nodes, since ledger writes are most utilized. We also analyze two cases: each organization has multiple endorsing peers. and validator pipeline architecture. Parallelization of transaction endorsements can significantly reduce the endorsement queue length and increase the maximum value. If the endorsement process is the performance bottleneck, the throughput of the system. The pipeline architecture provided about a 1% improvement in average block validation latency, but provided no other performance benefits. In future work, we will consider a hierarchical modeling approach to overcome model generation and solution bulk.