Performance Benchmarking & Optimizing Hyperledger Fabric Blockchain Platform

Transcription

1 Performance Benchmarking & Optimizing Hyperledger Fabric Blockchain Platform Parth Thakkar, Senthil Nathan N, Balaji Vishwanathan IBM Research, India 1

2 Outline Background Blockchain & Hyperledger Fabric Motivation for performance benchmarking Experimentation methodology Performance metrics Parameters which Impact the Peer Components Setup and workload Results: Impact of various configurable parameters on the performance Guidelines on parameter configuration & Bottlenecks Research opportunities Optimizations (improved the overall throughput by 16x -- from 140 tps to 2250 tps) 1. Cache to minimize crypto operations 2. Parallel validation 3. Bulk read/write 2

3 Background Blockchain & Hyperledger Fabric 3

4 Blockchain Defined Blockchain is a design pattern made famous by its use in Bitcoin. But its uses go far beyond. Blockchain can reimagine the world's most fundamental business interactions and open the door to invent new styles of digital interactions. 4

5 A Business Network Without Blockchain Organization A Organization B Bank Ledger Ledger Ledger Business-2-Business Transaction Ledger Ledger Ledger Organization C Organization D Auditor Without Blockchain: Inefficient, Expensive, Vulnerable, Conflicts 5

6 A Business Network With Blockchain Organization A Organization B Bank Ledger: Shared, replicated, permissioned Organization C Organization D Auditor With Blockchain: Consensus, Provenance, Immutability 6

7 Hyperledger Fabric: Permissioned Blockchain Platform Org. B s Client App (3) Simulation results & org B digital signature (7) Validation as per SC endorsement policy (8) MVCC Validation & Commit (2) Simulates the Tx P 1 P 0 P 2 (1) Submit the Tx (6) Delivers the Block (4) Tx s Simulation Results & Endorsers Signature (9) Tx Status (3) Simulation results & org A digital signature (6) Delivers the Block (2) Simulates the Tx Org. A s Client App Org. A (7) Validation (8) MVCC as Validation per SC endorsement & Commit policy C C C C P 3 Org. B Org. C Org. D (6) Delivers the Block Org. D s Client App (5) Consensus for creating a block with Txs Consenters (or) (7) Validation as per SC Ordering Service (8) MVCC endorsement Validation & policy Commit State DB Smart Contract Endorsing & Committing Peer Committing Peer Consenters Steps for Executing a Tx (1) Submits the Tx (2) Simulates the Tx (3) Simulation results & digital signature (4) Submit results & endorsers signatures to consenter (5) Consensus (6) Delivers the block (7) Policy validation (8) MVCC & commit (9) Send Tx s status A block is delivered when (a) the timeout occurs or (b) the block size limit is reached 7

8 Our Focus: Three Sub-Components in a Peer 1. Endorser Invoke the chaincode and simulate the transaction to collect read/write set Sign the transaction response using ESCC (endorser system chaincode code) 2. Validator: On receiving a block Validator validates all transactions in a block by invoking VSCC (validator system chaincode) Any two organization endorsement signature Or (And(A, B), And(A, C), And(A, D), And(B, C), And(B, D), And(C, D)) 3. Committer & Ledger Committer calls the ledger to commit the received block Performs MVCC validation on read/write set of each transaction Commit the block to block storage and valid transactions to state/history DB 8

9 Motivation for Performance Benchmarking 9

10 Motivation for Performance Benchmarking 1. To find out bottlenecks and gaps for improvement 2. To propose and implement optimizations to improve the performance blockchain solutions throughput requirement is between 100 tps and 20,000 tps (for RTGS) 3. To provide guidelines on parameters configurations 100s of questions on parameters configuration on Hyperledger Fabric mailing-list 4. To identify research opportunities A new research topic and a lot of potential for enhancement Increase research activities in Blockchain domain 10

11 Experimental Methodology Performance Metrics, Parameters, Setup and Workload 11

12 Performance Metrics Transaction Throughput Transaction Latency Transaction Latency 1. Endorsement Latency Client Endorser 2. Broadcast Latency Client Orderer 3. Ordering Latency Orderer Peer 4. Commit Latency Peer Client Block-Level Latency 1. VSCC Validation Latency 2. MVCC Validation Latency 3. Ledger Update Latency 12

13 Parameters which Impact Peer Components Parameters Endorsers Validator Committer & Ledger (1) Block Size (2) Endorsement Policy (3) Number of Channels (4) Resource Allocation (5) State DB Choice (CouchDB vs GolevelDB) (6) Transaction Size (number of keys read/write, size of key/value) 13

14 Experimental Methodology Perform experiments by varying values set to configurable parameters to understand 1. Impact of various load generation rate and block size on the performance. 2. Impact of endorsement policy on the performance. 3. Impact of number of channels & resource allocation on the performance. 4. Impact of transaction size and state DB choice on the performance. 14

15 Setup & Workload Setup Workload Org a MSP MSP Org b Built our own workload by surveying around 12 internal customer solutions Peer 1 Peer 2 Ordering Service OSN Peer 1 Peer 2 Kafka/Zookeeper Write-only transactions 1 KV write 3 KV writes 5 KV writes Org c Peer 1 Peer 2 MSP Datacenter Network Peer 1 Peer 2 MSP Org d Read-Write transactions 1 KV read write 3 KV reads writes 5 KV reads writes 15

16 Results: Impact of various configurable parameters on the performance Guidelines, Bottlenecks, & Research Opportunities 16

17 (1) Impact of Transaction Arrival Rate & Block Size Throughput (tps) Throughput (block size: 10) Throughput (block size: 30) Throughput (block size: 50) Throughput (block size: 100) Latency (block size: 10) Latency (block size: 30) Latency (block size: 50) Latency (block size: 100) Increased with block size Throughput Latency Saturation: 140 tps Latency (msec) Transaction Arrival Rate (tps)

18 (1) Impact of Transaction Arrival Rate & Block Size Guidelines on configuring the block size: transaction arrival rate < saturation point à lower block size transaction arrival rate >= saturation point à higher block size Bottleneck: CPU resources are under-utilized (~7% utilization) - Serial validation of transactions by VSCC Parallel validation has some challenges but could improve the performance Research opportunity: On higher transaction arrival rate, the latency increased by >10x Admission control for transactions is necessary to get lower latency very challenging in an untrusted and distributed system 18

19 (2) Impact of Endorsement Policy And/Or 1 Endorser : Or (A, B, C, D) 2 Endorsers: Or (And(A, B), And(A, C), And(A, D), And(B, C), And(B, D), And(C, D)) 3 Endorsers: Or (And(A, B, C), And(A, B, D), And(B, C, D), And(A, C, D)) 4 Endorsers: And (A, B, C, D) NOutOf 1 Endorser : 1-out-of (A, B, C, D) 2 Endorsers: 2-out-of (A, B, C, D) 3 Endorsers: 3-out-of (A, B, C, D) 4 Endorsers: 4-out-of (A, B, C, D) Guideline on configuring the endorsement policy To achieve higher performance, define endorsement policy with lesser #sub-policies and #endorsers NOutOf policy is better performant than And/Or but less powerful 19

20 (2) Impact of Endorsement Policy Bottleneck: Validation of endorsers signature against the policy Repetitive crypto operation -- usage of cache could improve the performance Research opportunity: Matching digital signature set against the endorsement policy is a NP complete problem In a 5 minutes run, 96K out of 220K crypto operations performed were useless due to greedy way of matching the signature set against the policy. An algorithm that can reduce/avoid such unnecessary crypto operations is needed. 20

21 (3) Impact of Channels & Resource Allocation With increase in the number of channels à Throughput increases due to additional parallelism Impact of Resource Allocation: Homogeneous & Heterogeneous Peers 1200 Throughput 1200 Throughput * 16 = 128 vcpus 1000 Throughput (tps) Throughput (tps) * * 32 = 196 vcpus Allocated vcpus (for all peers) (a) Homogeneous Peers, 16 Channels 0 (8,0) (4,4) (2,6) (0,8) #Peers with (2, 32) vcpus (b) Heterogeneous Peers, 16 Channels 21

22 (3) Impact of Channels & Resource Allocation Guideline on configuring resources for channels Dedicated vcpus per channel à for the maximum performance & avoid interference Homogeneous resource allocation is important means the proportion as per demand and not mean the same amount of resources. For e.g., If Peer 1 receives 1000 tps, it might require 2 vcpus If Peer 2 receives 500 tps, it might require only 1 vcpus Research opportunity Each channel would receive different load at different period Need for efficient resource sharing among channels Dynamic resource provisioning to ensure homogeneous resource allocation as per the demand 22

23 (4) Impact of State DB choice & Transaction Complexities Conducted experiments with different transactions complexities for both GolevelDB & CouchDB Refer to the paper for results Research opportunity: The usage of DB such as GoLevelDB and CouchDB, without snapshot isolation level, results in the whole database lock -- not good to achieve higher throughput DB does not provide read & write set per transaction -- need to enhance DB Need to make DB more aware of blockchain operations and aid blockchain transactions to improve performance 23

24 Fault Tolerance Peer failure does not affect the performance during non-overloaded scenario App can collect endorsement from other available peers application dependent Node rejoining after a failure need to sync up with other peers Fetch missing blocks, validate and commit old transactions During overloaded scenario, rejoining peer could not sync up Rate(new blocks added) = Rate(old blocks fetched by the rejoining peer) = peek rate Research opportunity: protocol for rejoining peer s sync up so that with any block commit rate, peer should be able to sync Adding a new peer is even worse -- need to fetch blocks from the beginning of the time, validate & commit 24

25 Proposed Optimizations & Performance Improvement 1. Introduce crypto cache to avoid repetitive crypto operations 3X improvement tps to 520 tps 2. Parallelly validate multiple transactions in a block in VSCC validation phase 7X improvement tps to 980 tps 3. Bulk read/write during MVCC Validation and Commit 2.5X improvement tps to 115 tps for CouchD Performance Improvement from Hyperledger Fabric v1.0 to Fabric v1.1 Combining all three optimization resulted in 16X improvement tps to 2250 tps 25

26 Conclusion Conducted a comprehensive performance benchmarking to 1. Provide guidelines on configuring parameters 2. Improve the performance by 16X 3. Identity research opportunity Future work -- to reach 20,000 tps to support RTGS systems Sharding Distributed validation 26

27 Thank You! Questions? 27