BLOCKBENCH: A Framework for Analyzing Private Blockchains

By Tien Tuan Anh Dinh, Ji Wang, Gang Chen, Rui Liu, Beng Chin Ooi, Kian-Lee Tan. SIGMOD Conference, 2017

913 citations recorded by the source corpus.

Blockchain technologies are taking the world by storm. Public blockchains, such as Bitcoin and Ethereum, enable secure peer-to-peer applications like crypto-currency or smart contracts. Their security and performance are well studied. This paper concerns recent private blockchain systems designed with stronger security (trust) assumption and performance requirement. These systems target and aim to disrupt applications which have so far been implemented on top of database systems, for example banking, finance and trading applications. Multiple platforms for private blockchains are being actively developed and fine tuned. However, there is a clear lack of a systematic framework with which different systems can be analyzed and compared against each other. Such a framework can be used to assess blockchains' viability as another distributed data processing platform, while helping developers to identify bottlenecks and accordingly improve their platforms. In this paper, we first describe BLOCKBENCH, the first evaluation framework for analyzing private blockchains. It serves as a fair means of comparison for different platforms and enables deeper understanding of different system design choices. Any private blockchain can be integrated to BLOCKBENCH via simple APIs and benchmarked against workloads that are based on real and synthetic smart contracts. BLOCKBENCH measures overall and component-wise performance in terms of throughput, latency, scalability and fault-tolerance. Next, we use BLOCKBENCH to conduct comprehensive evaluation of three major private blockchains: Ethereum, Parity and Hyperledger Fabric. The results demonstrate that these systems are still far from displacing current database systems in traditional data processing workloads. Furthermore, there are gaps in performance among the three systems which are attributed to the design choices at different layers of the blockchain's software stack. We have released BLOCKBENCH for public use.

Subfields: Computer Networks and Communications, Information Systems

View Paper: https://www.semanticscholar.org/paper/97b4375a71e98fb5b4628b3cf9bf80c4e006e891

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

Citations in the corpus

A systematic literature review of blockchain cyber security (2020, neutral): “In fact, there is a large community who are interested in blockchain analysis (evidenced by the popularity of open-source tools such as bitcoin-abe [41] or BlockBench [42] for instance), so academic researchers should actively involve the community in the development, validation, and maintenance of ”
SoK: Consensus in the Age of Blockchains (2017, background): “2) Evaluation: BLOCKBENCH [55] is a framework for evaluating the security and performance of private blockchains.”
Blockchain's adoption in IoT: The challenges, and a way forward (2019, methodology): “However, PoA is also being used by Ethereum test network Kovan [81].”
Performance Benchmarking and Optimizing Hyperledger Fabric Blockchain Platform (2018, background): “BlockBench [22] was one of the first to look at permissioned blockchain.”
Towards Scaling Blockchain Systems via Sharding (2018, methodology): “ applied to blockchains. Instead, blockchains rely on Byzantine Fault Tolerance (BFT) consensus protocols which have 1 arXiv:1804.00399v4 [cs.DC] 12 Mar 2019 been shown to be a scalability bottleneck [21]. Thus, the first challenge is to scale BFT consensus protocols. Second, in a distributed databa”
Everything You Wanted to Know About the Blockchain: Its Promise, Components, Processes, and Problems (2018, neutral): “, [14]–[16]).”
A Vademecum on Blockchain Technologies: When, Which, and How (2019, background): “The results published in [75], [128] show that Hyperledger Fabric outperforms Ethereum in terms of evaluation metrics such as execution time, latency and throughput.”
PBFT vs Proof-of-Authority: Applying the CAP Theorem to Permissioned Blockchain (2018, background): “Differently from PBFT, PoA requires less message exchanges hence provides better performance [10].”
Combining Graph Neural Networks With Expert Knowledge for Smart Contract Vulnerability Detection (2021, background): “, Bitcoin and smart contract, are taking the world by storm [1], [2], [3], [4], [5], [6].”
Security Services Using Blockchains: A State of the Art Survey (2018, neutral): “However, the performance tests done in [25] show that both platforms still suffer from some aspects of scalability issues.”
Blockchain technology: Is it hype or real in the construction industry? (2020, neutral): “Blockchain architecture is illustrated in Figure 5 as five layers, namely Network layer, Data layer, Consensus layer, Execution layer and Application layer [36, 37].”
Blockchain for Internet of Things: A Survey (2019, methodology): “The scalability of blockchains can be measured by the throughput of transactions per second against the number of IoT nodes and the number of concurrent workloads [25], [113].”
Performance Analysis of Hyperledger Fabric Platforms (2018, background): “Another recent experiment studied the scalability of three blockchain platforms,Hyperledger, Ethereum, andParity [49, 50].”
Performance Analysis of Private Blockchain Platforms in Varying Workloads (2017, background): “A recent to-be-published paper explores the performance of private blockchain platforms [1] and cited performance evaluation as a research opportunity.”
Blockchain for AI: Review and Open Research Challenges (2019, background): “Unlike public blockchains, the private blockchains are designed as permissioned systems where users and participants in the systems are pre-approved for read/write operations and are always known within the network [87].”