A Consensus-Based Load-Balancing Algorithm for Sharded Blockchains (original) (raw)

An Adaptive Load-Balancing Approach for Sharded Blockchains

International Journal of Computing and Digital Systems, 2024

The emergence of blockchains as a way to organize and protect user data across the internet has come with some concerns, mainly how to deal with the issue of scalability while still maintaining the security standards as well as the decentralized nature inherent to blockchains. Many different implementations will be elicited in this work, resort to the concept of sharding, consisting of partitioning a blockchain into smaller clusters which would, with proper load-balancing, drastically improve the performance of the overall chain. First, we will explore the existing literature on the subject as well as related algorithms. Then provide a detailed explanation of the functioning of the existing algorithms, both centralized and decentralized, as well as the one proposed in this paper. Next, we will then elicit the settings and conditions of the simulation environment, both in terms of data collection and preparation. Finally, we will provide the results obtained as well as a comparative analysis of the tested algorithms and give an overview of possible future endeavors in the advancement of load-balancing algorithms in respect to sharded Blockchains.

Distributed load-balancing for account-based sharded blockchains

International Journal of Web Information Systems, 2022

Purpose Sharding of blockchains consists of partitioning a blockchain network into several sub-networks called “shards,” each shard processing and storing disjoint sets of transactions in parallel. Sharding has recently been applied to public blockchains to improve scalability through parallelism. The throughput of sharded blockchain is optimized when the workload among the shards is approximately the same. The purpose of this paper is to investigate the problem of balancing workload of account-based blockchains such as Ethereum. Design/methodology/approach Two known consensus-based distributed load-balancing algorithms have been adapted to sharded blockchains. These algorithms migrate accounts across shards to balance transaction processing times. Two methods to predict transaction processing times are proposed. Findings The authors identify some challenging aspects for solving the load-balancing problem in sharded blockchains. Experiments conducted with Ethereum transactions show ...

Towards Scaling Blockchain Systems via Sharding

Proceedings of the 2019 International Conference on Management of Data, 2019

Existing blockchain systems scale poorly because of their distributed consensus protocols. Current attempts at improving blockchain scalability are limited to cryptocurrency. Scaling blockchain systems under general workloads (i.e., noncryptocurrency applications) remains an open question. In this work, we take a principled approach to apply sharding, which is a well-studied and proven technique to scale out databases, to blockchain systems in order to improve their transaction throughput at scale. This is challenging, however, due to the fundamental difference in failure models between databases and blockchain. To achieve our goal, we first enhance the performance of Byzantine consensus protocols, by doing so we improve individual shards' throughput. Next, we design an efficient shard formation protocol that leverages a trusted random beacon to securely assign nodes into shards. We rely on trusted hardware, namely Intel SGX, to achieve high performance for both consensus and shard formation protocol. Third, we design a general distributed transaction protocol that ensures safety and liveness even when transaction coordinators are malicious. Finally, we conduct an extensive evaluation of our design both on a local cluster and on Google Cloud Platform. The results show that our consensus and shard formation protocols outperform state-of-the-art solutions at scale. More importantly, our sharded blockchain reaches a high throughput that can handle Visa-level workloads, and is the largest ever reported in a realistic environment.

Ostraka: Secure Blockchain Scaling by Node Sharding

arXiv (Cornell University), 2019

Cryptocurrencies, implemented with blockchain protocols, promise to become a global payment system if they can overcome performance limitations. Rapidly advancing architectures improve on latency and throughput, but most require all participating servers to process all transactions. Several recent works propose to shard the system, such that each machine would only process a subset of the transactions. However, we identify a denial-of-service attack that is exposed by these solutions-an attacker can generate transactions that would overload a single shard, thus delaying processing in the entire system. Moreover, we show that in common scenarios, these protocols require most node operators to process almost all blockchain transactions. We present Ostraka, a blockchain node architecture that shards (parallelizes) the nodes themselves. We prove that replacing a unified node with an Ostraka node does not affect the security of the underlying consensus mechanism. We evaluate analytically and experimentally block propagation and processing in various settings. Ostraka allows nodes in the network to scale, without costly coordination. In our experiments, Ostraka nodes transaction processing rate grows linearly with the addition of resources.

Scaling Blockchains: A Comprehensive Survey

IEEE Access

Blockchain (e.g., Bitcoin and Ethereum) has drawn much attention and has been widelydeployed in recent years. However, blockchain scalability is emerging as a challenging issue. This paper outlines the existing solutions to blockchain scalability, which can be classified into two categories: first layer and second layer solutions. First layer solutions propose modifications to the blockchain (i.e., changing the blockchain structure, such as block size) while second layer solutions propose mechanisms that are implemented outside of the blockchain. In particular, we focus on sharding as a promising first layer solution to the scalability issue; the basic idea behind sharding is to divide the blockchain network into multiple committees, each processing a separate set of transactions. More specifically, (a) we propose a taxonomy based on committee formation and intra-committee consensus; and (b) we compare the main existing sharding-based blockchain protocols. We also present a performance-based comparative analysis (i.e., throughput and latency), of the advantages, and disadvantages in existing scalability solutions.

Trust-Based Shard Distribution Scheme for Fault-Tolerant Shard Blockchain Networks

IEEE Access, 2019

Blockchains guarantee data integrity through consensus of distributed ledgers based on multiple validation nodes called miners. For this reason, any blockchain system can be critically disabled by a malicious attack from a majority of the nodes (e.g., 51% attack). These attacks are more likely to succeed as the number of nodes required for consensus is smaller. Recently, as blockchains are becoming too large (making them difficult to store, send, receive, and manage), sharding is being considered as a technology to help improve the transaction throughput and scalability of blockchains. Sharding distributes block validators to disjoint sets to process transactions in parallel. Therefore, the number of validators of each shard group is smaller, which makes shard-based blockchains more vulnerable to 51% attacks than blockchains that do not use sharding. To solve this problem, this paper proposes a trust-based shard distribution (TBSD) scheme that assigns potential malicious nodes in the network to different shards, preventing malicious nodes from gaining a dominating influence on the consensus of a single shard. TBSD uses a trust-based shard distribution scheme to prevent malicious miners from gathering in on one shard by integration of a trust management system and genetic algorithm (GA). First, the trust of all nodes is computed based on the previous consensus result. Then, a GA is used to compute the shard distribution set to prevent collusion of malicious miners. The performance evaluation shows that the proposed TBSD scheme results in a shard distribution with a higher level of fairness than existing schemes, which provides an improved level of protection against malicious attacks.

OptChain: Optimal Transactions Placement for Scalable Blockchain Sharding

2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), 2019

A major challenge in blockchain sharding protocols is that more than 95% transactions are cross-shard. Not only those cross-shard transactions degrade the system throughput but also double the confirmation time, and exhaust an already scarce network bandwidth. Are cross-shard transactions imminent for sharding schemes? In this paper, we propose a new sharding paradigm, called OptChain, in which cross-shard transactions are minimized, resulting in almost twice faster confirmation time and throughput. By treating transactions as a stream of nodes in an online graph, OptChain utilizes a lightweight and on-the-fly transaction placement method to group both related and soon-related transactions into the same shards. At the same time, OptChain maintains a temporal balance among shards to guarantee the high parallelism. Our comprehensive and largescale simulation using Oversim P2P library confirms a significant boost in performance with up to 10 folds reduction in cross-shard transactions, more than twice reduction in confirmation time, and 50% increase in throughput. When combined with Omniledger sharding protocol, OptChain delivers a 6000 transactions per second throughput with 10.5s confirmation time.

DEXON: A Highly Scalable, Decentralized DAG-Based Consensus Algorithm

IACR Cryptol. ePrint Arch., 2018

A blockchain system is a replicated state machine that must be fault tolerant. When designing a blockchain system, there is usually a trade-off between decentralization, scalability, and security. In this paper, we propose a novel blockchain system, DEXON, which achieves high scalability while remaining decentralized and robust in the real-world environment. We have two main contributions. First, we present a highly scalable sharding framework for blockchain. This framework takes an arbitrary number of single chains and transforms them into the \textit{blocklattice} data structure, enabling \textit{high scalability} and \textit{low transaction confirmation latency} with asymptotically optimal communication overhead. Second, we propose a single-chain protocol based on our novel verifiable random function and a new Byzantine agreement that achieves high decentralization and low latency.

OmniLedger: A Secure, Scale-Out, Decentralized Ledger via Sharding

2018 IEEE Symposium on Security and Privacy (SP)

Designing a secure permissionless distributed ledger (blockchain) that performs on par with centralized payment processors, such as Visa, is a challenging task. Most existing distributed ledgers are unable to scale-out, i.e., to grow their total processing capacity with the number of validators; and those that do, compromise security or decentralization. We present Om-niLedger, a novel scale-out distributed ledger that preserves longterm security under permissionless operation. It ensures security and correctness by using a bias-resistant public-randomness protocol for choosing large, statistically representative shards that process transactions, and by introducing an efficient crossshard commit protocol that atomically handles transactions affecting multiple shards. OmniLedger also optimizes performance via parallel intra-shard transaction processing, ledger pruning via collectively-signed state blocks, and low-latency "trust-butverify" validation for low-value transactions. An evaluation of our experimental prototype shows that OmniLedger's throughput scales linearly in the number of active validators, supporting Visa-level workloads and beyond, while confirming typical transactions in under two seconds.

A Random and Scalable Blockchain Consensus Mechanism

Asian journal of convergence in technology, 2022

The central piece of blockchain technologies is the consensus algorithm. The consensus is reached via consensus algorithms in the distributed network of the blockchain. The consensus becomes stronger if nearly all the nodes in the blockchain network take part in building the blockchain. Even if some nodes misbehave or malfunction, the consensus should not break down then. Since blockchain networks are distributed, even if some regions of the network or some nodes of the network leave, the blockchain network should not malfunction and it should be consistent. Therefore, the consistency of the blockchain transactions should be assured by all the nodes or nearly all the nodes, or most of the nodes. In other words, the consensus should be the issue of all the nodes. In this work, a novel consensus algorithm is presented to diffuse the mission of building the blockchain to all the nodes. In other words, the consensus should be the issue of all the nodes. In this work, a novel consensus algorithm is presented to diffuse the mission of building the blockchain to all the nodes. The algorithm increases the randomness of nodes and enforces the blockchain network to be more decentralized. Randomness is realized by employing the power of cryptography, especially by using public keys as a characteristic for miners, which are also called signers. Moreover, since the algorithm is realized with a few operations, it contributes to the scalability of the blockchain. Furthermore, digital signatures improve the security level and consistency of the blockchain.