Deep Dive: Sharding

Increasing the number of users on-chain requires faster execution times, higher security, and cheaper transaction fees. Sharding is one solution that claims to scale blockchains to process up to a million transactions per second.

What is sharding? How does it work?

Read this article to find out!

In non-sharded blockchains, each node is responsible for processing every transaction and storing the entire state. Even if there are 1000 nodes (?? nodes = ?? security), the TPS of the network is limited to the processing power of one node. This creates a bottleneck.

Sharding is one of the most viable solutions to this problem. It focuses on splitting the blockchain into smaller pieces called "shards." And, each shard can process a portion of the total transaction processing load, increasing the network's overall processing power.

For instance, To get 10x performance, split 1000 nodes into 10 shards with100 nodes each.

The nodes in the shards execute transactions and store states limited to that shard. Increasing the chance of Sybil attacks (by making it easy to execute) as the validator set is split into shards.

One challenge in this approach is that you can scale up to a certain capacity depending on the beacon chain capacity. Although adding infinite shards is possible, you also have to scale the beacon chain as well to adjust that load. The solution is Quadratic Sharding.

So, what is Quadratic Sharding?

If nodes include the beacon chain, then each shard will be able to process 4x more transactions and the beacon chain will be able to maintain 4x more shards. Throughput across the system will increase by 4 × 4 = 16.

A node is responsible for three things:

• Processing transactions
• Storing the transactions data
• Relaying transactions to other nodes in the network.

Sharding partitions these three operations using the network, state and transaction sharding.

Network sharding: It handles the way the nodes are grouped into shards. This can be used to optimize communication since message propagation inside a shard is much faster than propagation to the entire network.

State Sharding: The state of the blockchain is split across shards. Nodes in a particular shard only process and relay transactions limited to the state of that shard. This linearly reduces storage & compute requirements and introduces problems of data availability and cross-shard transactions.

Transaction Sharding: Sharding of the compute power of a blockchain network. This is easy because every node has the whole state of the blockchain. The only need is to ensure that shards don’t update the same state concurrently. Zilliqa blockchain uses this.

Certain challenges arise due to sharding designs:

1. Cross-chain communication
2. State Validity/Data Validity
3. Data Availability

But, how are these tackled? Let's take a look below!

Cross-shard communication problem:?

If smart contracts are hosted on different shards & they want to send a message from one shard to another, it's not possible for nodes in one shard to complete these transactions.

There are two approaches to solving cross-shard communication.

1. Synchronous: The blocks in both shards relating to the cross-shard transactions are produced in sync, and validators collaborate on executing such transactions. e.g., Merge blocks technique.
2. Asynchronous:?This approach executes transactions asynchronously where it's possible for one shard to verify transactions in another shard. e.g., the Beacon chain technique used in Polkadot.

State validity problem:

A shard cannot confirm the current shard state they are interacting with without downloading and validating the entire shard history. This problem doesn't exist in non-sharded blockchains.

The image above represents such cross-shard transactions with a shard containing an invalid block.

Validator rotation:

Rotating validators on a shard reduces the probability of malicious validator behaviour, i.e., colluding to produce a fork or invalid block. Allowing cross-shard transactions only between neighbouring shards. These nodes can verify blocks in their shards & neighbours as well.

Fishermen:

Whenever a block header is published to the beacon chain, there is a challenging period for honest validators to prove that a block is invalid.

Succinct proofs:

Cryptographic proofs could prove a computation's validity, i.e., block production based on transactions.

There are two types of nodes:

Full nodes:?Download all blocks and validate every transaction.

Light nodes:?Download only block headers.

Validators in the shard are effectively?full nodes?for that shard, while other participants in the system, including the beacon chain, operate as?light nodes.

Data Availability problem:

Light nodes don't know when a new block is produced in a shard because all the data in that block has already been published. The dilemma is if block producers don't release all data in the block. Light nodes can't detect malicious transactions that are hidden in that block.

Proof-of-Custody:?Have notaries in shards on a rotating basis, downloading the blocks and confirming that they can do so.

Erasure codes:?Light nodes verify data availability of block by downloading random chunks and verifying using Merkle root for validity. Ethereum and Polkadot have solutions based on this.

Benefits of sharding

• High TPS:?Limited nodes execute transactions rather than all nodes.
• Reduced compute & storage requirement:?State size and processing load of shards is much less than the whole blockchain.
• Decentralization:?Node requirements decrease, making it easy to run and maintain.

Multiple blockchain networks have introduced sharding to their scaling plans. Some of them are as follows:

~ Zilliqa

~ Polkadot

~ Near Protocol

~ Ethereum 2.0

~ Harmony

~ Elrond

~ Shardeum

Now let's look at some of their design choices.

In Zilliqa, shards handle processing but not storage i.e., the transactions are processed parallelly across shards while every shard holds a full blockchain state. The same state is shared across all shards, so different techniques are used to make sure that shards don’t update the same state concurrently.

Near sharding has a single blockchain design rather than beacon chain and shard chains. Each block contains all shard transactions and can change all shard states. Nodes only maintain the state for their shard. Transactions are split into chunks, and one chunk represents one shard.

Polkadot uses Parachains (parallelizable chains) to achieve distributed database sharding. These Parachains attach to the security a relay chain which provides this rather than providing their own. A key feature of parachains is that the computations they perform are inherently independent.

Ethereum sharding design is based on shard chains and beacon chains. It uses a receipt mechanism for cross-chain communication. There is also a proposition to create 64 shards to scale Ethereum up to 100K transactions per second. However, it is yet to launch after the most awaited merge happens.

Harmony has sharding on three dimensions, i.e., state, transactions, and network. It has a beacon and shard chain design. Crosslinks are used to post valid block headers to the beacon chain. It allows cross-shard transactions as well.

Elrond uses a novel state sharding technique called Adaptive state sharding. It combines state + transaction + network sharding and allows to dynamically change the number of shards based on nodes available and network usage to achieve efficient sharding.

Shardeum is an EVM-based sharded blockchain that uses dynamic state sharding. It has unique features such as transaction-level consensus, which eliminates atomic processing complexities faced by block-level consensus designs.

Sharding has huge potential to scale blockchains to serve the growing demand. It's currently in development or has a distant roadmap across an array of chains. The first blockchain to solve this complex design will likely attract users and accrue higher value.

Follow GravityX Capital for more in-depth research and insights into all things Web3, DeFi and crypto, today!