Comprehensive analysis of EIP-4844: A small step for sharding, a big step for Ethereum scalability.

Original Title: "Foresight Ventures: The First Step of Ethereum Towards Full Sharding, Comprehensive Analysis of EIP-4844"

Original Author: Yihan Xu@Foresight Ventures

Overview

In this report, you can learn:

EIP-4844；Proto-danksharding；Danksharding；KZG Commitment；KZG Ceremony；Dynamic state sharding；…

Recently, the KZG Ceremony with excessive community participation has been widely discussed. What exactly is this ceremony doing?

Simply put, KZG Ceremony is an initialization step that implements EIP-4844, which is a preliminary version of the Ethereum sharding process that cannot be bypassed.

One, Sharding: A Long-term Solution for Ethereum's Scalability

Sharding expands from the perspective of data availability, while rollup mainly expands from the execution level, together alleviating the congestion problem of the main network; I think sharding may be an attempt to break the impossible triangle of blockchain. The trend chart of Ethereum block size below can also illustrate the necessity of expanding at the data level from another perspective. From the genesis block, Ethereum has been iterating rapidly from the underlying architecture to the upper-layer applications, but the average block size is still around 90Kb, and there has been no qualitative breakthrough at the highest point. Although Rollup cuts in from the calculation layer to solve the congestion problem of Ethereum, its performance is still limited by the data storage capacity of layer 1.

Due to the complexity of the implementation process and security considerations, the Ethereum development team has divided sharding into multiple stages, including the recently mentioned proto-danksharding and danksharding. The entire process will be an update that lasts for several years. Under the current data storage mode of Ethereum, only a few specific high-configured machines can join the network as nodes. After full sharding, Ethereum no longer needs every node to save all data, which reduces the cost of mainnet data storage while enhancing security (reducing the threshold for becoming a node, further increasing decentralization, and reducing the risk of attack).

 

二、EIP-4844: Short-term high return, a simplified version of Sharding

EIP-4844 = Proto-Danksharding；

Proto comes from Ethereum researchers.

Due to the high complexity of implementing danksharding (which will be analyzed in the next section), the development cycle is at least several years. Therefore, proto-danksharding is a scaling solution for Ethereum before the implementation of danksharding, mainly implementing the transaction format and precompile design in danksharding.

1. Proto-danksharding Overview

Proto-Danksharding mainly introduces a new transaction type, which is the blob-carrying transaction. With this, Rollup data is transmitted to layer 1 in the form of blobs at a lower cost and is not permanently stored. At the same time, blobs are much larger than current calldata, which can better support high TPS on Rollup.

About blob:

Each transaction can have a maximum of 2 blobs attached;

Each block in an ideal state contains 8 blobs, which is approximately 1MB, and can contain up to 16 blobs, which is approximately 2MB.

Blob does not need to be permanently stored as a history log like calldata;

Compared to danksharding, nodes still need to verify the complete DA.

2. Blob-carrying transaction interpretation

Attach a blob (data unit) to the transaction.

作用
translates to

Function
in English.

Referencing the diagram provided by Vitalik in the proposal, Data blob is similar to current calldata, and rollup can upload transaction, proof, and other data through blob to layer 1 to ensure data availability.

Cost

The original intention of designing Data blob is to support high-throughput transactions in rollup. Compared with calldata of the same size (using on-chain storage), the cost of blob will be greatly reduced (no need for permanent storage). Therefore, the gas cost of maintaining data availability in rollup will be significantly reduced compared to before.

Capacity

Each blob is approximately 125kB in size (fact: the current average block size is only ~90kB).

3. Blob-carrying transaction value and challenges

Value

You can think of a blob as a cache, and the transaction data submitted by Rollup exists in the form of a cache. This reduces the requirements for storage hardware, provides additional data expansion for Ethereum, and reduces gas costs.

Challenge: Hardware performance requirements for Ethereum nodes.

Ethereum's current average block size is only about 90kB, but one blob is about 125kB.

According to the design of EIP-4844, each slot is approximately 1 MB under normal circumstances. Therefore, the amount of data added each year is:

1 MB/block * 5 block/min * 43200 min/month * 12 month/year = 2.47 TB per year

The amount of data added each year far exceeds the total amount of data on Ethereum, making this storage solution clearly inefficient.

From the perspective of short-term expansion, since each node still needs to store the full historical data, while implementing EIP-4844, automatic deletion of blobs outside a certain time window (the specific limit time has not been finally determined, it may be 1 month or 1 year) will be carried out.

From the long-term perspective of sharding, implementing EIP-4444 means that nodes do not need to store full historical data, but only need to store data after a specific time reference point based on history expiry.

These two solutions alleviate the tradeoff of blob-carrying transactions on storage space to varying degrees.

4. KZG Commitment

KZG Commitment is a polynomial commitment scheme adopted in EIP-4844.

解析 KZG commitment

Analysis of KZG commitment

KZG is an abbreviation of the surnames of the authors Aniket Kate, Gregory M. Zaverucha, and Ian Goldberg. They published a paper in 2010 titled "Constant-Size Commitments to Polynomials and Their Applications" which introduced a polynomial commitment scheme. This scheme has been widely used in plonk-style zk-SNARK protocols.

Refer to the diagram in Dankrad Feist's speech, KZG root is similar to Merkle root, but the difference is that KZG root commits to a polynomial, where all positions are on this polynomial. In the context of proto-danksharding, KZG root commits to a bunch of data, where any of the data can be verified to belong to the whole.

This is also why KZG commitment is more compatible with the implementation of DAS in terms of compatibility.

The process of KZG commitment is as follows:

Prover: Provides proof, calculates the commitment of data. The prover cannot change the given polynomial, and the commitment used for proof is only valid for the current polynomial. Verifier: Receives the commitment value sent by the prover and verifies that the prover has provided a valid proof.

KZG Commitment Advantages

I believe that the main considerations for this are cost and security, which can be summarized but not limited to the following points:

Cost

KZG commitment has the characteristics of fast verification, relatively lower complexity, and simplicity.

There is no need to submit additional proof, making the cost lower and saving bandwidth.

The Point evaluation precompile required for data access can achieve lower costs.

Security

Assuming a failure occurs, it will only affect the data in the blob corresponding to the commitment, without any other far-reaching effects.

More compatible

Looking at the overall solution of sharding, KZG commitment is compatible with the DAS scheme, avoiding the cost of duplicate development. 5. KZG Ceremony (trusted setup).

KZG Ceremony provides trust setup for KZG Commitment, which has attracted over 20,000 participants and contributions, making it the largest trust setup in history.

Recently, the KZG Ceremony, which has seen a surge in community participation, is providing trust setup for the KZG commitment adopted by EIP-4844.

KZG Ceremony Procedure

Refer to Vitalik's flowchart, anyone can contribute a secret as participants and mix it with the previous result to generate a new result, and so on, to obtain the final SRS through nesting, and assist in completing the trust setup of KZG commitment.

trust setup

EIP-4844 adopts a common multi-participant trust setup, namely powers-of-tau, which follows a 1-of-N trusted model. Regardless of how many people participate in the generating setup process, as long as one person does not disclose their generation method, the trusted initialization is valid.

Necessity

KZG commitment's trust setup can be simply understood as generating a parameter that is required for each execution of a cryptographic protocol, similar to the trusted initialization required for zk-SNARK.

When providing proof, Prover uses KZG commitment C = f(s)g1, where f is the evaluation function and s is the final secret obtained from KZG trusted setup.

It can be seen that the final secret is the core parameter for generating polynomial commitments, and as a trusted process for obtaining this core parameter, the KZG Ceremony is very important for the implementation of the entire sharding. 6. Changes brought by EIP-4844.

Rollup

Referring to the schematic diagram provided on ethresear, rollup needs to include the versioned hash of state delta and KZG commitment in calldata for submission (zk-rollup also needs to submit zk proof).

It can be observed that the difference is that calldata only contains small amounts of data, such as state delta and KZG commitment, while the transaction batch containing a large amount of transaction data is placed in the blob.

Effectively reducing costs, putting data in calldata is expensive; reducing the occupation of block space.

Security

Data availability: Blobs are stored on the beacon chain, equivalent to layer 1 security; historical data: nodes will only store blobs for a period of time, and permanent data storage relies on layer 2 rollups, so security depends on rollups.

Cost

Proto-Danksharding has introduced a new transaction type. The addition of low-cost data format blob will undoubtedly further reduce the cost of rollup. Depending on the progress of on-chain applications and implementation, the cost of optimized rollup may be reduced by x10 or even x50.

Meanwhile, EIP-4844 introduced blob fee;

Gas and blob will have adjustable gas price and limit respectively; the charging unit for blob remains as gas, and the gas amount will vary with traffic to maintain the target of an average of 8 blobs per block (limiting the additional amount of data).

Precompile Implementation

The data in Blob cannot be directly accessed, and EVM can only obtain the commitment of the data blob. Therefore, rollup needs to provide precompiles to verify the validity of the commitment.

Below we analyze two precompile algorithms mentioned in EIP-4844.

Point evaluation precompile (for more information on the mathematical principles, refer to Vitalik's analysis).

Proving that multiple commitments point to the same data; mainly for zk-rollup, which requires 2 types of commitments: 1. KZG commitment; 2. zk-rollup's own commitment. For optimistic rollup, most have adopted the mechanism of multi-round fraud proof, which requires less data for the final round fraud proof. Therefore, using point evaluation precompile can achieve lower costs.

Blob verification precompile

Prove that versioned hash and blob are validly corresponding; optimistic rollup needs to access the full data when submitting fraud proof, so first verify that the versioned hash and blob are legal before conducting fraud proof verification.

 

Three, Danksharding: An important step towards full sharding

The name Danksharding comes from Ethereum researcher Dankrad Feist.

1. Expansion: Further expanding the cache space of Rollup

After the implementation of proto-danksharding, the introduction of a new transaction format with blob has enabled each block to gain an additional 1MB of cache space on average. With Danksharking implementation, each block gains an additional 16MB, with a maximum allowance of 32MB.

2. Data availability: more efficient storage and verification strategies

Compared to proto-danksharding, Danksharking only requires Ethereum nodes to sample blobs instead of downloading the full data. The sampled data is distributed among all nodes in the network and can be combined to form a complete data blob.

DAS: Efficient Sampling Inspection

By using erasure coding, the probability of discovering lost original data is increased, making it easier for all network nodes to detect data loss when downloading partial data, thereby improving security. 3. Security: remains basically unchanged.

Due to considerations of data availability, backup, and sampling inspection, each node no longer stores complete historical data. From the perspective of security, the security of the system has changed from at least one node storing complete data to multiple nodes storing partial data and ultimately restoring complete data.

Although single-point dependency may seem more secure than multi-point dependency at first glance, the number of nodes in the Ethereum network is sufficient to ensure the need for data backup, so security will not change significantly.

4. New Challenge: Higher Requirements for Block Builders

Although validators do not need to download and retain full historical data, the requirements for bandwidth and storage hardware have been eased. However, block creators still need to upload blob data containing full transaction data.

Here is a brief introduction to PBS (proposer/builder separation). Please refer to Dankrad's application diagram of PBS in the Danksharding scheme:

The role of the block producer will be split into proposer and builder. Initially proposed for anti-MEV design, in the design of danksharding, this is to reduce the bandwidth requirements during block creation.

Four, Other Sharding Solutions: Shardereum's Dynamic Sharding

Shardeum is an EVM-compatible layer 1 public chain. Unlike Ethereum's static sharding solution, Shardeum improves the underlying scalability and security through dynamic state sharding, while naturally ensuring a high degree of decentralization.

1. Dynamic state sharding

Advantages

Dynamic state sharding brings the most intuitive advantage of linear scaling, where nodes connected to the network can be efficiently grouped by the sharding algorithm and respond quickly, improving the TPS of the blockchain network. In the design of dynamic state sharding, each node covers a different range of addresses, and the coverage range is redundantly designed to ensure efficient sharding and security.

Implementation within the scenario

Ignoring the complexity of the ecosystem, from the perspective of the two technical implementations, the difficulty of dynamic state sharding is greater than that of static sharding. It can be seen that the technical team of Shardeum has a deep accumulation in the technical aspect of sharding. The team's previous research and development on Shardus technology also made great contributions to the underlying technology of this public chain, and even demonstrated the linear scaling brought by dynamic state sharding very well in the early stages of the project.

2. Shardeum Summary

Product

Divide the nodes into different groups, following the divide and conquer approach, to distribute the workload of computation and storage, allowing for higher levels of parallel processing. Therefore, more nodes can be accommodated, further improving the throughput and decentralization of the public chain.

Team

Experienced in the market, with excellent narrative ability and deep research on dynamic sharding.

Technology

Designed a suitable sharding scheme (dynamic state sharding) and consensus design (Proof of Stake + Proof of Quorum) for our own scenario, with the first goal of improving scalability and ensuring a higher degree of decentralization and security.

Progress

The betanet will be launched on 2023-02-02, which is worth paying attention to.

Five, Thoughts and Prospects on Sharding 

Sharding is a long-term solution for Ethereum's scalability, and it is also a valuable and far-reaching long-term plan. In the process of implementing sharding, all existing solutions may be continuously iterated, including the proto-danksharding and danksharding mentioned now, which are worth continuous attention;

It is important to have a good understanding of the sharding direction, but the technical solutions adopted in each proposal for implementing full sharding (PBS, DAS, multidimensional fee market) are also worth paying attention to. I believe that many excellent teams and projects will emerge in this regard.

Sharding is a general term for a scaling technology, but there is not only one specific implementation. It is important to recognize that different public chains will have sharding solutions that are suitable for their own scenarios. For example, some designs in danksharding are only suitable for the Ethereum network, and the security tradeoff requires a large number of nodes to offset it.

The reasonable combination of Sharding and other expansion solutions can improve scalability with the principle of 1+1>2. Currently, Danksharding is not an independent expansion solution, but complements other implementations in the Ethereum ecosystem. For example, Danksharding and rollup work together to achieve better scalability for Ethereum.   

Original article link

Welcome to join the official BlockBeats community:

Telegram Subscription Group: https://t.me/theblockbeats

Telegram Discussion Group: https://t.me/BlockBeats_App

Official Twitter Account: https://twitter.com/BlockBeatsAsia