header-langage
简体中文
繁體中文
English
Tiếng Việt
한국어
日本語
ภาษาไทย
Türkçe
Scan to Download the APP

Comprehensive analysis of EIP-4844: A small step for sharding, a big step for Ethereum scalability.

2023-01-30 11:52
Read this article in 27 Minutes
总结 AI summary
View the summary 收起
Original Title: "Foresight Ventures: The First Step of Ethereum Towards Full Sharding, Comprehensive Analysis of EIP-4844"
Original Author: Yihan Xu@Foresight Ventures


Overview


In this report, you can learn:


EIP-4844;Proto-danksharding;Danksharding;KZG Commitment;KZG Ceremony;Dynamic state sharding;…


全面解析 EIP-4844:Sharding 的一小步,以太坊扩容的一大步


Recently, the KZG Ceremony with excessive community participation has been widely discussed. What exactly is this ceremony doing?


Simply put, KZG Ceremony is an initialization step that implements EIP-4844, which is a preliminary version of the Ethereum sharding process that cannot be bypassed.


One, Sharding: A Long-term Solution for Ethereum's Scalability


Sharding expands from the perspective of data availability, while rollup mainly expands from the execution level, together alleviating the congestion problem of the main network; I think sharding may be an attempt to break the impossible triangle of blockchain. The trend chart of Ethereum block size below can also illustrate the necessity of expanding at the data level from another perspective. From the genesis block, Ethereum has been iterating rapidly from the underlying architecture to the upper-layer applications, but the average block size is still around 90Kb, and there has been no qualitative breakthrough at the highest point. Although Rollup cuts in from the calculation layer to solve the congestion problem of Ethereum, its performance is still limited by the data storage capacity of layer 1.


全面解析 EIP-4844:Sharding 的一小步,以太坊扩容的一大步


Due to the complexity of the implementation process and security considerations, the Ethereum development team has divided sharding into multiple stages, including the recently mentioned proto-danksharding and danksharding. The entire process will be an update that lasts for several years. Under the current data storage mode of Ethereum, only a few specific high-configured machines can join the network as nodes. After full sharding, Ethereum no longer needs every node to save all data, which reduces the cost of mainnet data storage while enhancing security (reducing the threshold for becoming a node, further increasing decentralization, and reducing the risk of attack).

 

二、EIP-4844: Short-term high return, a simplified version of Sharding


EIP-4844 = Proto-Danksharding;


Proto comes from Ethereum researchers.


Due to the high complexity of implementing danksharding (which will be analyzed in the next section), the development cycle is at least several years. Therefore, proto-danksharding is a scaling solution for Ethereum before the implementation of danksharding, mainly implementing the transaction format and precompile design in danksharding.


全面解析 EIP-4844:Sharding 的一小步,以太坊扩容的一大步


1. Proto-danksharding Overview


Proto-Danksharding mainly introduces a new transaction type, which is the blob-carrying transaction. With this, Rollup data is transmitted to layer 1 in the form of blobs at a lower cost and is not permanently stored. At the same time, blobs are much larger than current calldata, which can better support high TPS on Rollup.


About blob:


Each transaction can have a maximum of 2 blobs attached;

Each block in an ideal state contains 8 blobs, which is approximately 1MB, and can contain up to 16 blobs, which is approximately 2MB.

Blob does not need to be permanently stored as a history log like calldata;

Compared to danksharding, nodes still need to verify the complete DA.


2. Blob-carrying transaction interpretation


Attach a blob (data unit) to the transaction.


全面解析 EIP-4844:Sharding 的一小步,以太坊扩容的一大步


作用
translates to

Function
in English.


Referencing the diagram provided by Vitalik in the proposal, Data blob is similar to current calldata, and rollup can upload transaction, proof, and other data through blob to layer 1 to ensure data availability.


Cost


The original intention of designing Data blob is to support high-throughput transactions in rollup. Compared with calldata of the same size (using on-chain storage), the cost of blob will be greatly reduced (no need for permanent storage). Therefore, the gas cost of maintaining data availability in rollup will be significantly reduced compared to before.


Capacity


Each blob is approximately 125kB in size (fact: the current average block size is only ~90kB).


3. Blob-carrying transaction value and challenges


Value


You can think of a blob as a cache, and the transaction data submitted by Rollup exists in the form of a cache. This reduces the requirements for storage hardware, provides additional data expansion for Ethereum, and reduces gas costs.


Challenge: Hardware performance requirements for Ethereum nodes.


Ethereum's current average block size is only about 90kB, but one blob is about 125kB.


According to the design of EIP-4844, each slot is approximately 1 MB under normal circumstances. Therefore, the amount of data added each year is:


1 MB/block * 5 block/min * 43200 min/month * 12 month/year = 2.47 TB per year


The amount of data added each year far exceeds the total amount of data on Ethereum, making this storage solution clearly inefficient.



From the perspective of short-term expansion, since each node still needs to store the full historical data, while implementing EIP-4844, automatic deletion of blobs outside a certain time window (the specific limit time has not been finally determined, it may be 1 month or 1 year) will be carried out.


From the long-term perspective of sharding, implementing EIP-4444 means that nodes do not need to store full historical data, but only need to store data after a specific time reference point based on history expiry.


These two solutions alleviate the tradeoff of blob-carrying transactions on storage space to varying degrees.


4. KZG Commitment


KZG Commitment is a polynomial commitment scheme adopted in EIP-4844.


解析 KZG commitment

Analysis of KZG commitment


KZG is an abbreviation of the surnames of the authors Aniket Kate, Gregory M. Zaverucha, and Ian Goldberg. They published a paper in 2010 titled "Constant-Size Commitments to Polynomials and Their Applications" which introduced a polynomial commitment scheme. This scheme has been widely used in plonk-style zk-SNARK protocols.


全面解析 EIP-4844:Sharding 的一小步,以太坊扩容的一大步


Refer to the diagram in Dankrad Feist's speech, KZG root is similar to Merkle root, but the difference is that KZG root commits to a polynomial, where all positions are on this polynomial. In the context of proto-danksharding, KZG root commits to a bunch of data, where any of the data can be verified to belong to the whole.


This is also why KZG commitment is more compatible with the implementation of DAS in terms of compatibility.


The process of KZG commitment is as follows:


Prover: Provides proof, calculates the commitment of data. The prover cannot change the given polynomial, and the commitment used for proof is only valid for the current polynomial. Verifier: Receives the commitment value sent by the prover and verifies that the prover has provided a valid proof.


KZG Commitment Advantages


I believe that the main considerations for this are cost and security, which can be summarized but not limited to the following points:


Cost


KZG commitment has the characteristics of fast verification, relatively lower complexity, and simplicity.


There is no need to submit additional proof, making the cost lower and saving bandwidth.


The Point evaluation precompile required for data access can achieve lower costs.


Security


Assuming a failure occurs, it will only affect the data in the blob corresponding to the commitment, without any other far-reaching effects.


More compatible


Looking at the overall solution of sharding, KZG commitment is compatible with the DAS scheme, avoiding the cost of duplicate development. 5. KZG Ceremony (trusted setup).


KZG Ceremony provides trust setup for KZG Commitment, which has attracted over 20,000 participants and contributions, making it the largest trust setup in history.


全面解析 EIP-4844:Sharding 的一小步,以太坊扩容的一大步


Recently, the KZG Ceremony, which has seen a surge in community participation, is providing trust setup for the KZG commitment adopted by EIP-4844.


KZG Ceremony Procedure


全面解析 EIP-4844:Sharding 的一小步,以太坊扩容的一大步


Refer to Vitalik's flowchart, anyone can contribute a secret as participants and mix it with the previous result to generate a new result, and so on, to obtain the final SRS through nesting, and assist in completing the trust setup of KZG commitment.


trust setup


EIP-4844 adopts a common multi-participant trust setup, namely powers-of-tau, which follows a 1-of-N trusted model. Regardless of how many people participate in the generating setup process, as long as one person does not disclose their generation method, the trusted initialization is valid.


Necessity


KZG commitment's trust setup can be simply understood as generating a parameter that is required for each execution of a cryptographic protocol, similar to the trusted initialization required for zk-SNARK.


When providing proof, Prover uses KZG commitment C = f(s)g1, where f is the evaluation function and s is the final secret obtained from KZG trusted setup.


It can be seen that the final secret is the core parameter for generating polynomial commitments, and as a trusted process for obtaining this core parameter, the KZG Ceremony is very important for the implementation of the entire sharding. 6. Changes brought by EIP-4844.


Rollup


Referring to the schematic diagram provided on ethresear, rollup needs to include the versioned hash of state delta and KZG commitment in calldata for submission (zk-rollup also needs to submit zk proof).


It can be observed that the difference is that calldata only contains small amounts of data, such as state delta and KZG commitment, while the transaction batch containing a large amount of transaction data is placed in the blob.


Effectively reducing costs, putting data in calldata is expensive; reducing the occupation of block space.


全面解析 EIP-4844:Sharding 的一小步,以太坊扩容的一大步


Security


Data availability: Blobs are stored on the beacon chain, equivalent to layer 1 security; historical data: nodes will only store blobs for a period of time, and permanent data storage relies on layer 2 rollups, so security depends on rollups.


Cost


Proto-Danksharding has introduced a new transaction type. The addition of low-cost data format blob will undoubtedly further reduce the cost of rollup. Depending on the progress of on-chain applications and implementation, the cost of optimized rollup may be reduced by x10 or even x50.


Meanwhile, EIP-4844 introduced blob fee;


Gas and blob will have adjustable gas price and limit respectively; the charging unit for blob remains as gas, and the gas amount will vary with traffic to maintain the target of an average of 8 blobs per block (limiting the additional amount of data).


Precompile Implementation


The data in Blob cannot be directly accessed, and EVM can only obtain the commitment of the data blob. Therefore, rollup needs to provide precompiles to verify the validity of the commitment.


Below we analyze two precompile algorithms mentioned in EIP-4844.


Point evaluation precompile (for more information on the mathematical principles, refer to Vitalik's analysis).


Proving that multiple commitments point to the same data; mainly for zk-rollup, which requires 2 types of commitments: 1. KZG commitment; 2. zk-rollup's own commitment. For optimistic rollup, most have adopted the mechanism of multi-round fraud proof, which requires less data for the final round fraud proof. Therefore, using point evaluation precompile can achieve lower costs.


Blob verification precompile


Prove that versioned hash and blob are validly corresponding; optimistic rollup needs to access the full data when submitting fraud proof, so first verify that the versioned hash and blob are legal before conducting fraud proof verification.

 

Three, Danksharding: An important step towards full sharding


The name Danksharding comes from Ethereum researcher Dankrad Feist.


1. Expansion: Further expanding the cache space of Rollup


After the implementation of proto-danksharding, the introduction of a new transaction format with blob has enabled each block to gain an additional 1MB of cache space on average. With Danksharking implementation, each block gains an additional 16MB, with a maximum allowance of 32MB.


2. Data availability: more efficient storage and verification strategies


Compared to proto-danksharding, Danksharking only requires Ethereum nodes to sample blobs instead of downloading the full data. The sampled data is distributed among all nodes in the network and can be combined to form a complete data blob.


DAS: Efficient Sampling Inspection


By using erasure coding, the probability of discovering lost original data is increased, making it easier for all network nodes to detect data loss when downloading partial data, thereby improving security. 3. Security: remains basically unchanged.


Due to considerations of data availability, backup, and sampling inspection, each node no longer stores complete historical data. From the perspective of security, the security of the system has changed from at least one node storing complete data to multiple nodes storing partial data and ultimately restoring complete data.


Although single-point dependency may seem more secure than multi-point dependency at first glance, the number of nodes in the Ethereum network is sufficient to ensure the need for data backup, so security will not change significantly.


4. New Challenge: Higher Requirements for Block Builders


Although validators do not need to download and retain full historical data, the requirements for bandwidth and storage hardware have been eased. However, block creators still need to upload blob data containing full transaction data.


全面解析 EIP-4844:Sharding 的一小步,以太坊扩容的一大步


Here is a brief introduction to PBS (proposer/builder separation). Please refer to Dankrad's application diagram of PBS in the Danksharding scheme:


The role of the block producer will be split into proposer and builder. Initially proposed for anti-MEV design, in the design of danksharding, this is to reduce the bandwidth requirements during block creation.


Four, Other Sharding Solutions: Shardereum's Dynamic Sharding


Shardeum is an EVM-compatible layer 1 public chain. Unlike Ethereum's static sharding solution, Shardeum improves the underlying scalability and security through dynamic state sharding, while naturally ensuring a high degree of decentralization.


全面解析 EIP-4844:Sharding 的一小步,以太坊扩容的一大步


1. Dynamic state sharding


Advantages


Dynamic state sharding brings the most intuitive advantage of linear scaling, where nodes connected to the network can be efficiently grouped by the sharding algorithm and respond quickly, improving the TPS of the blockchain network. In the design of dynamic state sharding, each node covers a different range of addresses, and the coverage range is redundantly designed to ensure efficient sharding and security.


Implementation within the scenario


Ignoring the complexity of the ecosystem, from the perspective of the two technical implementations, the difficulty of dynamic state sharding is greater than that of static sharding. It can be seen that the technical team of Shardeum has a deep accumulation in the technical aspect of sharding. The team's previous research and development on Shardus technology also made great contributions to the underlying technology of this public chain, and even demonstrated the linear scaling brought by dynamic state sharding very well in the early stages of the project.


2. Shardeum Summary


Product


Divide the nodes into different groups, following the divide and conquer approach, to distribute the workload of computation and storage, allowing for higher levels of parallel processing. Therefore, more nodes can be accommodated, further improving the throughput and decentralization of the public chain.


Team


Experienced in the market, with excellent narrative ability and deep research on dynamic sharding.


Technology


Designed a suitable sharding scheme (dynamic state sharding) and consensus design (Proof of Stake + Proof of Quorum) for our own scenario, with the first goal of improving scalability and ensuring a higher degree of decentralization and security.


Progress


The betanet will be launched on 2023-02-02, which is worth paying attention to.


Five, Thoughts and Prospects on Sharding 


Sharding is a long-term solution for Ethereum's scalability, and it is also a valuable and far-reaching long-term plan. In the process of implementing sharding, all existing solutions may be continuously iterated, including the proto-danksharding and danksharding mentioned now, which are worth continuous attention;


It is important to have a good understanding of the sharding direction, but the technical solutions adopted in each proposal for implementing full sharding (PBS, DAS, multidimensional fee market) are also worth paying attention to. I believe that many excellent teams and projects will emerge in this regard.


Sharding is a general term for a scaling technology, but there is not only one specific implementation. It is important to recognize that different public chains will have sharding solutions that are suitable for their own scenarios. For example, some designs in danksharding are only suitable for the Ethereum network, and the security tradeoff requires a large number of nodes to offset it.


The reasonable combination of Sharding and other expansion solutions can improve scalability with the principle of 1+1>2. Currently, Danksharding is not an independent expansion solution, but complements other implementations in the Ethereum ecosystem. For example, Danksharding and rollup work together to achieve better scalability for Ethereum.   


Original article link


Welcome to join the official BlockBeats community:

Telegram Subscription Group: https://t.me/theblockbeats

Telegram Discussion Group: https://t.me/BlockBeats_App

Official Twitter Account: https://twitter.com/BlockBeatsAsia

举报 Correction/Report
This platform has fully integrated the Farcaster protocol. If you have a Farcaster account, you canLogin to comment
Choose Library
Add Library
Cancel
Finish
Add Library
Visible to myself only
Public
Save
Correction/Report
Submit