API/RSS

Social

Building Web3 products with ZK technology: Deployment, implementation, and Improvement

23-03-24 09:30

Read this article in 32 Minutes

AI summary

View the summary

原文标题：《利用 ZK 技术构建 Web3 产品：部署、实施和改进》

Original article by Mohamed Fouda

Lynn, MarsBit

Zero-knowledge proof (ZKP) is shaping up to be a fundamentally transformative technology for the next decade. In Web 3, the technology has addressed major scalability and privacy bottlenecks; These are the two main pain points of the exciting blockchain. In terms of scalability, several zk rollups, also known as effectiveness rollups, are being rolled out to scale up Ethereum 10-100 times while improving the user experience by reducing transaction costs. In terms of privacy, ZKP is expanding beyond the era of private transactions and mixed transactions into more complex and useful areas such as private on-chain transactions, identity and authentication certificates.

There is a lot about ZKP, including our own vision for how the ZKP space will evolve in the future and what start-ups will be needed to make that future happen. But there is still a gap in builder education on how to benefit from ZKP and where to start. This article tries to fill that gap by rounding up important resources to guide developers through how ZKP works in practice and how to use it in their applications.

At Alliance, we're particularly excited about the new use cases ZKP brings to the table. Builders in this area are encouraged to contact us to discuss their ideas and apply to join the Alliance programme.

How does ZKP work in practice

The standard definition of ZKP is that it is a process by which one party, called a prover, proves to another party (the verifier) that they know certain information without disclosing that particular information. In practice, at least in Web 3, ZKP tends to be used differently. Most applications do not use ZKP to show ownership of proprietary data. Instead, ZKP is used to increase trust through verifiability. We expect ZKP to become a standard trust model between entities in the future. The reason is that the two main components of ZKP, namely proof and verification, are separated in a way that enables a unique interaction scheme between the entity seeking trust and its users.

The main components of running ZKP are proof generation and proof verification. Proof generation involves running heavy calculations to generate proofs of execution. The effect of this proof is to remove confidence in the prober. Instead, anyone can run a simple procedure on a proof to verify the integrity of the process the verifier is running. This mental model allows the business to run a process, often a complex process, and allows the customer to trust the execution of the said process without having to repeat it. Let's think about an example. Suppose you subscribe to OpenAI's premium plan to use one of their large language models (LLMS), such as chatGPT. You have to trust that OpenAI will actually run the specific model you require, rather than replacing it with a simpler, less efficient model. What if OpenAI could send you a small amount of data to prove that it actually ran the particular model you asked for? Moreover, imagine if every proprietary SaaS product offered such a guarantee to its consumers.

The minimization of this trust is ZKP's commitment. For example, in Web 2, ZKP can guarantee a fair credit assessment or fair insurance claim processing by guaranteeing the same algorithm for all customers. ZK technology is not yet available because running a ZKP program is still relatively expensive. However, we see Modulus Labs And others are building a technique that uses ZKP to prove AI reasoning.

Technical requirements for high efficiency ZKP

On a technical level, an efficient ZKP system needs to simultaneously achieve the following goals:

1. Reduce the computational complexity and latency of the proof system, even if the prover is able to efficiently generate the proof and communicate it to the prover with minimal delay

2. Achieve a smaller proof scale

3. Achieve efficient validation with minimal validation costs

In addition to these primary goals, some secondary goals may be required depending on the use case, such as:

, & have spent Data privacy in privacy-conscious applications, which means that the proof system can process private inputs without being compromised in the generated proofs

, & have spent Avoid trust Settings whenever possible to simplify security assumptions

, & have spent Implement proof recursion to further reduce the cost of verification, i.e. one verification can verify multiple proofs and amortize the cost between different proofs

It can be challenging to achieve all of these goals at once. Depending on the use case, the ZKP system prioritizes some of these goals. For example, the SNARK proof system can produce concise proofs, but the complexity of the proofs increases. STRAK, on the other hand, has efficient validators, but the scale of proof can be 100 times larger than SNARK. Researchers at ZK constantly strive to push the technological frontier by inventing new proof mechanisms to improve all three metrics simultaneously.

Comparison of different proof systems

An important consideration for developers building ZKP related products is how to choose the underlying proof system. There are currently several implementations of ZKP validators, and many more are under development. ZKP back-end selection depends not only on the technical aspect, but also on the target product. Consider the proof system for selecting rollup. Key features of rollup, such as drawdown time, transaction costs, and even decentralization, will be primarily determined by the ZKP proof architecture, as described in the following table.

Demonstrate how system characteristics affect ZKP rollup performance.

In rollup, the proof happens on the business side, that is, by the rollup operator. Existing zk rollup (zkRU) such as Starknet and zksync currently use a centralized prover. Therefore, they can delegate the certification work to a specialized certificator, namely a certificate-as-service company, in order to improve the performance of the certificator. By professionalizing and utilizing optimized SW/HW, the proof time for Ethereum-compatible zkEVM can be reduced to a few minutes. For example, Polygon zkEVM's proof time is currently about 2 minutes. A few minutes of proof time, i.e., withdrawal delay, is acceptable for a rollup.

On the other hand, some use cases require proof that occurs on the user side, for example, to generate private transactions, such as Tornado Cash transactions. To ensure a reasonable user experience, the proof time should not exceed a few seconds. In addition, as users use wallets to perform these calculations in a browser on devices with limited resources, it is important to choose a proof system with a fast validator. A good example here is Zcash changing its proof system to Groth16 in the 2018 Sapling upgrade, which was a major factor in increasing the speed of blocked transactions.

Comparison of proof systems

In general, it is difficult to accurately compare the performance of different proof systems, especially proof and verification speeds, because they depend on the implementation of the library, the encryption curve chosen, and the hardware used. The Mina team provides a good high-dimensional comparison in this article. There are also efforts to create benchmarking tools for different zk systems.

The main demonstration technology performance comparison. Source: O(1) Lbas

The table provides a good comparison of SNARK implementations and their progress in terms of speed from Groth16 to Plonk to Halo. Despite the progress, STARK still wins when it comes to proving speed, at the cost of proving scale. The table also discusses two important characteristics of proof systems: setting discredibility and circuit programmability.

The section on Setting Untrusted discusses the preprocessing phase of circuit creation. Some proof techniques require the participation of multiple entities in the pre-processing phase to generate secret randomness, that is, multi-party computation. If a participant is honest, then the generated randomness is indeed secret and the preprocessing part is safe. This process is called "trusted setup" because it believes that at least one participant in the preprocessing phase is honest. Requiring a trusted setup is considered a weakness. In this sense, STARK and newer SNARK systems, such as Halo 2, have an advantage. However, some projects use trusted Settings as a tool to engage the community. And Manta That's the case.

The programmability section discusses proving whether the system can prove arbitrary calculations. SNARK is usually programmable for any calculation. However, proving efficiency depends on the type of computation being performed. This is not the case with some types of STARK systems, which have a harder time adapting to different types of computation.

Certification system for different ZKP related projects

How can you use ZKP for your product

It's actually hard to build a mental model of where to start building a product that benefits from this innovative technology. This section attempts to provide a framework for developers to choose the best way to integrate ZKP into their products. Developers will have several tools to choose from, based on product requirements, ecosystem consistency, and performance requirements. Some developers will be able to reuse their existing code, while others will have to learn a new domain-specific language (DSL) to create their applications.

Performance-focused zk applications

Developers can use ZKP to achieve higher throughput (TPS) or reduce fees by doing most of the application calculations off-chain and only publishing proofs on-chain. In this case, there are several frameworks to choose from. Each of these frameworks provides a process for compiling an application written in a high-level language into an equivalent representation suitable for proof generation, the zk circuit. They provide a set of tools to compile application code, generate zk circuits, implement zk validators, and generate validator code for the target ecosystem. We can divide these frameworks into two main baskets: EVM-focused and non-EVM-focused.

zk framework with EVM as the focus

This set of zk frameworks is consistent with Ethereum by building rollup on top of Ethereum. Transactions and applications are executed on the rollup's zk virtual machine (zkVM). Proofs are generated by specialized validators and published to L1 for verification by smart contracts.

The first subset of this group implements EVM-compatible zkVM and is therefore called zkEVM. These goals were achieved by allowing Ethereum developers to use solidity and familiar tools such as Hardhat And foundries; Without making changes, thus minimizing friction. They abstracted the complexity of zk by creating a circuit and validator that worked with EVM out of the box. This basket includes Polygon. And Scroll.

The second subset of this group is the zkVM. which is not compatible with the EVM itself. Although not compatible, this group reduces friction by creating a middle layer that enables developers to call this group Type 4 zkEVM.zkSync Era using Solidity.Vitalik; Starknet Is a good example of this group. The advantage of using zkEVM type 4 is that it can provide higher throughput and lower cost than the EVM compatible type. This makes them suitable for building high-throughput applications, such as on-chain gaming, or high-performance financial products, such as order book DEX.

Building an application for type 4 zkEVM requires more developer effort because the amount of Solidity code you can use is limited. Alternatively, developers can decide to learn a different language, such as Cairo, to develop native applications for these frameworks.

Source: zkNode, Scroll, Zksync-solc, zkSync SDKs, Protostar, Starknet Hardhat plugin, Warp

Non-evm zk framework

Another type of framework is those that do not target EVM architectures because they target competitive L1 or general purpose computing. However, they are still available through specialized SDKS, such as Sovereign, for building application-specific zkRU on Ethereum.

There are two ways to do this

1. Developers write code in a high-level language targeting a particular virtual machine architecture, which is then compiled into zk circuits.

2. Developers write code in a domain-specific language (DSL), such as Circom, to generate zk circuits directly.

The former approach is more developer friendly, but often results in larger circuits that take longer to prove.

Sources: Risc0, zkWasm, Circom, Snarkjs, Mina zkApp CLI, zkLLVM

A privacy-focused zk application

Developing privacy-focused applications with ZKP is often a more demanding task for developers. There is less work to develop privacy-focused solutions with ZKP than with scale-focused solutions, making the learning curve steeper. Existing privacy apps focus on payment privacy and don't allow for much programmability. Combining privacy and programmability is a challenging task. Privacy-conscious applications mainly follow one of two implementation scenarios.

1. Built on GM L1

To enable private payment applications on L1, you need to build the ZKP logic as a smart contract. Applications often use ZKP to create private pools of money. Users use these private pools of money as mixers to fund new wallets that are not connected to their original wallets. The well-known example here is Tornado Cash. For these applications, the proof is done by the user and the verification is done on the chain. Therefore, it is critical to use a ZKP system that has fast proofs, simple verification calculations, and does not reveal any user information in the generated proofs.

Because universal chains are not optimized for expensive cryptographic computing, verification costs are often high for mainstream users, limiting adoption of these applications. An intuitive solution for moving private trading applications into rollup to reduce gas costs can be challenging. In this case, the private transaction proof needs to be included in the rollup proof, i.e. proof recursion, which is not currently possible in Ethereum's universal zk rollup.

2. Create a new privacy-focused L1/L2

To make private transactions and applications cheaper, developers are forced to either create a new privacy enforcement L1, such as Manta Network And Penumbra, either set up a dedicated rollup such as Aztec. Most privacy-focused chains do not yet support general computing and instead focus on specialized use cases. For example, Penumbra and Renegade Focus on private transactions. Aleo A framework for private applications is being built to make programs written in a high-level language compatible with the corresponding zk circuits by creating a dedicated language, Leo. The interaction of the application is carried out off-chain, and only proof is posted on the chain as a private transaction. Aztec is moving in a similar direction, but as Ethereum L2. They recently announced that they are focused on creating a universal privatization rollup using Noir As the default smart contract language.

zk acceleration

After the developer has chosen the appropriate zk development framework for his application and selected the underlying proof system, the next steps are to optimize the application's performance and find ways to improve the user experience. This often boils down to improved validator performance and latency. As mentioned earlier, for rollup, reducing the prover's time means reducing the delay in submitting a proof to L1, and thus reducing the delay in withdrawing it. For user-generated proofs, i.e. privacy applications, faster proofs mean shorter transaction generation times and a better user experience.

As we have discussed in previous articles, speeding up the proof process often requires software optimization and dedicated hardware. In the past few months, the dedicated hardware competition has heated up as a number of companies have entered the fray. In this section, we'll discuss the current state of zk acceleration and how developers can benefit from this competition.

Proof as a service

Until now, the standard mode for performing zk proof work has been to use powerful servers with multi-core cpus and/or Gpus, and utilize optimized open source libraries, such as Filecoin's Bellperson, to improve proof performance. This pattern adds complexity to developers who need to maintain a proof infrastructure. A better model to address this complexity and allow for better specialization is the proof as a service model. In this mode, the entity that needs to generate proofs for a zk circuit or a use case connects to a vendor running proprietary software to perform proof calculations. Some companies can generate proofs specifically for specific use cases. For example, Axiom A system has been set up to generate Halo 2 proofs for Ethereum's historical data. Other players can focus on specific ZKP backends, such as Plonk or Halo 2, and build proprietary optimizations that make proven calculations faster and more efficient. The =nil Foundation takes this concept one step further by establishing a market for ZKP calculations. In this proof market, the proof buyer submits a bid to generate ZKP, which is matched and fulfilled by the proof generator. Mina has a similar concept called Snarketplace, but it is limited to the SNARK proof required for the Mina network.

Hardware acceleration

With the introduction of several L1 and rollup systems that rely on efficient generation of zk proofs, the competition to generate these proofs and obtain the associated rewards will heat up. If these chains and L2 succeed in attracting a large user base, then proof generation could develop into an arms race akin to the Bitcoin mining competition. There are different approaches to ZKP acceleration, namely GPU and FPGA and ASIC. This Amber Group article does a good job of discussing these different options and the challenges that each implementation faces. However, in the long run, the company that produces the proven most efficient AISC generated will have a significant economic advantage in the chain focused on ZK.

It's worth noting that zk proves there's a major difference between competition and bitcoin mining that's worth highlighting. In Bitcoin, the mining process is based on a simple calculation known as a SHA256 hash. This kind of computing is fixed and impossible to change, which keeps the focus of competition around chip design innovation and access to the most advanced semiconductor nodes. In the ZKP domain, there is a clear division between the different proof protocols. Even with the same proof backend, such as Plonk, the target circuit size can cause a difference in ASIC performance. This distinction between bitcoin mining and ZKP generation can lead to multiple winners specializing in different ZK backends.

There are multiple players entering the zk dedicated chip space. Each player focuses on one of the two main operations for improving proof generation: multiple scale multiplication (MSM) and number theory transformation (NTT). The last company to come out of stealth was Cysic, which announced its $6 million seed round on the sidelines of the ETH Denver conference. Cysic is focused on accelerating MSMS through the use of FPGas. The flexibility of FPgas allows it to support different ZK systems. The approach is similar to Ulventanna, which announced a $15 million seed round in January. Other players in zk chip development include Ingonyama, which publishes a platform called Icicle To speed up MSM and NTT computing on the GPU, and Accseal, Snarkify And Supranational. In addition to this list, there is research work from other stealth companies and notable people in the Web 3 field. Examples of the latter include Jump Crypto's CycloneMSM Implementation, using FPGA to accelerate MSM computing, and Jane Street FPGA Implement, accelerate MSM and NTT.

Due to the increasing importance of ZKP acceleration, and the need for fair evaluation of different implementation methods, The competition is becoming an important place to promote the development of this field. The 2022 race has more than $4 million in prize money.

Available educational resources

In this section, we have combined a list of educational resources to help builders understand the ZKP field. This is by no means an exhaustive list, as there is plenty of excellent content on topics. A comprehensive list of zk resources is available athereandhereFind it. This is an effort to create a friendly way for developers to learn about the space.

For those interested in learning the basics of ZKP and how they work, one of the first resources to check out is the zk Whiteboard session from zk Hack. Dan Boneh's three introductory courses, in particular, are high-level enough for people with some basic math knowledge. The rest of the series deals with space-specific themes.

This excellent beginner's guide is useful for developers who want to go straight to the zk tools. Poseidon LABS then created an Applied zk workshop to guide developers in building a zk application using Circom and Hardhat. Other workshops for other zk languages and frameworks include this one using Noir and this one using Risc Zero.

conclusion

As a believer in ZKP's potential, Alliance strives to help more builders enter the field and provide them with funding and guidance. In Web 3, ZKP is already addressing the pain points of scalability and privacy that hinder mainstream adoption. In Web 2, ZKP can bring the spirit of trust minimization to a wide range of businesses, including SaaS, insurance, and credit scoring. This article is designed to help builders integrate ZKP into their products. The article takes the builder through the different stages of ZKP integration by planning the different stages, covering the implementation, and performance improvements after deployment.

We encourage builders to contact us further and discuss entrepreneurial ideas in ZKP space together.

The authors would like to thank Shumo Chu, Anish Mohammed, O(1) Labs Members have had fruitful discussions and feedback on this article.