OPRR Flash Depth Data

Search Extension

Panoramic overview of Web3 middleware and infrastructure

2023-02-09 09:30

Read this article in 45 Minutes

Why should next-generation application builders choose Web3 middleware?

Original Title: "Web3 Middleware and Infrastructure Overview"

Original Author: Cam

Original Compilation: Translation Association saku, SeeDAO

< br>

Summary: This article categorizes and details the Web 3 middleware in the zee prime portfolio in categories including: Storage/Data, Data Model and Validation, Indexers, Access Control, and Integrated platform. It describes their relationship to the underlying blockchain, and how they combine to form Web 3 applications. In addition, the article explains why the next generation of developers should choose Web3 middleware.

It has been a year since we published a middleware paper with Pocket Network as the main content . During this time, the middleware field has evolved and grown exponentially. In this article, we revisit middleware and infrastructure from a top-down perspective, trying to connect the changes taking place in this rapidly evolving field.

It is worth noting that this paper focuses on Zee Prime's portfolio. We explored our portfolio inwards as we delved deeper, taking individual studies forward and combining them into a middleware "meta thesis".

We apologize for using buzzwords like composability, Web 3, etc. But it's hard to paint its full picture without relying on a few memes. Memes play an important role.

Middleware Thesis Volume 2

As higher-level applications on the blockchain continue to outperform early DeFi primitives, the need for robust infrastructure and middleware expands. We at Zee Prime have been loudly bullish on this "category" for years. Our first middleware paper - "Infrastructure LEGO Middleware Paper" highlights the importance and economics associated with robust data relay in the development of sustainable decentralized platforms.

This time, we tried to expand our thinking to the already emerging middleware ecosystem, Especially in implementing decentralized application middleware. While something like Dfinity exists for sure as a one-stop solution, end-to-end solution for all your troubles, until the day comes when a product like Dfinity is adopted on a mass scale, we'll have to piece things together Together, and gradually build independent solutions.

Meme said that DFINITY, whose goal is to become a "decentralized world computing cloud", is dead, and it cannot be rescued, so it has to be buried.

Now, narrative is a tough sell.

Through the boom in application complexity (yay, real applications) we see Explosive growth in demand for supporting infrastructure. From the early days of DeFi, massive amounts of network data, indexers, access control, and other middleware tools emerged—all of which are key glue pieces for next-generation applications.

The big picture about middleware can be difficult to paint. Middleware is a nebulous concept, like Web 3. Its boundaries are often blurred. Fundamentally, when we speak of middleware, we mean any item that supports other higher-level applications. The last time we discussed this topic, we pointed out that it will serve as a middle layer, providing connectivity between the upper and lower layers. However, the actual network interaction is very complex, and it is difficult to map the obvious hierarchical relationship between the upper and lower levels.

< /p>

Is this the map for Web 3 middleware?

In this updated middleware paper we describe other classes of middleware, while also clarifying their use cases and reasons for using them for builders. This is by no means an exhaustive list.

Storage/Data

A key element of the decentralized application stack, which is also an essential element of basic computing: storage. The Cambrian explosion of activity and complexity of Web 3 also demands a storage solution that goes beyond simply recording account state at a base layer. More and more decentralized applications are seeking technology stacks without centralized points of failure or scrutiny associated with Web 2 solutions.

Every application will require some form of service provided by Web 3 middleware. One of the challenges of this, however, is that Dev-Ops is complex, and not every builder has the expertise to implement Web 3 middleware in their projects. Therefore, we need abstraction layers so that it will be easier to assemble this infrastructure into new projects, just like assembling Lego components.

This reflects the challenges facing the wider adoption of cryptocurrencies. Wallets, mnemonics, and gas fees required by trading platforms are inherently user-unfriendly, and unless there is a higher level of abstraction, widespread adoption will face resistance. We have to assume that the average user doesn't have to deal with the complexities we see today.

Storage networks such as Arweave and Filecoin have been around for a while, providing for excess storage and storage needs Provides a distributed matching system - this is probably well known and needs no introduction from us.

Different solutions provide varying degrees of persistence and censorship resistance. They are the foundation of any decentralized technology stack. Storage can be broken down into two classes of protocols, the base storage layer, and aggregators that address scalability and facilitate wider adoption.

Banyan is one piece of the puzzle and we see it as an important aggregation layer for storage networking. Banyan focuses on intermediary services for storage, improving the economic incentives of storage protocols currently used for bridging, Banyan ensures that applications can use Web 3 storage solutions in a network-agnostic manner with guaranteed provenance.

Banyan can also integrate new Web 3 services for Web 2 applications (such as decentralized storage solution) acts as a bridge. Implementing these storage solutions is currently very complex, and Banyan's abstraction layer and marketplace lower the barrier to entry for these LEGO components.

Similarly, Spheron is also intended as an abstraction for a wider range of middleware solutions layer. The protocol aims to be a one-stop solution for deploying and automating Web 3 projects. It has an "app store"-like interface where non-Web 3-native users can easily choose from decentralized infrastructure offerings—like the digital ocean of Web 2.

Data Model and Validity

The blockchain is a state machine that continuously generates data in the form of state transitions while performing calculations. The number of accounts, states, and smart contracts grows rapidly over time. This can cause issues with everything from indexing to initial node synchronization and backup durability. These will affect the scalability and security of the underlying state machine.

KYVE takes a more granular approach to specific data validity issues, utilizing the Arweave Network Provides critical support for applications and protocols. KYVE is a decentralized data lake (raw unstructured data all in one pool) solution for storing, validating and retrieving data streams.

KYVE's initial product market fit largely helped with the node synchronization issue. By providing easily retrievable, verified profile status data, the synchronization time of the initial node can be greatly reduced, ensuring that new validating nodes can be added to the network, and network security is maintained. In another case, if the initial node synchronization time is too long (and still growing), if the number of validators decreases and new validator nodes join difficulties, the network security will be threatened.

While we have discussed where and how data may be stored, we also need to consider Models and representations of these data. For applications built on these state machines, data generated during their activities will require flexible storage and computation, not just account balances.

Ceramic, at its most basic level, is a decentralized network of data models. Why this is important infrastructure for decentralized applications, in other words, "Layer 1" (L1) stores the state of account balances (accounting data). KYVE seeks to provide data validation for state changes at L1, while Ceramic provides application data state and model storage on top of the base layer. Users of the network can create IPFS data collections (streams), allowing static data (such as data on Filecoin or arweave) to become high-order mutable content.

In addition, Ceramic has also established an open source data model trading market to realize Composability of these data models. You may often see that Ceramic points out that doing so brings data the same level of composability as ERC20 brings to DeFi. It proposes a standard about data and allows them to be replicated in various applications. use. And just like that, the Lego of money meets the Lego of data. It's almost a return to open APIs like those built on social networks like Facebook.

Kwil takes a SQL-compatible approach to the Web 3 data model. The biggest advantage of such a model is that there are a large number of developers who are familiar with SQL. Kwil uses a network of nodes to maintain a relational database. These databases (called database moats) are maintained by a subnet of nodes and kept updated by scanning for new writes and querying other nodes within the same moat. Most interestingly, these nodes can run an advanced request mesh gateway for efficient logic execution of database interactions.

Indexer

With the proliferation of data generated by applications and networks, the need for an interpretation layer has increased. Just like in the early days of the web, people had to manually memorize addresses and maintain IP address books. DNS and search engines provide a human-readable indexing layer.

With the development of the Internet, the degree of centralization of index data increases with the development of economic scale , and makes querying the data more user-friendly. Similarly, in L1 blockchains and storage networks, indexes are also very important. Due to the nature of distributed systems, a piece of data may be divided and stored in multiple places, making it difficult to retrieve. An index layer helps speed up the query process and creates a normalized program.

Zee Prime's investment company Subsquid is a case study on indexers. Subsquid takes a multi-tiered approach to indexing on-chain data in a decentralized manner. Their ultimate goal is to support the next generation of APIs for Web 3. The protocol supports both the Substra and EVM ecosystems, specifies the types, schemas and definitions of on-chain data, and subsequently enhances newly indexed data by switching it to their API-based call solution (rather than using RPC) retrieval capabilities.

The purpose of Subsquid is to obtain stronger retrieval capabilities of new index data at the same speed

This layer consists of two types of nodes: Squid classifies data and supports subsequent API queries, while Archive continuously ingests raw data from the underlying state machine and saves it in the database.

The above figure describes the two types of Subsquid For nodes, Archive ingests raw data from the underlying state machine (blockchain) and saves it in the database, while Squid classifies the data and supports subsequent APIs (such as GraphQL Gateway) to query.

Similarly, SolanaFM is an indexer as well as a block explorer. Raw blockchain data is processed into a queryable format to serve Solana's ecosystem, much like other ecosystem indexers. The immediate use case provided by SolanaFM is to use their API to power DeFi applications on Solana. If you've ever worked with Graph and Subquery, these solutions might sound familiar to you. Both solutions target a wide variety of end markets.

Glitter completely solves another problem: decentralized storage. It can be thought of as an indexing service for developers. As Web 2 applications seek opportunities to move into Web 3. They will bring huge data volumes to the new world. While increased data helps Web 3, it also confronts developers and the community with the daunting task of storing and indexing this data.

Glitter creates a win-win solution for developers and community by providing hassle-free service in exchange for crowdsourced data. This model has proven effective in the cooperation of several social applications storing data on Filecoin.

This picture describes the operation mode of Glitter , the decentralized APP of the Glitter ecosystem is responsible for uploading and receiving data stored on the network, and distributing tasks to Glitter network miners, and receiving the results of their data inspection and indexing. The outside world can access the final index results through the centralized APP.

Access Control

One of the most important and historically underrepresented missing pieces of Web 3 application infrastructure is access control. Who can see things on the Internet? This is an important philosophical question, and access control becomes increasingly important when faced with security issues related to national/corporate/individual sovereignty. The semantic nature of public blockchain/Web 3 technologies allows us to better differentiate which users should be able to access what and how, and while these systems are inherently open, access control frameworks will allow Encrypt/decrypt.

The Lit protocol aims to solve this problem through threshold cryptography. At its core, the network can provide access to the resources and content of the entire network based on some public credentials (such as NFT in the wallet). The protocol runs a network of nodes that verify proofs and approve handshakes. This network can verify proofs presented and whether such proofs comply with previously set access control conditions, all within a computable security compartment. Once verified, the desired content can be exploited. Some people refer to the Lit protocol as a read solution to the writes provided by Ceramic.

Guild.xyz also tries to solve the access control problem from a different angle. Initially focused on creating token-permitted discord environments, the project has now expanded to focus on multi-chain access portals based on similar principles.

Integration Platform

To further integrate and piece together the blocks we imagined in the world of 3D bridges, Polywrap (formerly known as Web3api) is taking the integration of Web 3 protocols even higher level of efficiency. While Web 3 protocols are open and technically composable, truly achieving this composability in practice is much harder than it was in Web 2. This is because each protocol requires specific business logic to run in the application, and these business logics often make up an SDK in a specific language.

Integrating all these different SDKs is very inefficient due to lack of standardization. Additionally, they all support specific languages, which means that protocol developers often release functionally duplicate SDKs for different languages, creating a maintainability nightmare.

Polywrap's solution eases this burden by leveraging standardized patterns and WebAssembly. Polywrap's integration makes it simple to call against an easy-to-read schema (think REST API), rather than pre-bundling SDKs for various protocols in your application. The wrapper will be downloaded at runtime and executed in the application. In short, this means that any application that integrates Polywrap can gain access to any Web 3 protocol.

The user experience of Web 3 applications is still not silky enough. As we highlighted earlier, entering gas creates friction for the user. By integrating Biconomy's API, an application can improve this user experience. Biconomy's platform provides a range of tools to enable gas-free transactions, enjoy faster confirmation times, pay in ERC20, and instant cross-chain transactions.

Gas free transactions by using meta transactions (via ERC 2771) and some clever forwarding design became a possibility. The cross-chain function is realized by the supported layer/on-chain liquidity pool, and the off-chain server (executor node) is used to monitor the transaction into the pool, and then "release" the other party of the transaction.

Tools like this are critical to making the user experience smooth for the next billion crypto apps Slippery is crucial. Our goal should be a continuous effort to achieve a seamless flow of interactions across Web 3 systems.

Although it cannot be clearly and neatly placed into a category, Sepana is building a search for Web 3 engine. Whether it is DeFi, social, DAO activities, or NFT, Sepana's solution will provide a full-text search engine that enables users to browse the entire Web 3. By utilizing the aforementioned indexers, as well as its own indexing of Web 3 applications and data, the protocol will act as a gateway to the wider ecosystem. Furthermore, Sepana's transparent and open-source algorithms can be used to drive other applications such as social media feeds based on social graphs stored in database solutions such as Ceramic and Kwil.

Long-term vision has very interesting branches. Just imagine a world where you could tailor your social media experience (i.e. your feeds) to achieve a specific mood or outcome (such as less polarization or broader happiness as Sam Harris often discusses). All of this can be achieved through open source algorithms, so that the platform can be adjusted according to your needs, or the right to adjust can be handed over to users.

How does it all come together?

Most modern technology companies and applications can be distilled into some form of the following business model : Data production/digestion, models built on said data, and control and distribution of data/models. The smooth user experience of modern web applications and the exploitation of user dopamine are built on these basic processes.

As a result of this workflow, we hope that middleware solutions can learn from these technical requirements and support them in the context of Web 3. We can see this by looking at the items/types of middleware described earlier. Every piece of Web 3 middleware fits into the general workflow we describe.

The above figure summarizes the applications at all levels of the Web 3 middleware application stack layer by layer from top to bottom. The access control includes Lit Protocol and Guild.xyz, and the integration platform includes Polywrap, Gasless, And Sepana, data models include Ceramic, Kwil, data validity includes KYVE, indexers include SolanaFM, SubQuery, Glitter, Subsquid, etc., storage networks and solutions include Filecoin, Banyan, Arweave, and Spheron.

While the above categories can be used as a broad overview, in reality, many The functionality of middleware may span multiple categories. In 2022, it is difficult for us to precisely define these categories because of these overlaps. To make this question more realistic, let's take a common example, say, a social media network, and extend such a mental model to a broader Web 3 middleware application stack.

Our imaginary social media network will be aptly named twatter. In practice, we can see the product of the platform — the social media experience — flowing through the components in the middleware stack in the diagram above. Note that we do not consider Web 3's social network to be "decentralized Twitter". We imagine Web 3 social networking as more of an emerging phenomenon, perhaps even citing Web 2 apps for proof (if they decide to open up the API) through something like Sismo.

Zee Prime's imaginary social media network is called twatter, which aims to implement social network applications through middleware stacks.

Starting from the most primitive form, all data of the platform (username, profile picture, history activities, social graphs, etc.) can all be stored and indexed in IPFS format on one of the storage networks. The data models that give this structure and meaning are stored on Ceramic or Kwil, a database solution for Twitter accounts that would have models for every part of the platform mentioned earlier.

For example, if the platform requires you to hold a free NFT to access the platform (spam reduction mechanism or other purposes), then users need to connect their wallets to the platform, and the access control protocol will be verified and handshaked before the platform is presented to the end user. The integration platform can be subsumed by the application layer so that it can natively enable other Web 3 services, while Sepana's algorithms can be used to design social graph based pushes.

The above figure describes the middleware stack The workflow of the social network constituted, the data of the basic layer is sent to the data model through the state machine indexer, all the data of the platform are stored on the storage network, and the storage index is established, the data model of these indexes is called by the APP, and the access control layer Responsible for checking that the user has access before exposing the platform to the end user. An integration platform may be included in the application layer.

The most interesting thing is that while writing this article, we stumbled upon Orbis Social , which has effectively built the social network described above, using an application stack nearly identical to our description. The next generation of applications is in development, and we expect to see many more different use cases in the coming months.

An important point that must be explained here is that the middleware on the right in the figure is more related to the Nothing to do (Translator's Note: less dependent on the chain). Parts of this structure are interchangeable with their competitors, however, many of these business models lead to monopoly based on network effects. In contrast to the monopoly of Web 2, these platforms ultimately redistribute this pseudo-normalized compound value to the users of the platform.

Why Should Next Generation Application Builders Choose Web3 Middleware?

While Web 3 tools continue to emerge, we must continue to ask ourselves why so? Do they really bring more benefits than Web 2 solutions?

Web 3 middleware should be built on the same foundational principles as its early cryptographic predecessors. Teams should choose Web 3 middleware solely because of its benefits, because Web 3 middleware enables their application to achieve its goals. Whether it's from a security, durability, or censorship-resistant perspective, the benefits of Web 3 middleware should stand on their own. Many new functions may be unlocked because of some inherent features of Web 3 middleware that we can't even imagine.

These infrastructure LEGOs enable deeper integration, often with the Semantic Web— Related to Tim Berners-Lee's vision of an open and composable Internet, and offering cheaper hosting and computing solutions than their Web 2 counterparts. As Denis-Nazarov pointed out, if a complex computing system is to become a thriving ecosystem like a garden, it needs to be modularized and specialized, and in a Web 1/Web 2 world, users give up on Management of state to obtain the ability to implement connections. The giants of the Web 2 keep valuable state information private because having that information captures value as a result.

The public state machine allows this pattern to be subverted, state is maintained in an open manner, and , the introduction of the token economics model can strengthen the union of both parties, so that both parties involved can obtain better results. This is the nature of reflexive assets.

Before apps notice, Zee Prime's middleware solution is here!

Zee Prime Viewpoint

In many ways, middleware is the B2B part of cryptocurrency. As such, good middleware solutions tend to be both highly technical and less intuitive to the end user (but never mind, the end user is not the target audience). Rather than keeping an eye on new DeFi protocols, NFT projects, or GameFi studios, we believe that helping to make sausage on the factory floor (Translator's Note: Performing invisible but important work behind the scenes to end users) is crucial to the continuous development of new applications. important.

Does the future belong to middleware? of course.

In summary, this infrastructure (and future infrastructure) will serve the following Role:

Increasing resistance to censorship

Promoting a positive-sum economy Game

Improve efficiency

Realize a new business model

An additional potential impact of this interchangeable infrastructure modules and abstraction layers is that applications will:

Getting farther and farther away from the base layer, and

As a result, Will become more and more chain-agnostic

This is not about fat protocol theory (for base layer hypothesis of accumulation of value) rather as a sign of this branch of continuous progress. In principle, this could be seen as reducing switching costs. The earliest on-chain applications (mainly DeFi) are characterized by extremely high binding to the base layer (that is, financial products built on financial accounts).

More complex non-financial applications will have looser affiliation with such chains, reducing Switching costs are reduced (free NFT access controls are easily ported to new blockchains and wallets). Applications are already attracting users across chains.

We firmly believe that adding the transfer of value to the transfer of information is a meaningful step change, but realizing that potential, and the various applications and user experiences that enable it, requires a large number of infrastructure LEGO components.

The value capture of this "part" is the most debated topic when discussing middleware investments one. In a way, really critical middleware looks a lot like public goods, although one could argue that this applies to some successful future applications as well (Twitter wants to be a public good).

One might therefore expect profit margins/expenses/revenues to converge towards the lowest possible bound. While we believe this will be the case to some extent, falling to a publicly acceptable rate is a more reasonable assumption.

Although it may seem unattractive at first glance, for the first time in the world In the context of an asset-light/resource-light business in the global technology revolution, this scale could easily be multi-billion dollar value capture for these LEGO components. Since these middleware perform specific functions for applications, their total available market is independent of the blockchain below them, or the applications above them, at any one time. Rather than relying on specific upper and lower components, middleware provides functionality on a larger scale.

While middleware and DeFi do share the self-referential nature of token-based economic models, But they ultimately differ in their ability to return value. Middleware projects typically benefit from a clear supply and demand drive for their tokens (like network nodes) used to deliver the services they provide. In contrast, most DeFi projects have less clear demand drivers for tokens, and regulatory considerations for cash flow distribution make the situation even more ambiguous.

It is for these reasons that we continue to search for new middleware solutions to enable More convincing next-gen applications for continued adoption of cryptocurrencies. We believe that a new generation of applications will unlock financial and online commerce/activity on a massive scale. What a16z-esque said about this is that we don't want knock-off apps, we want native apps.

If you are a founder, builder, or pair that needs to be built to make this happen Target's critical infrastructure has ideas, and you know where to find us.

ZeePrime doesn't care about MEV, ZeePrime only cares about middleware value!

Glossary

Web 3 - You can refer to the article: Beyond buzzwords

< /p>

Apps - We don't use dAPPs (decentralized applications) because we think "how decentralized is your app?" is a spectrum. Applications may be driven by decentralized and centralized solutions. Does this mean they are not decentralized applications?

WASM—an acronym for WebAssembly, can be considered a stack-based virtual machine, It executes in binary format, but can be compiled in various languages such as Rust, C, Python, etc.

SDK - stands for Software Development Kit. It is a set of software construction tools for a specific platform, usually in the form of libraries, frameworks, artifacts or debugging tools.

IPFS - Interplanetary File System. It is a distributed system that uses content addressing to store and access information.

Immersive Design – The official definition is as follows: Immersive design involves making a new thing look More like old things or more familiar features. In practice, it means new ideas that are easy to digest. An immersive design app is the opposite of a native app.

API - Application Programming Interface. This is a software that can be exploited by other software.

RPC - Remote Procedure Call. In the context of public blockchains, this basically means that one part of a computer invokes a process (which can be considered an action) in another part of the network (i.e. another computer/application).