The original title: "ARE MINING POOLS BECOMING A PROBLEM?"

The original author: Bitcoin Mechanic

The original translation: Luccy, BlockBeats

Editor's note:

Mining is one of the solid and elegant designs that Satoshi Nakamoto created, and it is also one of the most striking aspects of Bitcoin. However, mining is not just about hashing.

Bitcoin advocate and educator Bitcoin Mechanic delves into the various issues and challenges in Bitcoin mining, with a particular focus on the dynamic relationship between mining pools and miners. He proposes solutions to existing problems, focusing on improving miners' autonomy and the overall trend towards decentralization.

Among them, the discussion of the StratumV2 protocol and the future development of the mining ecosystem are full of unique insights. In addition, the article also points out the latest developments in ASIC manufacturing and mining pool infrastructure, providing new possibilities for decentralization in the mining field. Through a critical analysis of existing mining patterns and a look at possible improvements in the future.

Bitcoin miners provide a crucial service to the entire ecosystem. As part of ensuring network security, they are rewarded from the network, which is one of the solid and elegant designs that Satoshi Nakamoto created and one of the most notable aspects of Bitcoin.

However, more and more people seem to overlook the fact that mining is not just about performing hash calculations.

Everyone involved in the process must run a node to obtain the latest state of the blockchain and begin building a new block. This includes verifying the validity of the previous block, discovering unconfirmed transactions, and typically selecting the most profitable transaction among them. The miner pays themselves in the generated transactions, builds multiple Merkle trees for these transactions, and finally hashes them to actually solve the block. The transactions in the block template will constantly change as new transactions are broadcasted to the network. When someone else finds a new block, the miner must switch to building on top of it and dump all the transactions already in the blockchain to fill a new template.

Fork Activation

As you can see, hashing to actually solve blocks is only part of the process. Bitcoin mining ASICs can only perform hashing. In the current environment, all other aspects of mining are typically delegated to mining pools. This has led to some confusion. For example, in any discussion of activating a soft fork through the version bits in a block template, people always refer to this process as a MASF, or "Miner Activated Soft Fork," and there is always someone who will stand up and say that this is the responsibility of the mining pool, not the miner. They may also point out that miners are still ultimately responsible because if they want to upgrade but their pool does not, they can simply switch pools. For clarity's sake, in the rest of this article, I will only refer to those who are only involved in hashing and leave all other aspects of mining to the pool as "hashers."

Returning to the soft fork, in the current environment, when blocks built by the same organization exceed 99%, it may be more accurate to refer to them as "pool-activated soft forks," although no one currently uses this term. This creates a dangerous illusion that mining is decentralized only because of the distribution of computing power. When all computing power is limited to a few pools, this claim becomes unreliable. Therefore, the content of the Bitcoin blockchain will ultimately not include any content deemed unacceptable by these few entities, as well as a series of other issues.

By only performing hash calculations on blocks constructed by mining pools, Bitcoin miners have largely relinquished a key component of their role. This is not only possible, but also the smoothest path, indicating that there are systemic issues at play.

Mining Pools and Block Space Market

Simply performing hash operations and letting the mining pool handle everything else has far-reaching implications beyond soft fork activation. For example, currently miners have no idea what the solved block will look like, which means they are working blindly trusting that the block only contains the expected transactions. However, in blocks like this, you will see a blatant violation of this trust, which is the famous block that sparked the "ordinal" frenzy. Note that the miners working on this block can actually only enjoy about $200 in Bitcoin transaction fees, while the average transaction fee on both sides of the block is about $5000 in Bitcoin.

Block space has value, which is part of the reason why Bitcoin operates over the long term. However, in a world where only a few participants can construct the template that ultimately enters the blockchain, these entities have almost exclusive rights to sell this space and earn excess returns. Do they have a responsibility, or even a duty, to be transparent with their miners about what they are doing? Of course, in this case, the intention is to surprise everyone. Will they forward the funds they receive for selling off-chain block space to their hashers in the future?

In short, although the incentives for mining pools and their hashers are usually aligned towards maximizing profits, mining pools may sell block space to obtain something other than regular Bitcoin transactions, while miner income is more limited unless the mining pool chooses to be transparent and agree to share revenue. Even if they do so, validation requires permission from the mining pool, rather than earning funds from subsidies and transaction fees (which is also tricky when using FPPS mining pools, which will be discussed in detail later).

As a builder of centralized block templates for Bitcoin, the influence of mining pools stems from a more fundamental fact - at a more basic level, there are twelve "super nodes" that have their own "super memory pools".

This leads people to deal directly with mining pools, completely ignoring the memory pool. Some people believe that the memory pool is doomed to fail anyway. The current centralized template construction state only accelerates this process, but it is not acceptable in any case, and in a world where a truly decentralized template construction is considered feasible, making such an assumption may be too pessimistic. Then, if people who buy block space want to include it in the blockchain within the same time frame, off-chain payments must be passed on to a larger audience. This may be more transparent and similar to the current way of working. Instead, "super nodes" are expected to be broken down into smaller parts, so they can no longer provide the same guarantees.

In order to deviate from the mining aspect, let's shift our focus to how the current payment method is being handled.

Mining Pool Payment Model

Almost all mining pools pay their hashers through FPPS (Full Pay Per Share) or similar methods. The only exception is ViaBTC, which offers PPLNS (Pay Per Last N Shares) in addition to FPPS. Antpool also offers PPLNS, but hashers must give up all transaction fee income, which illustrates the point I am about to make. Essentially, in a world that focuses on transaction fee income rather than subsidies, FPPS is not a well-functioning model. It is worth noting that Braiins Mining Pool (formerly known as Slushpool) uses a system called "score," which is actually very similar to PPLNS.

Why do we have such a strong preference for FPPS? From the perspective of hashers, they can get paid no matter what happens on the blockchain. This is consistent with the purpose of mining pools, which is to achieve more consistent income. FPPS provides more consistent payments because mining pools pay based on expected income and settle independently of the blockchain.

This makes life very easy for miners who want to minimize problems caused by cash flow interruptions, but of course there are drawbacks - significant drawbacks, which I would like to emphasize here.

FPPS requires the mining pool to become the custodian of all newly mined bitcoins. These bitcoins cannot be forwarded to miners within at least 100 blocks after mining, so the newly mined bitcoins cannot be used before that. In fact, the mined bitcoins are not related to the bitcoins that miners ultimately receive when they withdraw from the mining pool. The risks of third-party custody should be obvious to almost everyone reading this article, so I will skip this point and continue to discuss other issues with FPPS.

The next focus comes from a more general question, that is, FPPS mining pool is an important intermediary between hashers and the network itself. We have determined that hashers cannot know in advance what the block they are processing will look like until it is solved. FPPS means that they don't even care if the block is found now, which is the problem of the mining pool. Ignoring the increased predictability of payments (which will never happen if the mining pool decides to cheat hashers), we must acknowledge the trade-offs of doing so.

Miners who are paid directly by Bitcoin itself - which is possible in alternative schemes such as PPLNS or independent mining - can expect to receive full rewards, including transaction fees. An FPPS pool can only calculate this retrospectively, as it is impossible to predict how much fees will be received when determining how much each hash contributor actually received. The pool cannot simply assume that fees will be some value greater than 0 and include it in the miner's income while mining, as they will simply pay the miner out of their own pocket if the fees are lower than this value. They must allocate fees regularly and attribute them to the miners actually held in the pool.

From the perspective of a hasher, it is necessary to fully trust the mining pool, because without complete transparency and cooperation from the pool, verification is almost impossible. As mentioned above, this was not a big issue in the past because most mining revenue came from subsidies, and transaction fees were just scattered parts of the satoshis. However, this is not, and cannot be, the future of the growing Bitcoin mining industry. In the future, miners will primarily earn income through transaction fees, which are more difficult to predict and monitor than subsidies when using mining pools.

Compared with payment schemes like PPLNS, Hashers have undergone greater changes (the luck of the mining pool also becomes the luck of the Hashers). We see that the mining ecosystem is extremely inclined to prioritize payment consistency rather than the ability to verify the received content. What's even more distorted is that some Hashers actually prefer this way - hoping to present themselves to government agencies as a "hash service" completely detached from Bitcoin - some even take pride in it. This is because FPPS is far from the ideal miner/mining pool dynamics, making it difficult to describe what Hashers are actually doing, even if it is "Bitcoin mining".

Actually, FPPS pool is a large independent miner that pays hashers to solve its blocks. Afterwards, they have an internal and opaque process to determine the amount paid to the hashers. To truly illustrate this point, hashers can even (in some not too difficult to imagine scenarios) pay their fees with something other than Bitcoin.

Why not? If you don't care about finding any blocks, let alone what they look like before they're built, why not just get an independent miner to pay you fiat currency and point your ASICs to them with the most convenient currency? Bitcoin isn't always the most frictionless choice, but even so, it's reasonable to imagine continuing down the path where "hashing" can be performed by many entities you like, but all on behalf of a small group of "mining pools" that the entire network needs permission from to truly record anything on the blockchain.

Who is doing the hashing?

Let's look at this issue from a broader perspective. As we have mentioned, some larger participants want to stay as far away from Bitcoin as possible, so they are willing to delegate Bitcoin-related activities to their mining pools as much as possible. These mining pools are open to regulation and are very satisfied with their large computing power.

This once again introduces the irrationality of the economy from the perspective of the network, manifested as the behavior of mining blocks that meet certain arbitrary standards. When this happened in the past, it did not last long due to strong community opposition and the absurdity of trying to please a constantly changing regulatory system in a jurisdiction without being required to do so. However, in fact, this is an option that exposes the risk of centralized block template construction. Will a miner in one jurisdiction try to ban or refuse to process transactions from another jurisdiction? Are miners just an extension of government or influential bad behavior? There are specific examples that some mining pools refuse transaction fees for illegal profits, sometimes just to comply with regulatory pressure. From the perspective of the network, this once again appears economically irrational.

The most extreme recent example is the 19 BTC transaction fee paid in a single transaction, which was ultimately discovered by F2Pool and appeared to be a mistake. As an FPPS mining pool, they became custodians of the 19 BTC mining fee and chose to return it to the person who made the mistake. This perfectly illustrates the cost of placing too much intermediary between miners and the Bitcoin network. In PPLNS mining pools, this is less likely to happen. Not because PPLNS mining pools are necessarily untrusted or unmanaged, but because it may be more difficult for the mining pool to monitor and verify fee income at the exact moment the block enters, which may have caused a bigger backlash as it may have already deposited the miners' share of the mining funds into their accounts internally. Although there is no difference in principle, until you compare what happens if a mining pool pays its miners in coinbase/generation transactions. In that case, the money is already in the miners' hands and the mining pool cannot intercept the fee income. Therefore, in this example, the mining pool wanted to appear generous or fair, but instead caused its miners to lose $500,000 in fee income, a position it should not have made a decision on.

In fact, any entity with more than 20% network share can cause problems through various attack methods, some of which are executed in the wild but rarely discussed. I will discuss them in detail later. However, we can see that only two entities in this network reliably exceed 51% of the total computing power. Even worse, one of the largest mining pools did not carefully conceal the additional 10% of blocks it discovered through another large mining pool, and this mining pool maintains a strategic partnership with its parent company. The fact that this farce is still ongoing is not convincing.

In this way, we have limited knowledge about the network. 30% of the blocks can be found by the largest mining pool and accepted by everyone, while the other 10% of the total network hash rate still points to the same mining pool, but is secretly transferred to one or more smaller mining pools. The miners responsible for this 10% may not be aware of how it is being used, and it is more difficult to detect using stratumV2. I will provide a detailed explanation later in the article.

When considering that the mining power redirected through block mining attacks can be used to harm smaller mining pools, this already unfavorable situation becomes even worse.

Specifically, attackers mainly participate in the mining process as normal users of the victim mining pool. As a result, they receive a portion of the rewards expected to be found in any block from the pool. The rewards ultimately flow to the attacker, who can then pay actual miners without losing any funds. So far, the only harm caused is the incorrect impression of the mining pool's computing power, making it appear smaller than it actually is, but smaller mining pools are still unaffected.

Now, if the attacker decides not to tell the victim mining pool when they find a block, harm will occur. This will make the victim mining pool look less lucky. They seem to have found fewer blocks than they should have and paid more rewards to participants who are actually honest miners, resulting in inevitable losses, assuming they do not make up for these losses in other ways.

If attacked in this way, FPPS mining pools would have to burn their income and pay miners out of their own pockets to make up for the shortfall. With PPLNS, miners would wonder why they didn't get what they were supposed to. Either way, block mining attacks are anti-competitive and could destroy a victimized mining pool by bringing it a bad reputation.

The smaller the mining pool, the more vulnerable they are to this type of attack. The larger the mining pool, the more likely they are to prevent smaller mining pools from competing. As large mining pools approach levels that cause community panic, this risk increases, which further encourages them to hide their hash power in smaller pools, even if they do not actually use it for attacks or the frequency of attacks is not high enough, resulting in the problem being ultimately attributed to variance. Since the network provides more consistent payments, larger mining pools have already enjoyed reduced variance, which means they can operate within tighter margins and charge their miners less fees. From the perspective of each unattacked miner/mining pool, this attack means they will enjoy lower difficulty, as the Bitcoin network adjusts to accommodate fewer total blocks.

Is block holding just theoretical? Absolutely not. Even in early 2015, several mining pools were attacked in this way. It is very difficult to prevent such attacks because the mining pool must monitor all miners and carefully decide whether to kick them out of the pool and/or refuse to pay them if their luck reaches statistically impossible levels. The mining pool can reasonably assume malicious behavior. Such attacks also encourage mining pools to want to "know their miners" and regulate payments, which of course makes life more difficult for those who want to mine without permission.

Regardless, the overall effect of all this is that people are more willing to mine with larger mining pools for another reason.

We have heard large-scale miners publicly state that they are abandoning smaller mining pools because the payments they receive are not meeting their expectations.

This is highly undesirable because larger mining pools and the larger miners who use them are more susceptible to regulatory burdens, making them more likely to engage in behavior that harms Bitcoin, even surpassing the centralization of block templates and temporary custody of all block rewards.

Regarding the last point about block withholding, in addition to the threat of making life more difficult for smaller mining pools and those who wish to mine with them, I say to those who still may try to view it as purely theoretical, even though it has clearly happened in the past, do we believe that mining pools can naturally maintain a consistent and clearly tolerable size? This means that new hashing power always comes online in a somewhat evenly distributed manner. We must believe that a mining pool can suddenly appear, grow rapidly, and then stop near the threshold required before people feel uneasy. Have we seen mining pools asking people to stop mining with them, or simply limiting account creation and kicking out miners who exceed the allowed hashing power in existing accounts? Of course not.

There are two more likely scenarios, one being that miners are collectively self-adjusting. However, this is unlikely because it is well known that mining with smaller pools means earning less bitcoin, even if the reasons I presented in this article do not fully explain why. Not to mention, every time there is a notable mass migration of quality from one pool to another, or pools are simply misleading people about the hash rate they are pointing to.

In addition, smaller mining pools have another problem: they may not find a block for several consecutive days, while larger mining pools will not exceed a few hours. This is a resolution issue, the higher your computing power, the closer you are to the expected value in the short term. Unfortunately, this leads to a lower limit, and mining pools cannot expect to make up for losses during unlucky periods below this limit, making it impossible to compete at that time.

The two cycles between difficulty epochs mean that enough blocks must be found within these two weeks so that any unfavorable luck has a chance to be balanced out by subsequent good luck. If not, for example, if a mining pool's expected block production rate is 1 block every 13 days and they have not found a block before the difficulty adjustment causes their projection to drop to 1 block every 15 days, the previous time window will be forever closed. If it is a PPLNS mining pool, the miner's income will be less than they could have earned. If it is an FPPS mining pool, the pool will burn a lot of cash and/or go bankrupt.

This means that only so many mining pools can exist, at least according to the way mining pools operate today. In simple terms, it's not possible to have hundreds of them because many of them may collapse during unfavorable periods, and since they have less than 1% of the network's computing power, they may not even reliably find a block every day, encountering periods of several weeks without a block. This is a limitation imposed on us by Bitcoin itself.

How do miners and mining pools communicate?

The communication protocol between miners and mining pools is Stratum (which is gradually being replaced by StratumV2). StratumV1 is both ancient and deeply flawed. First of all, all communication is transmitted in plaintext. This means that not only do internet service providers know that you are mining, but they also know the scale of your mining. Anyone who can eavesdrop on your network traffic can perform a man-in-the-middle attack, causing your computer and computing power to be used for someone else's mining. This has been previously abused by unknown attackers to steal computing power and divert it away from the intended mining pool.

Except for some inefficiencies, StratumV1 has not been able to provide miners with a practical way to build their own block templates and still mine in the mining pool. All of these issues are addressed in the highly attractive StratumV2 (initially called "GBT" and then "Better Hash"), which we will come back to later.

Hardware/Firmware

Before discussing the dynamics of mining pools/miners, we will digress for a moment, because if we don't mention that only two companies, Bitmain and MicroBT, manufacture ASICs on any meaningful scale, then this article would be incomplete. There are other companies, but in reality, almost all hashing occurs on machines manufactured by these two companies.

This is obviously not good, the root cause is that chip manufacturing is very difficult, therefore too centralized.

The scope of this article does not include discussions on solutions, but some people are working to make home mining more practical (in North America, the main issue is the need for 220-240 volt voltage and dealing with harsh noise). Those who are dedicated to the "pleb-mining" project argue that if enough ordinary Bitcoin users can do this, they can start to occupy a considerable proportion of the network's total computing power, which is more desirable for most large-scale mining operations because they are more susceptible to regulatory intervention.

The difficulty of this task lies in the fact that the firmware is closed source. Even custom firmware that can "jailbreak" ASICs is often closed source to ensure that users pay development fees. In other words, the cost of the market firmware you use represents the mining efforts of the production team.

In fact, the original firmware does this, which is shocking and clearly highlights the urgent need for competition in the ASIC manufacturing industry.

If the network rules are executed by closed-source Bitcoin nodes, would anyone feel comfortable? Additionally, imagine if we all knew that these nodes would cause users to lose Bitcoin to the developers of the software, would anyone accept it? In terms of mining, almost no one cares about the sovereignty of participants. Of course, the importance of node software and ASIC firmware is not equal, and we should certainly be more concerned about the former, as we should, but the latter is not insignificant and clearly overlooked.

P2POOL

Regarding this, there is not much to say except that it basically disperses all aspects of pool mining. Although it has done many ideal things on a small scale, it requires each user to download, verify, and track each other user's share, and prove to each other that they have correctly calculated all content in the template. Achieving this in any adversarial environment is basically an impossible task. Due to the fundamental nature of pool mining, the resources required are much greater than those required to run a full node of Bitcoin, let alone make the work of miners more complex.

Because of these reasons, most people ignore it, and only more technical users or idealists will use it. It can be understood that they are unable to use other alternative methods for mining.

STRATUMV2

This is undoubtedly the easiest problem to solve, and it provides practical solutions to many of the issues mentioned in this article.

Secondly, perhaps most importantly, it also allows miners to build their own block templates. Therefore, although the mining pool will still be the trusted coordinator for reward distribution and may still be the custodian of block rewards, this will represent a shift in power from the mining pool to the miners, which is undoubtedly a good thing.

Welcome to join the official BlockBeats community:

Telegram Subscription Group: https://t.me/theblockbeats

Telegram Discussion Group: https://t.me/BlockBeats_App

Official Twitter Account: https://twitter.com/BlockBeatsAsia