Bitcoin Programming

Click here for cryptocurrency Links

Machine Consensus Via Proof-of-Work
How does Bitcoin use a peer-to-peer network of computers to enforce the rules agreed upon by human participants?
In the last section, we discussed how hackers organize to create a system like Bitcoin, and established that the machines in the network are used to enforce rules upon the participants. But it can also be said that the machines enforce rules upon each other, such that clever humans are frustrated when trying to change them. This section explores how computers are used to keep human participants honest.

So far, we have contended that the “problems being solved” by Bitcoin are not abstractions (ie., “central banking” or “soft money”) but the concrete challenges of coordinating specialized human labor outside a command-and-control structure. We’ve established that the motivations for avoiding a command-and-control structure are threefold:

To minimize the opportunity and motivation for the managers of the system to cheat or hassle the participants.
To attract skilled technologists to build the system without direct compensation (ie., FOSS and open allocation).
To eliminate gatekeeping, and allow anyone to use the system without permission; this achieves maximum growth and success of the software.
Next, we’ll talk about how Bitcoin accomplishes this feat of machine cooperation without losing these three desirable qualities.

How machines agree on a shared transaction history
Recall the first section, discussing Nakamoto’s message in the Genesis Block. About every 10 minutes, the system collates, validates, and bundles the new transactions. These bundles are called blocks. Block producers are called miners.

Each block contains a hash of the data from the previous block. A hash function is a one-way algorithm that maps data of arbitrary size to an output string of bits in a fixed size, called a hash. Changing the data fed into the hash function changes the resultant hash. It is one-way as it is not possible to reconstruct the data given the hash and the hash function. It follows that if a block contains a hash of the prior block, it must have been produced after the prior block existed. Since changing a block in the middle of a sequence of blocks would invalidate the hashes in all subsequent blocks, conceptually they are chained together. Blocks can only be appended to the end of the chain.

The data structure which results from creating a new block and including the hash of the prior block in a continuous manner is known as the blockchain. In a blockchain-based system all participants validate the hash of a new block before updating the state of their ledger.

How block producers are selected
We have established that all machines mining on the Bitcoin network work to bundle the transactions since the last block. If they are the first to report a new block, they have a chance at being paid a coinbase reward (currently 12.5 bitcoin).

But since most honest miners will report the same bundle of transactions, there will be many “correct” blocks, and only one reward winner. How does the system choose who wins, and how are clever miners prevented from winning every block?

Bitcoin’s consensus design selects a winner pseudo-randomly from among many potential miners by requiring the winning block to meet certain hard-to-predict characteristics. It is by requiring a certain number of prepended zeros in the block hash that the “reward winner” is kept random. This is what is meant when Bitcoin miners are described as playing a “guessing game.”

The screenshot below is taken from a blockchain explorer, a free public service which allows anyone to see all Bitcoin transactions. Note the block hash with 18 prepended zeros, required by the difficulty factor at the time this block was mined:

0000000000000000001fb8f591a114473c582cea6057afd97488cf4f532fc33f

Satoshi Nakamoto set as a constant a 10 minute average block time. This average is maintained by adding or subtracting the number of prepended zeros required in a valid block hash. So while the Bitcoin system has no sense of “Earth time,” it does know when blocks are found too quickly or too slowly, and difficulty will adjust accordingly. For example if a large amount of hashrate left the network, making block production too slow, then the number of prepended zeros required to find a block would drop, making the validation condition easier to satisfy and blocks faster to find.

Unlike block #544937 above, block #0 below only has 10 prepended zeros. Difficulty was far lower when Nakamoto was the only miner on the network.

000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
Once validation criteria are met, the lucky block is propagated around the network and accepted by each full node, and it gets appended to a chain of predecessor blocks; at this time the winning miner is also paid.

Minting bitcoins for block producers
Each time a block is produced and a miner is paid, new bitcoins come into existence. The computer which finds a lucky hash is paid a reward automatically by the network, in Bitcoin. This is called the coinbase reward. Like everyone else, miners must have a public key to receive these funds.

The coinbase reward is cut in half every 210,000 blocks, an event known as halving. Halvings make bitcoin a deflationary currency; eventually the emission rate of bitcoins will drop to zero. Only about 21 million will be created by the network. Miners are theoretically incentivized to continue mining after the reward period ends around the year 2140, because they will continue to receive transaction fees set by the sender of an individual transaction.

In this way, Bitcoin creates its currency through a distributed process, out of the hands of any individual person or group, and requiring intensive computing and power resources.

Turning energy into hashes crystallizes value
As more blocks gets added to the chain, the cost of reverting a past transaction increases, and hence probability of the transactions in the block being finalized increases. Proof-of-Work is cumulative in the sense that with more computing power on the network, it becomes more expensive to attack it, making the ledger more secure.

In Bitcoin’s original whitepaper, Section IV “Proof-of-Work” is written as the following:

“To implement a distributed timestamp server on a peer-to-peer basis, we will need to use a proof-of-work system… Once the CPU effort has been expended to make it satisfy the proof-of-work, the block cannot be changed without redoing the work. As later blocks are chained after it, the work to change the block would include redoing all the blocks after it.”

Conceptually, Proof-of-Work burns energy in block-issuance, which allows network participants to view immutability objectively. Proof-of-Work reduces the entropy level within the system by consuming energy to create machine consensus around an ordered set of transactions. The cost of electricity consumption is borne collectively by miners to find “order” in “chaos” without a central coordinating agent. This is the process through which physical resources (ie., energy) are transformed into digital resources in the form of blocks of transactions, and the coinbase rewards which are the outcome of block production. Because these digital assets (ie., blocks and transactions) are encoded on physical computer memory, it can be said that the Proof-of-Work process sublimates electricity into a physical bearer instrument, similar to the way that gold mining and minting can produce gold coins.

Blocks order transactions
We have said that Bitcoin hashes groups of transactions to create a single, verifiable block. We’ve also said that the blockchain creates a transaction history that cannot be changed without expending enormous amounts of energy. But accomplishing these two feats required some ingenuity on Nakamoto’s behalf.

Bitcoin users exist all over the world, and their individual transactions must travel slower than the speed of light, so latency causes nodes to receive messages at different times, or out of order.

In any financial system, errors in transaction-logging can create disagreements between parties because balances will appear incorrect, or transactions will be missing. If disagreements are constant, the system is not usable. Whether in a paper ledger or a digital database, cheaters or saboteurs who want to erroneously increase their own balance (or simply wreak havoc) need only to change the order of transactions (ie., their timestamp) or delete them outright to cheat other participants.

The practice of “writing” ledger data into a hard-to-alter physical record is at least 30,000 years old, as exemplified by the clay tablets used by the ancient Sumerians used before the development of paper, and the more recent wooden “tally sticks” (seen below) which were still legal tender in the United Kingdom until the 19th century.

Of course, keeping track of changes is no sweat for a spreadsheet on a single computer. When applications span multiple computers, networks are required to carry messages between them. Multi-computer applications deal with slow connections by using asynchronous algorithms, which are tolerant of dropped, latent, or out-of-order messages and are not driven by a time-based schedule. In an asynchronous system, computers engage in parallel processing, but without moving forward in lock-step. Instead, messages (often user actions) trigger a change on each and every machine as it hears about the message.

Nakamoto consensus is highly reliable
Bitcoin too is an asynchronous event-driven system. But unlike conventional distributed systems, participants are not permissioned, meaning they have not been authenticated and authorized prior to participating. Yet somehow they all transition the state of their ledger together without a leader or any sort of coordinating mechanism beyond their own self interest. How can self-interest be used to coordinate a group of disparate, unvetted, and possibly hostile individuals?

One of the many strokes of brilliance in Bitcoin is the use of economic incentives to keep miners producing valid blocks on schedule. Miners earn rewards denominated in the unit of account for the ledger they maintain; that is, in bitcoin. Nakamoto’s conjecture was that the desire to corrupt the ledger, which threatens the coin of the realm, would be outweighed by the desires of those with a vested interest.

This way, miners in a distributed system like Bitcoin can come to agreement about the order of transactions, even if some of the nodes are slow or even maliciously producing invalid blocks. This happens without the restrictive requirements of permissioned consensus.

Bitcoin’s system has shown its resilience in both operational uptime and integrity of the ledger. Importantly, it can accomplish this feat without needing to vet the individual nodes on the network; machines can join or drop off at will, and the properties of the system remain the same.

Industrial mining in a nutshell
Compared to launching an ICO, venture investing, or volatility-trading, a mining operation is the least exposed to capital market “narratives,” making it the most predictable cryptocurrency investment activity. Mining profitability is driven by semiconductor cycles, energy expenditure, and the overall performance of the cryptocurrency market. While a mining investment is fundamentally a long position, it comes with a lower cost basis, so long as a miner optimizes for overhead costs and buys their machines at a fair retail price. A miner’s decisions to buy hardware or support a given network are much less influenced by short term market fashions than on the fundamental qualities of the network protocol, and the technological life cycle of hardware being purchased. Considerations for miners include, but are not limited to, fundamental factors such as:

Choosing a viable network.
Sourcing from the right hardware manufacturers, at a fair price.
Timing the purchase with the hardware cycle.
Cost of energy and other overheads at host facility.
Security and staffing at host facility.
Liquid reward management.
Local regulation and tax.
There are two main main factors driving mining market dynamics: hashrate growth and price movement. Fundamentally the two factors are deeply intertwined. Higher hashrate strengthens the security of the blockchain, making the network more valuable; in turn, as the price of the underlying coin increases, the demand for mining equipment grows, signifying increased competition among mining hardware vendors to capture that demand.

Bitcoin hashrate has been increasing at a breathless pace despite the spot price having been butchered year-to-date. Since January 2018, Bitcoin miners and traders have lived in completely separate universes, with miners reinvesting in hardware and facilities, anticipating the next cycle of price appreciation that is expected to accompany continued engineering progress at the core protocol level. Because miners control liquidity, this amounts to a self-fulfilling prophecy. (An appendix discussing popular conceptions about price trends appears at the end of this paper.)

The mismatch between hashrate growth and price movement is the result of the different paces between hardware markets and capital markets. Under normal circumstances, mining difficulty can be predicted by semiconductor foundry TSMC’s wafer shipments, which account for a majority of Bitcoin ASIC production. Foundry lead times are longer than the Bitcoin price cycle, meaning wafers that are already in production during a downturn in the Bitcoin price would cause capacity to overshoot.

On the other hand, due to the cumulative nature of Proof-of-Work, higher hashrate poured into a network makes the system more secure and robust. A higher degree of finality means the system is more stable to support transaction volume, and more robust for third-party developers to build on the system.

In Proof-of-Work cryptocurrencies, capital markets and distributed networks are tied together by design. As Bitcoin price continuously climbed up over the past decade, mining grew into a huge industry. In the first half of 2018, the largest cryptocurrency ASIC manufacturer Bitmain, reported $2.5 billion in revenue and $1.1 billion in profit.

The rise of specialized hardware
Over the years, cryptocurrency mining has graduated from CPU to GPU to specialized hardware such as FPGA (Field-Programmable Gate Array) and ASICs. Because of the competitive nature of mining, miners are incentivized to operate more efficient hardware even if it means higher upfront cost paid for these machines. As some hardware manufacturers upgrade to faster and more efficient machines, others are forced to upgrade too, and an arms race emerges. Today, for the notable networks, mining is largely dominated by ASICs. Bitcoin’s SHA256d is a relatively simple computation; the job of a Bitcoin ASIC is to apply the SHA256d hash function trillions of times per second, something that no other type of semiconductor can do.

First introduced in the 1980s, ASICs transformed the chip industry. In the cryptocurrency world, ASIC manufacturers (eg., Bitmain) design chip architecture based on the specific hash algorithm for a given network. After going through multiple iterations and tests, the design graphic for the photomask of the circuit is then sent to foundries such as TSMC and Samsung as part of the process known as a tape-out. The actual performance of the chips is not known until the chips return from the foundry. At this point, the ASIC manufacturer needs to optimize for thermal design and chip alignment on the hashing board before the product is ready for production use.

The rise of application-specific hardware is inevitable and a natural trend in the computing hardware evolution. Much like how technology in gold mining and oil drilling developed over time as the base commodities became more and more valuable, application-specific hardware is improving quickly as the result of cryptocurrency becoming more attractive. While short-term price action is mainly driven by speculation and has been observed to decorrelate with hashrate, over the long run the two factors form a virtuous feedback loop.