Blockchain companies use sharding to partition databases and increase scalability, allowing them to process more transactions per second. Sharding is a key mechanism underneath the Ethereum Blockchain and one of its critical components. Indeed, sharding enables Blockchain protocols to overcome the Scalability Trilemma (as a Blockchain grows, it stays scalable, secure, and decentralized).
The Scalability Trilemma (Image Credit: vitalik.cal).
Blockchain is now used in everything from supply chain management to cross-border financial transactions.
While extremely versatile, one critical issue remains: scalability. This issue is already prevalent with cryptocurrencies such as Bitcoin and Ether where more computers joining a peer-to-peernetwork cause the entire system to degrade.
For fintech companies that must compete with rapid payment networks, a lack of scalability is a particular concern. How can Blockchain be a viable solution in this industry? How can scalability and throughput be increased to address latency concerns?
Developers think sharding is the answer. To increase transactional throughput, sharding describes the partitioning of computational and storage workload across a P2P network. This avoids a situation where a single node is responsible for processing the entire transactional load of a network.
Sharding can help Blockchain provide all three (scalability, security and decentralization).
How is the workload partitioned?
Sharding uses horizontal partitioning, a database architecture enabling businesses to scale their databases dynamically.
Each database table is separated into multiple tables by partitioning its rows. Data held in each partition is independent of other partitions and is unique, but the schema and columns of each partition remain the same.
Individual partitions are called logical shards, which are distributed across separate database nodes called physical shards. Each physical shard can hold multiple logical shards. Collectively, the shards comprise an entire logical dataset.
It’s important to note that database shards typify a shared-nothing architecture. Shards are autonomous in the sense that they do not share computing resources or the same data with other shards. However, shards can share information with other nodes if required. This maintains a decentralized ledger where all ledger entries can be viewed without every shard having to process and store all information.
Strengths and weaknesses of sharding
Strengths
Sharding allows horizontal scaling, or the act of adding machines to a stack to spread out the load and improve processing times. This method of scaling is preferable to vertical scaling, which involves adding more CPU or RAM to an existing server. In theory, horizontal scaling allows the network to be scaled up indefinitely. On the other hand, vertical scaling is inevitably constrained by the capabilities of the server.
Sharding also decreases query response times. Non-sharded database queries have to search through every row in the table to find the result. This process can become extremely slow in large, monolithic applications. Through horizontal partitioning, queries search fewer rows and results are returned more quickly.
Weaknesses
Implementing a sharded database architecture is quite complex. Mistakes during implementation can lead to corrupted tables and loss of data. Even when implemented successfully, sharding can be disruptive to some teams. Instead of managing data from a single entry point, individuals must manage data across several locations.
Sharded databases are almost impossible to return to an unsharded state. Backups of the database made before sharding won’t include any data written after partitioning. Returning to an unsharded architecture involves the combining of partitioned data with older backups or creating a single database from many partitioned databases. Both are expensive and time-consuming operations.
Key takeaways:
Sharding is the process of increasing the scalability of blockchain networks.
Sharding separates database tables into horizontal partitions called logical shards. These shards are autonomous and do not share resources or data. Collectively, they comprise an entire data set and can share uncommon information when required.
Sharding enables computing power to be increased by adding more machines to a stack. Partitioning also increases query response times as fewer rows need to be searched. However, implementing a sharded database is complex and is difficult to reverse.
Web3 describes a version of the internet where data will be interconnected in a decentralized way. Web3 is an umbrella that comprises various fields like semantic web, AR/VR, AI at scale, blockchain technologies, and decentralization. The core idea of Web3 moves along the lines of enabling decentralized ownership on the web.
A blockchain protocol is a set of underlying rules that define how a blockchain will work. Based on the underlying rules of the protocol it’s possible to build a business ecosystem. Usually, protocol’s rules comprise everything from how tokens can be issued, how value is created, and how interactions happen on top of the protocol.
In software engineering, a fork consists of a “split” of a project, as developers take the source code to start independently developing on it. Software protocols (the set of rules underlying the software) usually fork as a group decision-making process. All developers have to agree on the new course and direction of the software protocol. A fork can be “soft” when an alteration to the software protocol keeps it backward compatible or “hard” where a divergence of the new chain is permanent. Forks are critical to the development and evolution of Blockchain protocols.
A Merkle tree is a data structure encoding blockchain data more efficiently and securely. The Merkle tree is one of the foundational components of a Blockchain protocol.
The nothing-at-stake problem argues that validators on a blockchain with a financial incentive to mine on each fork are disruptive to consensus. Potentially, this makes the system more vulnerable to attack. This is a key problem that makes possible underlying blockchain protocols, based on core mechanisms like a proof-of-stake consensus, a key consensus system, that together the proof-of-work make up key protocols like Bitcoin and Ethereum.
A 51% Attack is an attack on the blockchain network by an entity or organization. The primary goal of such an attack is the exclusion or modification of blockchain transactions. A 51% attack is carried out by a miner or group of miners endeavoring to control more than half of a network’s mining power, hash rate, or computing power. For this reason, it is sometimes called a majority attack. This can corrupt a blockchain protocol that malicious attackers would take over.
A Proof of Work is a form of consensus algorithm used to achieve agreement across a distributed network. In a Proof of Work, miners compete to complete transactions on the network, by commuting hard mathematical problems (i.e. hashes functions) and as a result they get rewarded in coins.
An Application Binary Interface (ABI) is the interface between two binary program modules that work together. An ABI is a contract between pieces of binary code defining the mechanisms by which functions are invoked and how parameters are passed between the caller and callee. ABIs have become critical in the development of applications leveraging smart contracts, on Blockchain protocols like Ethereum.
A Proof of Stake (PoS) is a form of consensus algorithm used to achieve agreement across a distributed network. As such it is, together with Proof of Work, among the key consensus algorithms for Blockchain protocols (like the Ethereum’s Casper protocol). Proof of Stake has the advantage of security, reduced risk of centralization, and energy efficiency.
Proof-of-Activity (PoA) is a blockchain consensus algorithm that facilitates genuine transactions and consensus amongst miners. That is a consensus algorithm combining proof-of-work and proof-of-stake. This consensus algorithm is designed to prevent attacks on the underlying Blockchain.
According to Joel Monegro, a former analyst at USV (a venture capital firm) the blockchain implies value creation in its protocols. Where the web has allowed the value to be captured at the applications layer (take Facebook, Twitter, Google, and many others). In a Blockchain Economy, this value might be captured by the protocols at the base of the blockchain (for instance Bitcoin and Ethereum).
A Blockchain Business Model is made of four main components: Value Model (Core Philosophy, Core Value and Value Propositions for the key stakeholders), Blockchain Model (Protocol Rules, Network Shape and Applications Layer/Ecosystem), Distribution Model (the key channels amplifying the protocol and its communities), and the Economic Model (the dynamics through which protocol players make money). Those elements coming together can serve as the basis to build and analyze a solid Blockchain Business Model.
Blockchain companies use sharding to partition databases and increase scalability, allowing them to process more transactions per second. Sharding is a key mechanism underneath the Ethereum Blockchain and one of its critical components. Indeed, sharding enables Blockchain protocols to overcome the Scalability Trilemma (as a Blockchain grows, it stays scalable, secure, and decentralized).
A decentralized autonomous organization (DAO) operates autonomously on blockchain protocol under rules governed by smart contracts. DAO is among the most important innovations that Blockchain has brought to the business world, which can create “super entities” or large entities that do not have a central authority but are instead managed in a decentralized manner.
Smart contracts are protocols designed to facilitate, verify, or enforce digital contracts without the need for a credible third party. These contracts work on an “if/when-then” principle and have some similarities to modern escrow services but without a third party involved in guaranteeing the transaction. Instead, it uses blockchain technology to verify the information and increase trust between the transaction participants.
Non-fungible tokens (NFTs) are cryptographic tokens that represent something unique. Non-fungible assets are those that are not mutually interchangeable. Non-fungible tokens contain identifying information that makes them unique. Unlike Bitcoin – which has a supply of 21 million identical coins – they cannot be exchanged like for like.
Decentralized finance (DeFi) refers to an ecosystem of financial products that do not rely on traditional financial intermediaries such as banks and exchanges. Central to the success of decentralized finance is smart contracts, which are deployed on Ethereum (contracts that two parties can deploy without an intermediary). DeFi also gave rise to dApps (decentralized apps), giving developers the ability to build applications on top of the Ethereum blockchain.
The history of Bitcoin starts before the 2008 White Paper by Satoshi Nakamoto. In 1989 first and 1991, David Chaum created DigiCash, and various cryptographers tried to solve the “double spending” problem. By 1998 Nick Szabo began working on a decentralized digital currency called “bit gold.” By 2008 the Bitcoin White Paper got published. And from there, by 2014, the Blockchain 2.0 (beyond the money use case) sprouted out.
An altcoin is a general term describing any cryptocurrency other than Bitcoin. Indeed, as Bitcoin started to evolve since its inception, back in 2009, many other cryptocurrencies sprouted due to philosophical differences with the Bitcoin protocol but also to cover wider use cases that the Bitcoin protocol could enable.
Ethereum was launched in 2015 with its cryptocurrency, Ether, as an open-source, blockchain-based, decentralized platform software. Smart contracts are enabled, and Distributed Applications (dApps) get built without downtime or third-party disturbance. It also helps developers build and publish applications as it is also a programming language running on a blockchain.
An imaginary flywheel of the development of a crypto ecosystem, and more, in particular, the Ethereum ecosystem. As developers join in and the community strengthens, more use cases are built, which attract more and more users. As users grow exponentially, businesses become interested in the underlying ecosystem, thus investing more in it. These resources are invested back in the protocol to make it more scalable, thus reducing gas fees for developers and users, facilitating the adoption of the whole business platform.
Solana is a blockchain network with a focus on high performance and rapid transactions. To boost speed, it employs a one-of-a-kind approach to transaction sequencing. Users can use SOL, the network’s native cryptocurrency, to cover transaction costs and engage with smart contracts.
In essence, Polkadot is a cryptocurrency project created as an effort to transform and power a decentralized internet, Web 3.0, in the future. Polkadot is a decentralized platform, which makes it interoperable with other blockchains.
Launched in October 2020, Filecoin protocol is based on a “useful work” consensus, where the miners are rewarded as they perform useful work for the network (provide storage and retrieve data). Filecoin (⨎) is an open-source, public cryptocurrency and digital payment system. Built on the InterPlanetary File System.
BAT or Basic Attention Token is a utility token aiming to provide privacy-based web tools for advertisers and users to monetize attention on the web in a decentralized way via Blockchain-based technologies. Therefore, the BAT ecosystem moves around a browser (Brave), a privacy-based search engine (Brave Search), and a utility token (BAT). Users can opt-in to advertising, thus making money based on their attention to ads as they browse the web.
Uniswap is a renowned decentralized crypto exchange created in 2018 and based on the Ethereum blockchain, to provide liquidity to the system. As a cryptocurrency exchange technology that operates on a decentralized basis. The Uniswap protocol inherited its namesake from the business that created it — Uniswap. Through smart contracts, the Uniswap protocol automates transactions between cryptocurrency tokens on the Ethereum blockchain.
Gennaro is the creator of FourWeekMBA, which reached about four million business people, comprising C-level executives, investors, analysts, product managers, and aspiring digital entrepreneurs in 2022 alone | He is also Director of Sales for a high-tech scaleup in the AI Industry | In 2012, Gennaro earned an International MBA with emphasis on Corporate Finance and Business Strategy.
