A Detailed Explanation of CESS: Building a New Paradigm for Decentralized Storage Ecosystem

Defects of Web2 Storage and the Necessity of Decentralized Storage Systems

Centralized storage based on internet cloud has always been the mainstream storage model at this stage, providing necessary support for the operation of the traditional internet system. Internet technology giants have also become the biggest commercial beneficiaries due to extreme market demand. Although the centralized content storage model can ensure relative efficiency through centralized allocation, its fault tolerance rate is generally low. Failures in a single storage facility caused by human factors or natural disasters can lead to unavoidable, even fatal, economic losses.

Typically, this model can also lead to unfair phenomena due to factors such as politics and war. For example, during the Russia-Ukraine war, overseas cloud service providers refused to serve many Russian tech companies. The centralized storage model also faces security issues; for instance, Facebook has been accused of leaking personal information of over 533 million Facebook users from 106 countries/regions, further raising concerns among tech companies about data security issues with service providers. Additionally, some service providers with outstanding storage capabilities often monopolize the market, resulting in high service prices.

Thus, the drawbacks of the existing Web2 storage system highlight the urgent need for decentralized storage. By adopting a more distributed storage approach, the fault tolerance rate of the storage system can be further reduced. Connecting users' idle storage resources to the network will also lower data storage costs and enhance the utilization of idle resources.

Currently, well-known decentralized storage systems such as IPFS, Arweave, and Sia are advancing this field. Particularly, Filecoin, which introduced a blockchain incentive layer to IPFS, further boosted the popularity of decentralized storage after its mainnet launch in 2020. Overall, blockchain mainly plays a role in incentivizing more people to connect to the network and become nodes, helping the system gain storage capacity, with these systems typically positioned as decentralized storage.

The application of various technologies should be based on efficiency and security. Although the existing Web2 storage methods are overly centralized, they at least meet the basic needs of most users. However, the current decentralized storage systems struggle to balance decentralization, security, and efficiency. For example, Filecoin often faces issues of low storage efficiency and supply-demand mismatch due to unreasonable matching mechanism design. Similarly, the vast majority of decentralized storage systems also have bottlenecks in storage security, as content is usually stored on specific nodes (building replicas requires additional payment), and the security threshold depends on the security of the nodes themselves. Therefore, constrained by the drawbacks of security, efficiency, and functionality, there are few demanders willing to try.

We see that the emerging decentralized distributed storage network Cumulus Encrypted Storage System (CESS) is innovating the existing decentralized storage system's consensus mechanism and technology to further provide secure, high-performance, and unlimited cloud data storage services to meet the needs of various storage users, including individuals and enterprises. At the same time, CESS intends to build a high-performance public chain based on storage to serve developers and creators, constructing a Web3 ecosystem aimed at applications. This article will interpret CESS to help more readers understand and recognize the CESS system, thereby better disseminating distributed storage technology.

Detailed Explanation of CESS Decentralized Cloud Storage

CESS is a multi-layered architecture blockchain system developed based on the Substrate open-source framework, serving as the infrastructure for decentralized cloud storage networks. CESS adopts a random selection rotating consensus node mechanism (R²S) to coordinate network resources and loads, and employs multi-replica recoverable storage proof (PoDR²), multi-type data rights confirmation (MDRC), decentralized proxy re-encryption, and other mechanisms and technologies to ensure data integrity, security, traceability, and privacy. CESS not only enhances the performance and security of the existing distributed storage system but also meets the requirements of "information on-chain" as much as possible, utilizing efficient on-chain processing to avoid the inefficiencies brought by decentralization, achieving a good balance between "decentralization" and "efficiency."

Multi-layered Architecture (Brief Structural Introduction)

CESS is a multi-layered architecture system that includes a blockchain service layer, distributed storage resource layer, distributed content distribution layer, and application layer. Currently, the vast majority of storage networks positioned as distributed storage do not include an application layer.

The blockchain service layer in the CESS system includes functions related to consensus algorithms, storage proofs, payments, and incentives. In addition to being a storage system, CESS is also a decentralized storage ledger that records the entire network's storage space and metadata of stored content. Nodes completing CESS block packaging tasks need to reasonably allocate network storage resources based on supply and demand factors, in addition to basic tasks.

Thus, the blockchain service layer mainly plays a coordinating and consensus role in this process, acting like a commander in an army, continuously coordinating and dispatching within the system to ensure the system operates efficiently and securely, such as in storage resource allocation, random selection of consensus nodes (R²S), incentive distribution coordination, and rewards and penalties.

The content distribution layer and storage resource layer mainly consist of four types of nodes, which are like an army ready to be deployed within the CESS system. The storage resource layer includes storage nodes and consensus nodes, while the content distribution layer includes cache nodes and retrieval nodes. Storage nodes obtain data allocation based on the storage proof mechanism in the system (proving their storage capacity) and pool technology (detailed later), while cache nodes and retrieval nodes can be understood as a decentralized CDN (Content Delivery Network) to further enhance the efficiency of content retrieval and distribution of popular information.

The work of the aforementioned nodes will be packaged into blocks after the consensus nodes are selected through the rotating consensus node mechanism (R²S), recording on-chain transactions and achieving consensus across the network. The service layer, distribution layer, and storage resource layer will be jointly maintained by the four types of nodes mentioned above.

The application layer is similar to existing Layer 1 systems that support smart contracts, providing developers with APIs, SDKs, and other toolkits (and will support WASM and EVM compatibility in the future), supporting the construction of various Web3 and Web2 applications. The Substrate framework will provide CESS with more cross-chain support, allowing not only developers to build applications but also supporting the migration of other on-chain DApps. By further empowering the storage system, it prepares for the explosion of the Web3 commercial system.

Consensus Mechanism

Typically, the consensus mechanism of a blockchain system plays a decisive role in on-chain efficiency. Although Bitcoin's POW can provide sufficient security for the chain due to a sufficient number of decentralized nodes, proof of work itself is an inefficient method and has a high threshold for node requirements.

CESS adopts a random selection rotating consensus node mechanism (R²S), which draws on POS and DPOS, selecting consensus nodes participating in block production in a randomly non-fixed manner, and employs a more secure Byzantine fault tolerance and verifiable random function to ensure consensus security.

Users who wish to become candidate nodes based on the R²S consensus mechanism (anyone can join) must first stake a certain amount of $CESS (governance token), which serves as an economic threshold against malicious behavior. The R²S consensus mechanism will select a window period (e.g., 10,000 blocks), and within each time window, only 11 formal rotating nodes will be selected from the candidate nodes to participate in block production. Candidate nodes that do not participate in block production can also prove their work capacity by participating in data preprocessing (sharding, encryption, redundancy) processes, thus participating in the next round of formal rotating node selection. (The CESS system will score the work of elected rotating nodes; those with too low a score will not be able to participate again).

During the rotation of the nodes, the network will randomly select 11 nodes from the eligible candidate nodes to serve as the formal rotating nodes for consensus maintenance in the next time window, and so on. If candidate nodes experience delays in block production, do not respond, or engage in malicious behavior, they will be economically penalized by the system and randomly replaced among the candidate nodes. By introducing threshold signature algorithms and the PBFT consensus algorithm of the collector role, along with random numbers, the system's security can be further ensured.

R²S inherits the efficiency of DPOS while endowing the network with decentralization characteristics based on randomness, achieving low gas fees and fast transaction processing throughput (10,000 TPS) while ensuring equal opportunities for various miner nodes to participate in the network.

On one hand, it separates consensus from storage, solving the long-term monopoly and inaction of large nodes. On the other hand, the efficiency of CESS consensus further guarantees the efficiency of putting metadata on-chain and can directly ensure the authenticity of data through on-chain addressing (only when there are issues with off-chain data will on-chain addressing be performed), reducing the cost of on-chain data requests and improving efficiency, thus greatly enhancing efficiency under a decentralized framework and achieving on-chain transaction processing. Currently, the vast majority of distributed storage systems place more storage data off-chain, which poses certain data security risks.

The on-chain efficiency and scalability are also the guarantees for the future ecosystem as the underlying layer of Web3.

Multi-Replica Recoverable Storage Proof (PoDR²)

In addition to pooling technology, CESS also relies on Multi-Replica Recoverable Storage Proof (PoDR²) to further ensure the security of system storage. When users upload data, the data will be automatically copied three times (the number of copies can be customized), and auxiliary verification metadata required for generating recoverable proof for each data replica will be created and stored in the blockchain system.

Candidate nodes in the system will preprocess the data, such as sharding and redundancy, and further distribute it to storage nodes for storage. In addition to needing to report their stored data within an effective period for the CESS system to confirm whether the data is damaged, the PoDR² mechanism will statistically monitor and check all data segments that make up a single file (including all replicas) as a whole. Once a data segment is identified as damaged, CESS will automatically generate a new data segment as a supplement and send it to a new storage miner, ensuring the recoverability of replicas and enhancing the robustness of data storage in the system.

Therefore, even if some storage miners lose storage resources due to uncontrollable factors, the system guarantees data integrity.

Currently, Filecoin only uploads a single replica to the storage node when storage users have no special requirements (replica protection requires further payment), and the loss of that node's data will lead to the permanent loss of that content. Thus, from a security perspective, the moat of distributed storage systems like Filecoin lies in the security of storage nodes, while CESS employs multiple defenses to enhance fault tolerance.

The general storage process of the CESS system is as follows:

1. The CESS client publishes the data uploaded by the user and preprocesses it through the multi-format data rights confirmation mechanism (MDRC), including data metadata extraction and data fingerprint extraction (features), and publishes it in the CESS system.

2. Conduct replica copying (usually three copies) and perform sharding and redundancy.

3. For each data segment, generate verification parameters for auxiliary storage proof for subsequent verification.

4. The responsible scheduling node randomly allocates the data segments to storage miners.

5. Through Multi-Replica Recoverable Storage Proof (PoDR²), periodically check the data of storage miners to ensure data availability.

As an emerging distributed storage system, CESS has improved upon the existing distributed storage system, enhancing security and efficiency. More importantly, it also possesses rights confirmation and anti-counterfeiting functions, which are not available in older storage systems like Filecoin.

Smart Cloud Space Management

Filecoin is more like a distributed P2P storage system that can establish market matching between storage demanders and storage resources. For example, for a storage demand of a 3T file, the system will allocate it to storage miners capable of providing more than 3T of storage after encryption. However, when the storage demand of the demander is too large, it will give an advantage to storage miners with large storage capacities. This can lead to many storage resources not being fully utilized, especially among smaller storage miners.

CESS's pooling technology is more advanced. CESS pools storage supply in the system through virtualization technology, building a "storage pool" based on storage resources, providing a unified, on-demand storage method for top-level applications or external applications, shielding the instability caused by underlying hardware differences. The "pooling" technology effectively utilizes idle resources, maximizing utilization efficiency.

Any storage node that can provide effective storage space, data volume, traffic contribution, and operational performance can join the pool. When users upload data, consensus nodes will first encrypt the data (using trusted execution environments for encryption), perform sharding and redundancy, and other preprocessing (decentralized proxy re-encryption mechanism). The processed data will then randomly select miners that meet the user's storage requirements (e.g., storage duration) for storage. This means that any storage node has the opportunity to store a part of the file to be stored, and large storage miners no longer have the advantage of quantity.

Not only can small storage miners gain profits, but since CESS's storage miners only need to execute storage tasks without complex professional operations, this also optimizes the way more idle storage resources can earn profits, further making the overall storage ecosystem of CESS more decentralized, distributed, and secure.

The CESS network pools global node space, forming a "decentralized cloud storage pool," providing users with an on-demand cloud storage service model. CESS sets up scheduling services in the network space to intelligently allocate users' data to global nodes, maximizing and effectively utilizing storage space, achieving smart cloud space management.

Privacy and Rights Confirmation

Privacy and rights confirmation are areas that are still not addressed in the current storage system, especially after the explosion of the NFT track, where data rights confirmation is particularly important. We see that currently, during the preprocessing phase of candidate nodes for data, CESS employs a multi-type data rights confirmation mechanism (MDRC) technology to extract data fingerprints from each data file to generate data certificate IDs. This means that these contents will be confirmed in the system through features, solving the difficulties of data storage and rights confirmation during data circulation, starting rights confirmation from the data source, and visualizing the data lineage through a blockchain explorer. For example, after NFTs are stored through CESS, they will achieve rights confirmation on the CESS chain for the corresponding content, further enhancing the absolute rights confirmation characteristics of NFTs.

Especially in the Web2 domain, the lack of copyright protection for traditional UGC has led to rampant content duplication. With the popularization of CESS as a storage method, it is expected to be reshaped.

Moreover, through the decentralized proxy re-encryption mechanism, data holders can interact in a decentralized manner between target users without disclosing data content, a feature not available in existing distributed storage systems. This also provides protection for users' data security and rights, ensuring safeguards in a Web3.0 manner.

Distribution

The demand for storage resources is enormous. In addition to storage, data providers may also continuously extract the content they have stored. Typically, after retrieving data through retrieval miners, storage miners will upload the stored data to provide it to users. However, in the Filecoin system, storage miners tend to download user-stored data based on storage proofs rather than uploading it when users request that data. The reason is that the network cost required for uploading is relatively high, leading to actual network unavailability. Traditional cloud services use CDNs to distribute the data stored by storage providers, but some distributed storage systems like Filecoin do not have such functionality, which is one reason many people are reluctant to try distributed storage systems.

CESS will further increase the number of cache nodes based on retrieval nodes, which will cache popular data to achieve faster call speeds, ensuring quick data retrieval and delivery. Thus, cache nodes and retrieval nodes together form a distributed CDN, dedicated to rapid data indexing and return. With the overall efficiency improvement, CESS provides the necessary foundation for multi-tier users to store data in a distributed manner.

Underlying Web3 Applications

Currently, the vast majority of storage systems do not support further expansion because they do not support smart contracts. This means that distributed storage technologies like IPFS are facing narrowing commercial paths as storage cannot gain further recognition.

CESS, based on its random selection rotating consensus node mechanism (R²S), endows the chain with high scalability and efficiency while supporting smart contracts. This means CESS can support large-scale commercial applications and is the first decentralized storage protocol in the industry designed to support large-scale commercial applications natively.

In addition to its storage advantages, CESS, developed based on the Substrate framework, not only supports WASM but will also be compatible with EVM, providing modular development tools and services such as APIs and SDKs for application developers, allowing them to build and launch Dapps/Apps directly on-chain. The full-stack solution and infinite scalability will provide convenient development and operational conditions for developers.

In terms of application scenarios, decentralized distributed cloud storage, NFTs, DeFi, the metaverse, blockchain games, audio and video, supporting streaming media and data, data factor trading markets, and Web3.0 storage can all easily deploy applications using the CESS network, which has the potential to build a rich and powerful application ecosystem.

Additionally, the storage public chain CESS has cross-chain capabilities, allowing on-chain assets from other ecosystems to cross-chain into CESS, providing rich asset liquidity and diversity for the ecosystem.

Therefore, from the perspective of applications, the CESS ecosystem will be widely applicable to most commercial purposes.

Overall, CESS restructures the distributed storage system through the rotating consensus node mechanism (R²S), on-chain storage resources, "pooling" technology, and Multi-Replica Recoverable Storage Proof (PoDR²), creating a distributed storage system capable of supporting large-scale commercial use and building a high-performance public chain system widely applicable to most commercial purposes. CESS has constructive significance for both the storage field and the blockchain field.

Moreover, with the promotion of CESS, more Web3 applications will be able to obtain resources through Web3 infrastructure, breaking free from dependence on the Web2 world and accelerating the development pace of the Web3 world.