Messari: A Detailed Explanation of the Data Layer Collaboration Solution Ceramic Network

Original Title: 《Ceramic Network: The Collaborative Data Layer》

Original Author: Eshita Nandini, Messari

Compiled by: Biscuit, Chain Catcher

Summary

• Ceramic is building an on-chain dynamic file storage system aimed at enabling developers to build applications faster and provide richer user experiences.
• The IDX standard is being developed to create user identity shared data tables, allowing users to seamlessly switch between applications in the Ceramic ecosystem.
• Compared to wallet-as-identity, user identity is represented by DID and can be bound to wallet addresses across multiple chains while retaining their identity information.

Data is at the core of internet applications. Today, centralized platforms have absolute power over where data is stored, how it is stored, and who can read or write it. This phenomenon leads to users losing ownership of their personal data, the island effect of web applications, and a poor user experience. Furthermore, this model wastes developers' time.

Self-sovereign data built on blockchain creates a permissionless, composable, and interactive internet. The grand vision of Web3 is to become a shared, decentralized data layer. Ceramic is building an on-chain dynamic file storage system that enables developers to build applications faster and provide richer user experiences.

Ceramic Protocol

Ceramic is developed by 3box Labs, a team focused on identity and development tools for composable networks. Earlier this year, the team completed a $30 million Series A funding round led by Multicoin and Union Square Ventures.

User Experience in Web2

Internet users are accustomed to entering personal data each time they register a new account, and only after spending significant time on a platform do they receive a good personalized experience. Users must re-establish accounts when entering a new social media platform, making it difficult for these platforms to launch smoothly.

Most Web2 applications do not allow users to manage their social graphs, preferences, and other data independently; instead, this data is collected and stored internally by the applications. Centralized user data is highly resource-wasting, as duplicate information about users may be found across multiple platforms. So why not let each application use the same data table?

Because of fierce competition. Facebook may have the richest information about users, making it hard for people to choose to leave the platform or create a monopolistic competitive market. Ultimately, data equals money. For example, Instagram and Facebook offer similar experiences because they use the same data table. TikTok can quickly learn new user preferences during brief usage periods and provide personalized experiences. If Facebook were to open-source user data, it would destroy its digital advertising business, which accounts for 98% of its total revenue. The technical stack structure shows that the platform resembles a fat data layer with a thin application layer.

A quality experience should start the moment a user enters the platform. The power of Web3 lies with users, not centralized companies. The fat protocol paper discusses how a thin application layer realizes the true vision of Web3. Applications built on a shared data layer help create a fair competitive environment while emphasizing the importance of data. Ceramic is the data highway for the next generation of social applications.

Thus, emerging social applications will choose to build on a composable shared user data network, making it easier to incentivize users to join and manage their personal data.

Data Storage Engine

Ceramic is a decentralized data network that aids in achieving Web3's composability. The core functionality of Ceramic is to derive mutable dynamic data from static and immutable data on the storage protocol. This is crucial because user data, especially social data, is highly dynamic.

Data Stream Objects

A Data Stream is an instance of the Ceramic network that is modified when users in the system sign transactions. A data stream is a way to transfer data from the storage end to the request end. In Web2, data streams are used to capture near-real-time data before it is directed to output and use. Data models or collections of data streams are used to represent features on the platform, such as social graphs or user profiles, which are collected to build Ceramic applications. These models allow composable shared data to be compatible with any application in the Ceramic ecosystem. The content within each data stream is arbitrary and can reference content from any other data stream.

Data stream collections track dynamic documents using immutable streamIDs. Each document on the IPFS protocol is bound to an immutable hash (CID). Each data stream is initiated, signed, and anchored to the blockchain by the Ceramic Anchor Service. Ceramic will eventually launch a native token to support the data network. Any changes or updates to a data stream will also trigger signing and anchoring events. The later anchoring service will gradually phase out, and Ceramic will resemble a complete blockchain.

Each data stream forms consensus independently, so to some extent, the behavior of each data stream acts like a ledger. Nodes are responsible for a subset of all data streams.

The Rest of the Ceramic Network

Ceramic is based on the IPFS protocol's distributed file storage protocol, which features a content-based naming system for storing files in its network. Each piece of data on IPFS is static and identified using a content identifier (CID). Although IPFS operates separately from Ceramic, each Ceramic node can connect to IPFS nodes via HTTP.

Specifically, Libp2p and IPLD are important components of Ceramic:

• Libp2p: A modular resource system separated from IPFS that supports peer-to-peer networking and can also be used on the Ethereum network.
• IPLD: The data model developed for IPFS. It makes it easier to develop hash-linked data systems.

IPLD helps optimize data structures to facilitate the formation of data streams (i.e., Ceramic objects). Data streams typically consist of submissions linked through a DAG (Directed Acyclic Graph). Permanent data storage layers like Arweave, Filecoin, and Sia utilize IPFS to ensure data stability before the Ceramic computation data layer is implemented.

Source: @masonnystrom

Scalability of Ceramic

Ceramic is expanding the availability of data. Each user is represented by an account in Ceramic, which is associated with the data streams that represent them. State changes are limited to the account connecting to other independent accounts and transmitting information, without modifying the target account's data.

Each node serves as an execution environment for validating transactions, responsible for handling a subset of account data, ensuring that updates to user data do not require the entire network to update. This means that as the number of nodes increases, the capacity for parallel processing of streams also increases. Nodes can transmit transactions offline and then synchronize with the rest of the network.

Decentralized Identity

Once users have an online identity, they can directly control their personal data on-chain. Decentralized identity refers to the ability of users to operate independently of any centralized product while displaying personal identifiers. Ceramic adopts the DID standard developed by W3, which is an identifier used to distinguish personal identification from any centralized system.

In the Ceramic system, each account is associated with a Decentralized Identifier (DID). Accounts can own data streams and sign transactions to those streams. Unlike wallets as identity, accounts represent users and can be responsible for multiple wallet addresses across multiple chains. DIDs are used to log into Ceramic applications. Each transaction or update to a data stream is authenticated by the user's (account's) DID. This allows users to manage their data independently of any server or application.

On top of DID, Ceramic has developed the IDX standard to aggregate various cross-chain data types associated with DID-related user data. IDX is a user-centric decentralized application user table alternative designed to make it easier for developers to store and modify user data in a readable manner. This standard is central to developers building on the Ceramic network and alleviates the burden of managing internal user registries for applications. In summary, IDX helps associate any profile, account, or generic data with a unique DID (or user).

Additionally, every operation (transaction) on Web3 applications requires user approval, which can become cumbersome. Every piece of data written to Ceramic needs to be signed by a DID, making the user experience smoother.

Source: Ceramic Network

Innovations of Ceramic

Ceramic provides developers with an ecosystem that reduces the time required to develop databases and functionalities. Developers need access to DID data storage and the location of information related to each DID when building user data. This capability based on pre-existing dynamic data tables allows developers to quickly meet customer expectations and understand user preferences.

Ceramic also offers a data model marketplace that allows developers to combine application functionalities to create applications faster. As mentioned earlier, data models are collections of data streams that build a feature on the application, such as an article or profile. Once a data model is deployed, developers can publish it to Ceramic for use by other teams.

For users, composable data allows them to migrate personal information freely and enter more personalized network platforms. Meanwhile, this shared data layer means developers can focus on application development rather than data handling.

Drawbacks of Distributed Open Source Storage

To achieve secure, self-sovereign data, open-source storage may not be the best solution. Although Ceramic provides dynamic updating and maintenance capabilities for data, the data is not privatized. To ensure data security, public encrypted networks are not the best solution, and users need to be cautious about what information they store on networks like Ceramic. While we have become accustomed to storing personal data in social media applications, this is already an insecure method of data storage.

To build applications on Ceramic, users must use reliable community-hosted nodes. As Moxie pointed out, most users are not keen on doing this. Although Ceramic provides users with a way to manage online data, a significant portion of the 5 billion internet users today is less willing to use it.

Reflection

Among the numerous DeFi innovations, few projects provide developers with the ability to easily build applications. Ceramic simplifies the process of building decentralized data for developers by managing state transitions, associating that data with data structures, and storing it for users through its IPFS data layer. As one of the earliest infrastructures in the Web3 social space, Ceramic can offer a good experience for both developers and users.