A Review of the Development History of Filecoin

This article is translated from the latest blog of the Filecoin Protocol Lab on October 29, 2020.

After years of work, the Filecoin mainnet is officially live! As we shared last week, the mainnet launch transition is a remarkable milestone for the entire community. Now, over 100 useful applications and tools have been built on Filecoin, with up to 600 TiB of storage orders putting Filecoin's storage capacity into practical use, and the total storage capacity of the Filecoin network is growing at a rate of 500 TiB per hour to nearly 700 PiB!

In addition to the incentive mechanism that rewards high-quality, long-term data storage on the Filecoin network, what sets Filecoin apart is the amazing ecosystem integrated with a decentralized storage layer. From the ConsenSys toolkit used for the Filecoin network, to the Filecoin Archives curated by the Internet Archive, to the builders in the entire IPFS community, it is the investment and assistance of these outstanding and impressive community participants that make Filecoin great. We thank you for that!

The launch of the Filecoin mainnet is a highly anticipated release—a giant leap for the Web3 ecosystem. Filecoin brings a verifiable storage layer to decentralized networks, adding an important storage incentive structure for many Web3 projects using the InterPlanetary File System (IPFS), including IPFS, libp2p, IPLD, and Multiformats. As with any launch moment, it is easy to forget where things came from—the tremendous progress and innovation needed to go from "impossible" to "live!" Let’s take a look back seven years ago, when the idea of Filecoin first took shape…

01

In the Beginning

Back in 2014, when the idea of Filecoin first emerged, "Web3" was still in its infancy. Vitalik Buterin had just released the Ethereum white paper, and "blockchain" was just beginning to attract a large following. When Gavin Wood published "What Dapps Will Look Like: Web 3.0," he outlined four components of the next generation of the Web: "static content publishing, dynamic information, trustless transactions, and integrated user interfaces." This laid the groundwork for new projects to meet these needs.

Since the emergence of Plan 9 at Bell Labs in the 1980s, computer scientists have been pursuing distributed operating systems, and the seamless integration of cryptography, blockchain, and git-like content addressing technologies has made it possible to create a verifiable, reliable, distributed storage network composed of hundreds of independent, untrusted peer nodes that can collectively help preserve humanity's most important knowledge. As early as 2014, protocols such as IPFS (content-addressed file sharing), libp2p (peer-to-peer networking), and IPLD (hash-linked data structures), which form the foundation of the Filecoin data storage network, were just getting started.

It all began with a dream: what if we could use the tools from the early '90s P2P revolution to prevent disasters like the burning of the Library of Alexandria? The loss of generations of knowledge could hinder centuries of progress and innovation. As more and more people live their lives online, this has become an increasingly significant risk. If we could make knowledge more accessible, efficient, and resilient, we could lay a better foundation for our future ideas and discoveries—scientific research relies on the accurate data needed for reanalysis or reproduction, especially when centralized data storage providers cannot take away or shut down the data tools we need, and the entire interconnected knowledge graph—network—is redundantly preserved and connected to thousands of independent network participants.

In many ways, this vision can be traced back to the establishment of the internet—a network composed of decentralized but interconnected sites and routers, with no single point of failure. In the early 2010s, the initial conception of the network made concessions in favor of centralized powers around dynamically personalized content that collected user data to provide targeted advertising. These were not many small interconnected sites, but rather giants like Google, Facebook, Twitter, and Amazon. At the 2016 Decentralized Web Summit, Vint Cerf (one of the inventors of TCP/IP), Tim Berners-Lee (the creator of the World Wide Web), and Juan Benet (the creator of IPFS and Filecoin) came together to "radically reinvent the core technologies that underpin the web" (Clint Finley, WIRED). Filecoin, IPFS, libp2p, and similar protocols emerged as a new generation of faster, safer, and more open tools, beginning to reclaim the Internet.

02

Achieving Launch: All Systems Go

Filecoin's bold mission is to create a decentralized, efficient, and robust foundation for human information. Achieving launch, while just a step on this journey, is a tremendous accomplishment that required years of effort from a large community.

In the early years, the foundation for Filecoin was laid by building the web3 technology stack that most projects in the ecosystem now use. The core building blocks of web3 that Filecoin and other projects use already existed—such as libp2p (a flexible peer-to-peer networking layer), IPLD (a data model with hash links), and IPFS (a data exchange protocol for content-addressable file sharing). These modular projects help make the web3 ecosystem more interoperable, collaborative, and interconnected.

Long before we pioneered the unique and challenging research questions around Filecoin's proof of replication and proof of spacetime, the community was building tools and protocols to enable us to refine and improve the interoperable web3 technology stack to support the new InterPlanetary File System. A passionate community and ecosystem formed around these projects, bringing together like-minded collaborators to turn challenging ideas into reality. At the core of these communities are shared values that we strive to encode into the protocols we are building, such as transparency, openness, modularity, interoperability, and individual agency.

Since then, we have learned a valuable lesson: cryptography, P2P, and economic systems are incredibly difficult! Over the years, we have been amazed by the remarkable contributions and advancements made in cutting-edge research within these ecosystems (driving verification latency functions, random beacons, P2P network design, etc.), extensive designs and analyses of cryptoeconomics, and advanced systems engineering. We are fortunate to tackle many valuable problems in such a fertile environment—from building test environments to help simulate and verify network performance in distributed systems at scale, to creating Drand to provide production-grade distributed random beacons. These obvious ancillary requirements ultimately became the most critical projects to ensure we create a decentralized, robust, and efficient foundation for human information.

Over the past three years, we have formed a not-so-secret superpower—a vibrant and thriving community working towards the success of Filecoin. The three core pillars of Filecoin's strength are the miners, developers, and storage clients that power the market.

03

Mining

In addition to being the core operators of the Filecoin network, the mining community has been dedicated to refining and improving Filecoin software such as Lotus and go-filecoin since 2019. Miners have been the first core participants since the open development network of Filecoin was launched, from the initial launch in February 2019, to the mature Lotus testnet by December 2019, to the Calibration network in August and the Space Race, where we conducted comprehensive stress tests on the network. During this time, miners submitted thousands of detailed issue reports, spent hours discussing and debugging issues on Slack and Zoom, and submitted important patches, features, and optimizations upstream for the benefit of the entire mining community and ecosystem.

Miners have also become huge advocates for each other and the network. Throughout the Space Race, many community champions stepped up to help new miners, answer common questions, and debug network issues. These miners are superstars, but as co-owners of the network, it is also in the best interest of all miners to help manage network value and maximize the operational success of miners. Programs like the FIP process, bug bounty programs, and development rewards are common ways for miners to reinvest their expertise to continue improving Filecoin. Currently, the Filecoin improvement process has invested significant energy (with over 30 comments on the recently released FIP-4 and new proposals being released weekly), and the community website for security analysis and reporting at security.filecoin.io is continuously growing.

Filecoin represents a whole new type of blockchain mining. In addition to maintaining consensus and state updates for the public blockchain, Filecoin miners are also network storage service providers who provide long-term service quality to network clients in exchange for rewards. The cryptoeconomics of Filecoin rewards reliable and useful storage services—motivating a strong collaborative mining community to continuously improve Filecoin and onboard more storage applications and users for many years to come.

04

Building the Network

The network of developers, entrepreneurs, and builders is the heartbeat of the Filecoin ecosystem. The Filecoin ecosystem boasts over 90 meaningful collaborations across various applications, clients, developer tool providers, infrastructure services, and more. Over 250 new teams have entered the ecosystem through Filecoin Ignite to learn, build, and launch applications on the network. The future of Filecoin and web3 lies in the hands of these builders, and we are very optimistic that the next wave of incredible applications and use cases will emerge from their creativity and hard work.

The driving force behind decentralized applications built on Filecoin comes from the thriving IPFS ecosystem. For a long time, decentralized applications based on IPFS + Ethereum have been waiting for long-term storage solutions, and Filecoin Liftoff Week bridged the IPFS, Ethereum, and Filecoin builder communities in novel and exciting ways. The latest collaboration between ConsenSys and Protocol Labs will further build bridges between these thriving communities.

Some exciting applications and use cases in the ecosystem already include:

• Consumer storage applications, including Slate and Fleek, that easily store files on IPFS and Filecoin

• DeFi use cases, including Filecoin storage and the Filecoin DeFi Bridge, which help miners, clients, and token holders interact more efficiently

• Decentralized video applications, including File.Video and Voodify, both built on Livepeer

• Archive storage use cases, including Filecoin Discover, Kiwix, and Internet Archive, aimed at preserving many culturally significant and valuable datasets

• And many more!

Additionally, there are many infrastructure and tools to help developers build on Filecoin:

• Textile's Powergate, an API-driven solution for deploying multi-layer storage between Filecoin and IPFS

• Truffle Preserve, which simplifies the process of saving long-lived application data on IPFS and Filecoin

• Infura's Filecoin Network API, which allows developers to build powerful applications based on Filecoin without having to sync their own nodes

• Glif Wallet, a lightweight web interface for sending and receiving Filecoin via Ledger devices

Finally, hundreds of new teams have entered the ecosystem from hackathons to accelerator events, including:

• HackFS, a month-long hackathon run by ETHGlobal, which produced over 130 new applications and teams based on Filecoin

• Apollo, a mentorship program run by Gitcoin, which guided 50 teams through the initial MVP on the Filecoin network

• Filecoin Launchpad, an accelerator run by Tachyon, which has 13 teams building serious companies and organizations on Filecoin

• Filecoin Frontier, an accelerator run by LongHash in Singapore and Shanghai, is recruiting a new group of teams that will build applications on the network!

We firmly believe that the future of Web3 lies in the hands of these talented developers, entrepreneurs, and builders, and we are excited about the efforts of the Filecoin community to support them in transitioning Web3 into a mainstream application network.

05

Storage

Last but not least is the diversity of Filecoin storage clients leveraging this decentralized storage network. Filecoin clients store some of humanity's most important datasets on Filecoin and preserve this data through a resilient global storage network—otherwise, much of this data would be lost or stored at prohibitive costs. In several initiatives and partnerships, hundreds of individuals and organizations are working to preserve important datasets on this new decentralized storage layer of the Filecoin network.

Our society collectively generates 250 million bytes of data every day. This includes 29.4 billion emails, 6.4 billion WhatsApp messages, 500 million tweets, telemetry data, sensor data, and more. While much of this data is saved, analyzed, and otherwise utilized, over 90% of the data generated by us as a society is routinely discarded. This is just the beginning, but it has already started, and we are beginning to see Filecoin serve as a storage layer for some of these extremely valuable datasets that might otherwise be lost. In recent months, through the Filecoin Discover project, we have accessed 3 PB of important publicly accessible datasets on the Filecoin network through hundreds of miners. These datasets include Wikipedia, Google Landmarks, 1000Genomes, Arxiv.org, OpenNeuro, and more.

Through programs like Filecoin Slingshot, community members have long-term stored 700 TB of knowledge-sharing licensed podcasts and music archives, package manager registries, biodiversity datasets, astronomical data, and more on the Filecoin network. Our storage client community helps create a healthy, efficient, and affordable storage market; strengthens our storage software and user experience; and identifies and refines the core value propositions of Filecoin to meet the needs of storage users.

Filecoin storage clients and developers have also built several tools to make data storage and management on Filecoin as simple as possible, such as:

• The Starling project, a storage product for large data storage clients particularly interested in data provenance and verifiable Filecoin storage capabilities.

• Filecoin.tools, a tool that helps storage clients and storage developers monitor their storage status on the Filecoin network.

• Scripts for processing CSV exports to back up the indexes of data stored on Filecoin for record-keeping and retrieval.

• Codefi storage for tracking miner reputation and monthly storage prices per GiB.

• And more!

Since the journey began, we have dreamed of solving important problems to ultimately lay the foundation for the advancement of human information. A truly dedicated, resourceful, and inspired community of storage clients is helping to turn this vision into reality. We are excited to continue working with you to build this new network together.

06

The Future of Filecoin

The mainnet launch is a significant milestone for the Filecoin project—but this is just the beginning. We stand at the threshold of a new era of distributed networks—where any application, smart contract, or NFT can store and access its data on a decentralized storage layer without any intermediaries. Filecoin unlocks unprecedented applications, such as decentralized machine learning databases that can charge for model training fees based on usage to offset their storage costs, or decentralized social media platforms that can enhance replication speed by adding voting videos, thereby improving playback speed. The next innovative applications in web3 will leverage the new foundations laid by Filecoin, IPFS, and libp2p to build applications and networks we can hardly imagine.✨

We are still in the early stages, but the Filecoin ecosystem already has tremendous momentum— as the growth of Filecoin-based projects and the network gains new capabilities, we will see compounded returns. Since the network launched, the gears of innovation have begun to improve the Filecoin protocol—five FIPs (Filecoin Improvement Proposals) have been released since the network went live. For example, the Filecoin Plus program (FIP-0003) is designed to inspire all miners to search for the world's most important datasets and load them onto the network, ensuring that Filecoin is a valuable, high-growth network for effective data storage.

These improvements are just the beginning of supporting the growth and innovation of the Filecoin network—there is more to come. The entire Filecoin community is now a shared owner of the network, and Filecoin creates value for the world—therefore, all miners, clients, developers, ecosystem partners, and token holders have a voice and responsibility in the dialogue. The future of Filecoin depends on you.

When we started, our goal was to build a Library of Alexandria to preserve humanity's most precious knowledge from ever being burned. Thanks to the global community of storage miners, accessible network tools, and passionate storage clients, we are getting closer to realizing this vision and also bringing countless new use cases. If you want to be part of this, please share your ideas with Filecoin Archives, a new project in collaboration with the Internet Archive aimed at organizing, preserving, and disseminating humanity's most valuable data on Filecoin. Together, we can ensure that humanity's most precious resources—our hard-earned knowledge across fields from science to history—are protected from disaster or attack and can benefit future generations.?

We thank everyone, past, present, and future, who has contributed their talents, insights, hard work, and resources to lay the foundation for Filecoin's mission to "become the decentralized, efficient, and robust cornerstone of human information." We are very grateful to continue building this network, ecosystem, and community with you.