Filecoin Research Roadmap

Filecoin Research Roadmap

Free Download // Login or Register
This document describes the Filecoin Research Roadmap for 2017, including background, results achieved so far and to be achieved, known future work, remaining open problems, and more. The works outlined are works in progress, and are published as technical reports in need of further work. They have received significant amount of peer-review, but more is needed before we are comfortable declaring them published as proper results. Any open problem listed is an invitation to join us and contribute to their solution.

The first version of the Filecoin protocol was published on July 15, 2014. Without diving into detail, it was a useful achievement at the time, but left some things as open problems. It wrapped a Proof-of-Retrievability for a growing dataset into the mining function, effectively forcing miners to offer a storage service as part of the mining process. It introduced how to store and retrieve data while keeping miners focused on storing as many different pieces of users’ data as they could. It proposed the distinction between the price of the currency and the price of the storage itself, allowing the currency to float above. It proposed ways for users to tune the replication and availability guarantees. And more, but it did not fully eliminate the use of Proof-of-Work – or reliance on another consensus protocol. This problem would be ideal to solve in such a network. It also needed scalability improvements.
In the subsequent three years, a lot changed.

First, our group spent the majority of that time building, deploying, and growing the InterPlanetary File System (IPFS), Filecoin’s sister protocol and dependency. The original intention was to build IPFS and turn back to Filecoin in late 2015. However, the quick success and significant adoption of IPFS by a large swath of groups – and the lack of applications that require decentralized storage networks like Filecoin – kept us working on IPFS. In building and deploying IPFS, in studying our users’ needs, and in understanding how these networks must operate, we were able to refine the requirements for Filecoin and pave a road for adoption into the hands of thousands of developers and organizations. These have real-world practical use cases, which give us a clear picture of desired features, target system performance, required system guarantees, and scalability trajectories. IPFS use-cases and specific user groups have also built a market need for Filecoin.

Second, Ethereum was implemented, shipped, and matured as a platform. Ethereum paved the way for the next generation of blockchain protocols, it proposed and tested dozens of new ideas, it put smart-contracts into large-scale use, it built a vibrant ecosystem where many contracts and apps transact with each other (and need to store or read files), and much more. Ethereum′s technology stack has developed and matured to a point where large scale businesses and operations are depending on it. The EVM is developing as a potential standard for smart-contracts. Finally, the blockchain market developed and blockchain applications now require a decentralized storage network.

Third, academia and industry have proposed many improvments and whole new blockchain and consensus systems. Notable and relevant improvements include: (a) Payment Channels, as in the Lightning Network, and their evolutions (State Channels, Sprites); (b) the first provable Proof-of-Stake protocols, and several variant protocol constructions; the use of Proofs-of-Space and Proofs-of-Storage for Proof-of-Work (Permacoin, Spacemint, and more); the use of zkSNARKs and a trusted setup in a blockchain (Zcash); the invention of zkSTARKs which give SCIP without requiring a trusted setup (transparency); and more.

Fourth, Bitcoin and Ethereum mining have created the worlds most powerful computer networks. The magnitude of Bitcoin’s hash rate is staggering: Bitcoin is pushing 6 exa-hashes per second! This is an enormous amount of computing power, and an enormous amount of proof-of-work energy waste. About 13.72 terawatts are spent in mining Bitcoins, equivalent to the consumption of the entire country of Slovenia. The sheer magnitude of miner revenues is also staggering: Bitcoin miners are earning about $4.4 Billion USD between 2013 and 2017 (priced per year, including EOY 2017 estimate); Ethereum miners are earning $2.7 Billion USD between 2015 and 2017. This is no longer seen as an absurd, eccentric, bound-to-fail scheme, but rather as an odd but highly profitable industry. An entire industry of special-purpose machines (the ASICs) sprang up, an ecosystem of businesses is meeting the demand for more hash power, and everything else miners need. All of this means that (a) tens of thousands of crypto miners seek to mine as much as they can; (b) large businesses with lots of computing resources are much more willing to entertain mining crypto assets; and (c) the proof-of-work waste continues its unabated exponential growth. The original Filecoin aim to completely replace useless proof-of-work with a useful storage service is much more needed than ever.

Fourth, the broader industry landscape has developed to the point where both massive and small organizations are trusting blockchain systems for their critical operations. More and more decentralized applications are being built. More and more businesses need a decentralized storage network. The exponential growth of data continues to demand massive expenditures in data storage world-wide, though the prices for storing and transmitting data remain quite high (storing and distributing a PB of data in the cloud can cost half a million dollars per year). Lots of regional storage players are being put out of business by competitors who can provide storage world-wide. Regional ISPs and interconnects now house CDN points-of-presence world-wide, and ISP ability to cache is decreasing due to the move to https. The storage market needs to be optimized much more granularly, and algorithmically. Businesses are much more likely to trust blockchain solutions. Vast amounts of latent storage exist around the world, unused and depreciating. The market is ripe for Filecoin, and it needs to scale fast.