IPFS-based Approaches to Object Storage for Various Website Architectures & Use Cases

Continuing a conversation from Discourse about using WNFS / WN SDK / IPFS to implement object storage functionality that is currently often implemented with AWS S3. The overall question is: what are the best approaches to implementing common use cases using IPFS / WNS and other decentralized storage technologies?

These use cases might break down into a fairly straightforward quadrant analysis: one axis being read permissions (“access”, public <----> restricted) and the other being write permissions (“ownership”, centralized <----> individual).

Common use cases that come to mind initially:

  • Static site hosting (public access, centralized ownership): uploading the output of static site generators, etc.
  • CMS-managed asset storage (public access, centralized ownership): uploading publicly available assets into object storage via a web interface.
  • CMS-managed private file file storage (restricted access, centralized ownership): uploading of files via website interface, but access to uploaded files is restricted based on authorization).
  • CMS-managed decentralized storage proxy: this is more of an emerging pattern, given that it’s really enabled by decentralized storage technologies - users upload files that can be displayed within a site, but they retain write, edit, and delete privileges on those.

The first is probably fairly trivial with just IPFS and a web gateway. Prior art and existing implementations include gatsby-plugin-ipfs and https://developers.cloudflare.com/distributed-web/ipfs-gateway/. Enhancement request would be to be able to use a web UI to administer content, such as Netlify. There’s a ticket with discussion of that approach here: https://github.com/netlify/netlify-cms/issues/502. Oh, and also prior discussion in the forum here - Auth0/Netlify/Gatsby Cloud Extensions to allow developers to manage user files. Coveredin IPFS’s docs here Static-site generators | IPFS Docs (a few options for using WP as a static site generator too, eg Wordpress to IPFS : Unstoppable Domains)

The second, CMS-managed public storage, is also pretty straightforward to implement with a basic CMS, IPFS uploader extension, and IPFS gateway. There are Drupal modules such as https://github.com/amateescu/ipfs and Fission’s own https://github.com/fission-suite/drupal-ipfs.

Worth noting at this point that any IPFS-only approach won’t implement content expiration, deletion, etc. in parity with S3…by design. As I understand old content might be garbage collected at some point, but it’s a clear design decision of IPFS to ensure permanence and immutability. For most public access cases this should not be much of a limiting factor, but in cases where the ability to expunge published data is required, it might be.

As for the 3rd case, it is less clear, and I imagine would require WNFS along with some sort of authorization-based gateway. What existing implementations exist that could cover or be used to implement this one?

The 4th case would require CMS integration with a DID provider, as well as an encrypted file store and authorization-based gateway.

Does this way of categorizing the use cases and approaching them make sense? Are there other requirements that would more strongly favor WNFS over stock IPFS for the public access use cases?

I know that promoting DIDs and moving toward a more thoroughly decentralized architecture is a major part of Fission’s vision and goals. Are there strong arguments to avoid the more “hybrid” approach at this stage? or is there benefit to working through progressively more decentralized architectures (could provide an on-ramp to more developers)?

1 Like

Amazing, thank you for this write up!

I need to digest this a bit and turn it into a bit of a map. Directly “replacing S3” is (maybe) better done by running an S3 compatible open source clone like MinIO https://min.io/ – but these approaches can be mixed and matched, including using MinIO as a backend store for IPFS.

On the other side, long term inexpensive persistence with Filecoin, and then working on that stack to have great Filecoin <> IPFS retrieval promises to be much much cheaper.

For static site hosting, we’d love to have more people run the Fission produced webnative services – making it possible for an organization like yours to offer equivalent to Vercel / Netlify like experience for “static” and web app front end.

OK, more to write here. I’ll get the team to go through this and we’ll look at digging into this more.