Does WNFS deduplicate content?

Developers WNFS
1

Yes, WNFS automatically de-duplicates data by being built on IPFS. The same file has the same content address, whose blocks are stored once.

For public data, the same file stored twice only has one copy of its binary data, plus two small file system wrappers that differ.

Private, encrypted data is not de-duplicated. The file contents are encrypted and scrambled.

2 Likes
2

deduplication on insertion if you’re building a private store up from a filesystem is definitely possible and probably easy, deduplication on repeated insertion is probably harder… reading and thinking on this right now.

1 Like