I’m using CAR files as the way to get users who aren’t ready to run their own IPFS nodes to at least own the content addressing of their data; the user still chunks and hashes their files and derives the root CID “on their machine”, so that when they POST it to ye-olde web2 service, they don’t have to trust it as much… the CIDs they generated are the CIDs it will provide.
The js for the chunk’n’hash is the js-unixfs importer and exporter modules that are the same as used in js-ipfs
Move to CAR creation locally and send off over HTTP
The requirement for the blocks in a CAR to form coherent DAGs is not strict, so the CAR format may also be used to store arbitrary IPLD blocks.
That’s good news Unless we want to keep uploading all of the files every time (which may get pretty expensive and inefficient), theoretically this means that we can package up a diff (with dangling pointers) as a CAR, ship it to the server along with the expected final CID, unpack and merge the CAR in a go-ipfs node, and check that the resulting CID matches.
The downside is that this is a lot of manual manipulation on both ends. It probably works better with the pinset model than one where the user is maintaining their own data that we can’t see inside of, since analogous to an HTTP file upload model (where a server does the file/user association). The dangling pointers also means that we can run into broken links and non-mergable states that native IPFS handles for us.
Further study needed, but an interesting direction while js-ipfs is being less reliable than we’d like. My guts says that there is also a potential solution in providing autoscaling capacity, but we’ll chat with the operators group and find out more
Unless we want to keep uploading all of the files every time (which may get pretty expensive and inefficient), theoretically this means that we can package up a diff (with dangling pointers) as a CAR, ship it to the server along with the expected final CID, unpack and merge the CAR in a go-ipfs node, and check that the resulting CID matches.