TiddlyWiki File Upload Plugin - Webnative IPFS Funded

Apps TiddlyWiki
, ,
1

I have funded the initial development of a webnative IPFS file upload plugin for TiddlyWiki. This is personal funding, not from Fission, but obviously connects into the TiddlyWiki on Fission project.

Saq Imtiaz has agreed to be the lead developer for the project, and we’ve gotten the ok from Jeremy to use the core TW5 github repo to develop in.

The goal is to have a general core framework to support images, PDFs, and other file assets, as well as look at how this connects into different savers and backends, plus looking at static publishing in the same context.

For my personal use case, the initial features include:

  • support Webnative IPFS uploads
  • create a new tiddler for each file
  • link the file using _canonical_uri
  • resize images to create smaller thumbnails

This is fully funded and is going to get built. If you are interested in seeing this support for other saver backends, or would like to participate in prioritizing use cases and feature suggestions, you can contribute on OpenCollective.

Would you like to sponsor as a company? Contribute $500 or higher and get prominent placement of your logo and links.

If you are a TiddlyWiki developer, you are of course also welcome to contribute code directly to this open source project.

Links

Follow this post and other #tw-file-upload topics here in the forum for updates.

1 Like
2

Oooh, very cool. Also, folks on the Google Groups have been talking about how to fund features/projects for TW, so I linked this as the perfect example of the Open Collective model.

2 Likes
3

I am very pleased to have been given the opportunity and support to dedicate time to working on this, as it dovetails nicely with my own interests. Furthermore, the overall ability to save binary files to other backends is an oft requested feature in the TiddlyWiki community. Please note that I will be actively starting work on this from June 7th, however discussions can commence already.

I have previously done some work on a similar feature with the the Tiddlywiki node.js server, where images that are imported are saved as external files. This was always meant to be part of an exploration towards a more generic framework in TiddlyWiki for importing binary files and saving them to different external storage backends.

A desirable outcome for this project from my perspective would be:

  • a pluggable generic framework in TiddlyWiki for uploading binary files to different external storage providers.

  • support for Webnative IPFS as the first supported storage provider.

  • support for creating thumbnails from images. This would ideally take the form of a generic feature for TiddlyWiki that allows resizing images (and for JPG changing the quality/compression).

Support for other storage providers could then be added separately. How generic we can make the file upload mechanism in the first iteration will depend on the TiddlyWiki on Fission implementation details and dependencies, which I have not yet had the opportunity to study.

I had the realization this morning that using the core TiddlyWiki Github repo for the development of this plugin could be problematic. As a personal repository, there is no fine grained ability to grant write permissions to others. I’ve had a quick discussion with Jeremy this morning and he concurs. So the development will not take place in the core TiddlyWiki Github repository, but the goal will be to ultimately create a PR to merge this plugin into that repo.

Furthermore, having had a very quick peek at the TiddlyWiki on Fission (TWoF) code, it may make more sense for the development of this plugin to take a place in a fork or branch of that repository. Jeremy, perhaps you have some insight to offer here in terms of workflow for TWoF development?

Another important consideration is that TiddlyWIki already has affordances for saving to different backends via the saver modules which are used for saving the wiki file. However, these are currently implemented in a manner that is not re-usable for saving other assets, and in some instances can only be used for saving a single file at a time.

The work on static file publishing introduces publisher modules which amongst other things allow saving different files to their given storage backends. As discussed with Jeremy, not only is there overlap here between savers and publishers, but the same code should be re-usable for saving other files, for instance for file uploads. So instead of introducing yet another mechanism for supporting different back ends for uploading files, I am hoping to work with Jeremy to explore if we can consolidate the base functionality needed for each backend for savers, publishers and file uploads.

2 Likes
4

Just adding on to my previous post and the overlap between savers and publishers, there are also syncers to consider and how they fit in with uploading files. The most prominent example would be the core node.js server which uses a syncer to save changes per tiddler to the local file system.

Edit: another distinction to keep in mind is saving to the same backend that hosts the TiddlyWiki versus saving to another backend where cross-domain restrictions would come into play.

1 Like
5

There are some notes on a proposed architecture for the file uploads that I have discussed with @Jermolene and we are on the same page overall. Finer details of the implementation will become clearer as we work on this.

Several areas for extending the core with new features have been identified, which not only will be needed for the plugin but should have universal utility in TiddlyWiki and will open the door for other interesting developments as well. Initial work on this plugin will take the form of working on these new core features. The proposed implementation will be a fair bit more work than what I first anticipated when it seemed like this would be an extension of the import process (which has some issues that make it unviable). However, this is the route we need to take if we want this in the core, and the end product will be far more robust and flexible and will accommodate an offline use case as well.

I am nursing an impinged nerve in my shoulder this week which I am trying to rest. Once that is a bit better, I will convert those notes into an Github ticket tracking work on the file uploads plugin, as well as tickets for each of the individual core improvements identified.

One of the areas that requires further discussion is the image resizing and how we want to implement that. Options range from a simple <canvas> based resize option with no external dependencies, to image optimization via an external dependency like https://squoosh.app/. As mentioned above, the work involved in developing this plugin is already exceeding what I had first anticipated. So if we do want to implement a more extensive image processing feature like integrating squoosh, additional resources will be required.

1 Like
6

Hi @saqimtiaz — thanks for the update.

I think the simplest option should be aimed for.

And, for that matter, I see file uploads being step one, resizing being step two.

Are you going to use the core repo for tickets or somewhere else? I can’t remember where the discussion ended.

And — take care of your typing shoulder! Thinking and planning like this to get things core worthy are great, no rush on getting this done.

7

Agreed, this is my take on it as well. It is logically working out that way with the code architecture. The flow will be designed to allow an optional processing step and we can insert the image resizing later on. It would even be possible to switch between different image handling implementations.

Since the bulk of the upfront work will be on the TiddlyWiki core, I’ll create tickets in the TW GH repo and work in my own fork of it just like with any other PR for the core. Once we get to the more Fission specific parts of the work, we can assess where it makes best sense to work on the code for that.

Voice recognition works pretty well for discussions but doesn’t quite cut it for technical writing or coding, but I am already feeling better and a few more days should do the trick.

1 Like
8

I haven’t had a chance to delve into this in detail yet but the Pica library looks promising for entirely client side image resizing:

Resize images in browser without pixelation and reasonably fast. Autoselect the best of available technologies: webworkers, webassembly, createImageBitmap, pure JS.

Leaving this here as much for myself for later as for others that might find it of interest.

9

libsquoosh as well squoosh/libsquoosh at dev · GoogleChromeLabs/squoosh · GitHub

10

@Jermolene @boris @walkah I’ve had a pretty rough last day of trying to work with webnative and TiddlyWiki on Fission and I would appreciate some input and guidance on a few subjects.

a) Did the uploading of binary files (images) in the static file publishing wiki ever work? As in, did the image files saved on Fission successfully load as images? This question is mostly just for my sanity at this point.

I realized yesterday that while I had previously confirmed that image files were showing up on Fission, I had not tried to load and verify them. Turns out they aren’t valid images, see for example by opening it in the browser:

https://saqimtiaz.files.fission.name/p/xzh-diangu-2009.jpg

I had assumed that one could send base64 encoded string representations of images to webnative.add() since that is what the publishing work does and spent a lot of time trying to debug it, but I think my original premise was wrong. Saving base64 encoded strings via webnative.add() creates a corrupted file. Saving ASCII (text files) works just fine.

What made this even harder to debug and resolve yesterday was also a lack of confidence that things might have changed on the Fission end with regards to base64 encoding without it being documented anywhere (more on this theme further down).

I wasn’t able to test the publishing code myself to confirm that it doesn’t work for images either, as that is currently not in working condition. However, I did go back and check Jeremy’s demo and realized that the image seemed to display the same issue: https://vimeo.com/529416013 at 35:23 minutes.

b) The latest webnative (0.26) distribution no longer comes with a UMD compatible file, nor a non-minimized file at all. See: https://unpkg.com/browse/webnative@0.26.0/dist/

The index.min.js file is not a drop in replacement. Any thoughts on how to handle this? I imagine we need a wrapper around it but that isn’t easily done working with a minimized file and I am not particularly keen on trying to build webnative myself. There is very little in the way of documentation.

c) A big obstacle and timesink yesterday to resolving the issue with saving binary files via webnative.add() was that the documentation is quite vague and furthermore not versioned. So it is impossible to know what data types are supported in webnative 0.24 or even 0.25.2 (the latest with a UMD file) for add(). I even went through the Changelog in github but even that seems very scarce on useful information.

d) I am overall curious as to what the strategy is for webnative development, both with regards to backwards compatibility but also in terms of keeping developers informed.

  • Is webnative 0.24 or 0.25.2 on the client side still meant to be working?
  • As an external developer working with webnative how can I keep myself informed on changes in the API and any breaking changes?
  • Looking at the Changelog isn’t informative at all so I assume I am looking at the wrong channel?
    • For example, looking at the Changelog for 0.24 even though I am aware of the nature of the change regarding paths, I still had to read the relevant message a few times to try to be sure it referred to the same thing. Similarly, even though 0.25.2 drops UMD compatibility this doesn’t come across anywhere in the changelog.
1 Like
11

Thanks to a pointer from Matheus23 on discord to the typescript type file and then digging into the git history to find the previous version, what had me running into walls for a couple of hours last night only took 5 minutes to resolve now (though I need to do more testing before I say so with more confidence!).

The primary motivation for the above post was not the actual problem blocking me but the circumstances surrounding it, so I believe those issues are still worth discussing, whether here or via a different channel.

1 Like
12

I’d love some more input on this issue. What’s the exact problem you’re trying to solve. Why do you want an un-minified version of the file? Do you need it for stacktraces? An unminified, but bundled index file would still be fairly hard to read. There’s a sourcemap file right next to the index.min.js file (index.min.js.map) and the .js file has a comment pointing to that sourcemap so browsers should be able to point you to the typescript source code directly, although I can see lots of things going wrong there and it might not work right now (I’ll check).

If this issue needs to be fixed urgently to unblock you, I recommend running esbuild, but without the --minify flag.

I agree, we still have to improve our documentation. I’m hopeful for source-code-derived documentation (we have some at https://webnative.fission.app/, but unfortunately it’s not that easy to generate good docs from typescript right now: E.g. I was unable to link you to the FileContent type).

Yes, absolutely!

It should be the changelog.

We need to improve our process with the changelog I think. I’ll discuss some possible improvements in our process with the team :slight_smile:

Thanks for raising these issues. We’re still figuring things out, breaking stuff unintentionally, etc. Sorry that this has hit you in particular. And please come back with some info about the index.min.js stuff :grinning_face_with_smiling_eyes:

1 Like
13

This is no longer a blocker for me thanks to your assistance earlier today. It came up yesterday since I wasn’t confident about supported types for fs.add(), nor the documentation since I was on an older version of webnative. So upgrading TiddlyWiki on Fission to the current version was a means towards resolving that problem.

The real issue is that TiddlyWiki has its own module loader that is used instead of directly including scripts in the HTML file. This module loader requires CommonJS compatible modules. So replacing the UMD file with the index.min.js file does not work. So we will need to find a solution for upgrading TiddlyWiki on Fission to newer versions of webnative. ​@Jermolene is probably the best placed to comment on this in terms of a long term solution and may have tricks up his sleeve for these situations, and it would be good to wait for his input.

One of the common workarounds for non-compatible modules is to wrap them in some code that makes them CommonJS compatible and that is why I thought to look for a non-minified version of the file to temporarily unblock my work yesterday. However, I always consider this a last resort as it isn’t particularly sustainable unless automated. So we need have a strategy for how to smoothly handle upgrades of webnative for TiddlyWiki on Fission.

Regarding docs and changelog I very well understand that this is cumbersome work and the challenges but also appeal for automating as much of it as possible. A quick stop gap measure that I think would improve things significantly would be a manual review of the Changelog before each release to make sure that all externally facing changes are adequately described from the perspective of someone that has not been working on the code, but only interacting with it via the API.

No worries. It is to be expected when working with things that are on the cutting edge and relatively new. The motivation for wanting to discuss this despite my own work being unblocked is to hopefully impart value by providing a different external perspective and feedback. From my perspective when engaging with projects in this stage from the outside, it is a two way street. I get to play with your new shiny things, but then I try to provide feedback where I think it might be helpful.

1 Like
14

At one point, I thought it did work, but in fact I think I was just confused by images that were being rendered as base64 data:// URIs.

That’s right: we need to have either a CommonJS-compatible version of the module, or be able to devise a prefix/suffix to bookend the module to make it compatible.

15

We can totally turn our index.min.js file into a UMD file. We’re using esbuild to create that bundle and it supports headers and footers in a way that keeps our sourcemap intact. I’ll look into that :+1:

2 Likes
16

@matheus23 Thank you, that would be terrific and no doubt a lot more robust than any work around we might concoct.

17

For anyone following along here, there is now a dedicated section on talk.tiddlywiki.org for the FileUploads plugin.

There is a public beta now available:

There was a demo on the recent TW community call:

1 Like
18

Thanks Saq. There are a couple of other posts here that I should copy / paste over into the new Talk TW.