TiddlyWiki File Upload Plugin - Webnative IPFS Funded

I have funded the initial development of a webnative IPFS file upload plugin for TiddlyWiki. This is personal funding, not from Fission, but obviously connects into the TiddlyWiki on Fission project.

Saq Imtiaz has agreed to be the lead developer for the project, and we’ve gotten the ok from Jeremy to use the core TW5 github repo to develop in.

The goal is to have a general core framework to support images, PDFs, and other file assets, as well as look at how this connects into different savers and backends, plus looking at static publishing in the same context.

For my personal use case, the initial features include:

  • support Webnative IPFS uploads
  • create a new tiddler for each file
  • link the file using _canonical_uri
  • resize images to create smaller thumbnails

This is fully funded and is going to get built. If you are interested in seeing this support for other saver backends, or would like to participate in prioritizing use cases and feature suggestions, you can contribute on OpenCollective.

Would you like to sponsor as a company? Contribute $500 or higher and get prominent placement of your logo and links.

If you are a TiddlyWiki developer, you are of course also welcome to contribute code directly to this open source project.

Links

Follow this post and other #tw-file-upload topics here in the forum for updates.

1 Like

Oooh, very cool. Also, folks on the Google Groups have been talking about how to fund features/projects for TW, so I linked this as the perfect example of the Open Collective model.

2 Likes

I am very pleased to have been given the opportunity and support to dedicate time to working on this, as it dovetails nicely with my own interests. Furthermore, the overall ability to save binary files to other backends is an oft requested feature in the TiddlyWiki community. Please note that I will be actively starting work on this from June 7th, however discussions can commence already.

I have previously done some work on a similar feature with the the Tiddlywiki node.js server, where images that are imported are saved as external files. This was always meant to be part of an exploration towards a more generic framework in TiddlyWiki for importing binary files and saving them to different external storage backends.

A desirable outcome for this project from my perspective would be:

  • a pluggable generic framework in TiddlyWiki for uploading binary files to different external storage providers.

  • support for Webnative IPFS as the first supported storage provider.

  • support for creating thumbnails from images. This would ideally take the form of a generic feature for TiddlyWiki that allows resizing images (and for JPG changing the quality/compression).

Support for other storage providers could then be added separately. How generic we can make the file upload mechanism in the first iteration will depend on the TiddlyWiki on Fission implementation details and dependencies, which I have not yet had the opportunity to study.

I had the realization this morning that using the core TiddlyWiki Github repo for the development of this plugin could be problematic. As a personal repository, there is no fine grained ability to grant write permissions to others. I’ve had a quick discussion with Jeremy this morning and he concurs. So the development will not take place in the core TiddlyWiki Github repository, but the goal will be to ultimately create a PR to merge this plugin into that repo.

Furthermore, having had a very quick peek at the TiddlyWiki on Fission (TWoF) code, it may make more sense for the development of this plugin to take a place in a fork or branch of that repository. Jeremy, perhaps you have some insight to offer here in terms of workflow for TWoF development?

Another important consideration is that TiddlyWIki already has affordances for saving to different backends via the saver modules which are used for saving the wiki file. However, these are currently implemented in a manner that is not re-usable for saving other assets, and in some instances can only be used for saving a single file at a time.

The work on static file publishing introduces publisher modules which amongst other things allow saving different files to their given storage backends. As discussed with Jeremy, not only is there overlap here between savers and publishers, but the same code should be re-usable for saving other files, for instance for file uploads. So instead of introducing yet another mechanism for supporting different back ends for uploading files, I am hoping to work with Jeremy to explore if we can consolidate the base functionality needed for each backend for savers, publishers and file uploads.

2 Likes

Just adding on to my previous post and the overlap between savers and publishers, there are also syncers to consider and how they fit in with uploading files. The most prominent example would be the core node.js server which uses a syncer to save changes per tiddler to the local file system.

Edit: another distinction to keep in mind is saving to the same backend that hosts the TiddlyWiki versus saving to another backend where cross-domain restrictions would come into play.

1 Like

There are some notes on a proposed architecture for the file uploads that I have discussed with @Jermolene and we are on the same page overall. Finer details of the implementation will become clearer as we work on this.

Several areas for extending the core with new features have been identified, which not only will be needed for the plugin but should have universal utility in TiddlyWiki and will open the door for other interesting developments as well. Initial work on this plugin will take the form of working on these new core features. The proposed implementation will be a fair bit more work than what I first anticipated when it seemed like this would be an extension of the import process (which has some issues that make it unviable). However, this is the route we need to take if we want this in the core, and the end product will be far more robust and flexible and will accommodate an offline use case as well.

I am nursing an impinged nerve in my shoulder this week which I am trying to rest. Once that is a bit better, I will convert those notes into an Github ticket tracking work on the file uploads plugin, as well as tickets for each of the individual core improvements identified.

One of the areas that requires further discussion is the image resizing and how we want to implement that. Options range from a simple <canvas> based resize option with no external dependencies, to image optimization via an external dependency like https://squoosh.app/. As mentioned above, the work involved in developing this plugin is already exceeding what I had first anticipated. So if we do want to implement a more extensive image processing feature like integrating squoosh, additional resources will be required.

1 Like

Hi @saqimtiaz — thanks for the update.

I think the simplest option should be aimed for.

And, for that matter, I see file uploads being step one, resizing being step two.

Are you going to use the core repo for tickets or somewhere else? I can’t remember where the discussion ended.

And — take care of your typing shoulder! Thinking and planning like this to get things core worthy are great, no rush on getting this done.

Agreed, this is my take on it as well. It is logically working out that way with the code architecture. The flow will be designed to allow an optional processing step and we can insert the image resizing later on. It would even be possible to switch between different image handling implementations.

Since the bulk of the upfront work will be on the TiddlyWiki core, I’ll create tickets in the TW GH repo and work in my own fork of it just like with any other PR for the core. Once we get to the more Fission specific parts of the work, we can assess where it makes best sense to work on the code for that.

Voice recognition works pretty well for discussions but doesn’t quite cut it for technical writing or coding, but I am already feeling better and a few more days should do the trick.

1 Like