Imperial College Project Ideas

Wiki
1

We are connected to Imperial College through Outlier Ventures. These are some project ideas for students to look at and give feedback on.

Fission is building developer tools to make it easy to deploy apps directly from their laptop – with user controlled data and a distributed “backend as a service” – so that everything from hosting to database to user authentication is included. This is a “serverless” style of architecture.

Brooke’s recent presentation provides a high level overview of our technical roadmap.

The InterPlanetary File System (IPFS) is a key building block for us. ipfs.io is a starting point for learning more, plus the ProtoSchool Tutorials are very helpful.

You can also jump straight to our Guide on how to work with the Fission tools. After you download our command line tool, you can publish your apps directly from your laptop.

There are some existing App Ideas we’ve written up previously that students could also tackle, including extending and improving existing ones.

We’re interested in having any code, in any language, available as open source (Apache2 or MIT or similar license). This means it would add to the students’ Github portfolio and ideally live on over time.

We have client libraries available for a long list of languages – Clojure, C#, Elixir, Elm, Erlang, Go, Java, JS, Kotlin, PHP, Python, Rust, Scala, Swift, TypeScript. They are auto-generated, so improving them for a particular language would be valuable as part of any project.

Fission builds on top of IPFS, and we’ve opted for IPFS-focused projects that are broadly applicable to the whole community. We’re happy to add convenience features to the Fission platform to either integrate them, or make them easier to use with a Fission account.

We have also done extensive work with smart contract standards, such Fission Status Codes EIP 1066. We’re happy to support students who would want to work on projects related to this. Feel free to suggest your own project!

Expanded IPFS User Settings

We build a demo app for IPFS User Settings which allows users to store universal settings in IPFS and take them with them to other sites.

Students could expand on this application, adding additional and more in-depth settings.

Alternately, they could make a front end library that would make it easy for app developers to integrate portable user settings.

Another option is to build a similar application but for something other than user settings. One thing that we’ve discussed internally is portable dotfiles on IPFS.

IPFS Image Resizer and Metadata Generator

When images are uploaded to IPFS, they are given a hash that is unique to the original file, which serves as the content address.

Come up with some code that takes an original image on IPFS, resizes it to multiple sizes (and potentially croppings), and writes some metadata linking it to the original. This would likely be a directory which contains a JSON file that links the original IPFS content address to the newly created, resized versions. Research needed on the best approach :slight_smile:

This might mean just extending / integrating something like imgproxy to work with IPFS, but because of the content addressing rather than URI based nature of IPFS, rethinking how this works would be very valuable.

Storing metadata – eg. captions or tags – is another area of research. How do we keep various chunks of JSON around that point to the image? How do we, from an image, find that JSON metadata? Would encoding the metadata directly into the image be valuable?

Mobile IPFS Client App

Our friend Ligi has written IPFSDroid and IPFS API for Kotlin – both in Kotlin, which is similar to Java.

We would look to come up with a simple, single use app that allowed for using IPFS on device, either Android or iOS.

Some of the types of high level features that would be interesting:

  • file share / sync
  • photo gallery

There would need to be some research on a client-side IPFS server on the mobile device is possible. Having something cross-platform written in C, or cross-compiling some of Ligi’s Kotlin work. This is a large chunk of work and we would appreciate any POC or research findings alone.

Otherwise, extending on Ligi’s Android work would be the likely best path.

There is a Swift IPFS HTTP API but that is less interesting, as the mobile device can’t directly act as a server to serve the user’s files.

Cross Platform Mobile Library - C++ Implementations of libp2p, IPFS

For mobile use, a cross platform library for libp2p (the networking layer which is a dependency) and the IPFS protocol on top.

This is a VERY big task, so a lot of this will be compiling research and then deciding on a best course of action for a POC.

If you are interested in low level networking and distributed systems on reduced power devices, this would be perfect for you. While we’re interested in the output of this, we personally couldn’t give very much code support, but could plug you into various other groups like Protocol Labs.

1 Like
2

I’ll expand these, but some headers

Global Dynamic Image Resizing Cache

This one is the original idea, that we descoped in the short term. We would love to have the full version, and it does require some research on how to do this effectively P2P & with an unstable network topology. It is also a microcosm of a number of other related distributed caching problems.

One-Liner

A decentralized/P2P cache of resized images

Basic Idea

  • Client encounters an image that they’d like resized
  • Check the global cache for existing version(s) (likely probabilistic, e.g. Bloom or XOR filter)
  • If it doesn’t exist, resize localy and contribute!
  • Need to be able to efficiently filter bad submissions without a central authority

Discussion / Sketch

Fundamentally this is an optimization problem over a P2P CDN. Users can always resize the larger images themselves, but should contribute to the unified disk that is IPFS (or similar) their resizing so that others don’t have to transfer the larger image and resize locally.

As a sketch of a solution, we’re probably looking at some event stream and an overlay of relayers that already have the content cached (e.g. from looking at the image previously). The network topology cannot be assumed to be stable, and runs in an environment (browsers) that doesn’t have native access to a logically centralized blockchain with tons of storage.

There is an outsized benefit for commonly accessed (“popular”) images. By looking at the number of submissions that match (or in the case of nondeterministic formats, their Hamming distance or other heuristic), we can set some minimum threshold before making a version as “safe” to the network.

Is this pushed over pubsub as an event stream, and watched and gossiped by everyone? Do users store it in their Fission File System at a well-known path? How is cache lookup done efficiently?

Does this need Sybil-resistance? Possibly! Only read from users you trust? Also good! All open questions at this point.

Streaming- / Seamless-Compute

One-Liner

Content addressed, dynamic FaaS with web scale hotspot optimization

Background

  • With content addressing, we’ve abstracted over physical location
  • Computers have very different performance characteristics
  • Traditional FaaS (e.g. Lambda) requires pre-deployment and always running compute there
  • Wasm is a (mostly) deterministic VM
  • Any data can be content addressed
  • Code and Wasm modules can be content addressed
  • Curried function application can be expressed as a Merkle tree
  • Functions that would run longer than network latency can be sped up on a heavy server

Basic Idea

  • As a base case, all compute must be able to run locally
  • Dynamically (at runtime) choose if a function should run locally or remotely
  • Everything happens behind the scenes fully hidden
  • Data content address, signed, optionally symmetrically encrypted, and synced
  • Service providers can hotspot optimize over all function/argument pairs
    • Essentially “compute arbitrage”
  • Bonus points if the results can be automatically available to everyone

Discussion / Sketch

There’s several ways of approaching this. As a sketch, it would probably make sense for the programmer to have an automnated tool that analyses the cost dynamics some Wasm module or closure, and marks up the code (or a descriptor file). At runtime, it can check the (e.g.) size or type of input and the capability of the device it’s on / user settings, and decide if it should run locally or remotely.

Our plan for this is to pair it with an event stream on our nondestructive file system, so we can always roll back over bad results. A trusted setup would be the first step ti avoid this complexity, since it’s more at the data integration than the actual compute.

The hard parts here are:

  • Automatically building a good enough cost model for arbitrary functions
  • Analyzing nontotal functions, and esp. functions with additional network calls
  • Specifying a DSL for communicating any side effects that need to run on the client (e.g. console logs)
1 Like