Discord Link Harvesting

Discord chat gets used in many different community contexts to share info, surface interesting links, and share content.

The same issue applies to other chat platforms like Slack or Matrix.

For all of these systems, the performance of search of past content is highly variable, and the interface and patterns encourages to just ask the same question again, rather than necessarily searching.

When it comes to links, links get posted multiple times, without any ability to point out or connect to previous discussions or instances of the same link getting posted.

And, chat content is not discoverable in any way. It doesn’t appear in web search and isn’t really browse-able.

Social Bookmarking Tools

There are a variety of social bookmarking platforms that have existed over the years.

One example that is very programming / technical focused is Lobsters. Here’s a mobile screenshot:

Hacker News is another example, where the main focus is the lengthy comment threads around a like.

Pinboard is a social-but-personal tool, in the sense that you can look at your personal links but also aggregates.

Discord as source of links

The concept is to harvest links from Discord servers and turn this into a bridge / archive / cross posting interface to be able to power server level and potentially also personal social bookmarking interfaces and archives.

This can drive discovery and search, as well as be browsable and discoverable.

Publish structured data to IPFS

The first step in going from Discord API would be to publish structured data to IPFS as a long term archive and source that can drive multiple front ends.

This might be an existing format like JSON Feed, or a defined JSON schema and layout of files.

This IR can then be used to generate or power a variety of front ends.

Discord to Discourse

Fission heavily uses this Discourse forum for both internal and community purposes.

But, we share the majority of links through Discord chat, with only a few of them making it to a full write up here on the forum.

  • Pick a default category in Discourse to post links to (we might just default to #discussion)
  • Reactji could get mapped to Discourse reactji and/or tags
  • Pick a default Discord channel that always posts links to the forum
  • Pick a reactji in Discord that will post a link to a specific category in the forum

Stretch

Cross post comments in Discourse forum back to the links channel? Auto create a Discord thread per link?

Discord to Web App

Build a web app that looks like a group social bookmarking app. Drive it using static data that is exported from Discord.

Categories and/or tags based on Discord channels and/or reactji.

Voting can be reactji based, and updates over time.

Stretch

Search index

  • including storing a copy of the HTML source in atJSON

Take a screenshot of the link and store it.

If it’s a link to a PDF, add it to IPFS

  • convert / store as atJSON

This is still a work in progress, but I intend to post a bounty about this. Please leave a comment if you’d like to work on this.