Discord Link Harvesting

Discord chat gets used in many different community contexts to share info, surface interesting links, and share content.

The same issue applies to other chat platforms like Slack or Matrix.

For all of these systems, the performance of search of past content is highly variable, and the interface and patterns encourages to just ask the same question again, rather than necessarily searching.

When it comes to links, links get posted multiple times, without any ability to point out or connect to previous discussions or instances of the same link getting posted.

And, chat content is not discoverable in any way. It doesn’t appear in web search and isn’t really browse-able.

Social Bookmarking Tools

There are a variety of social bookmarking platforms that have existed over the years.

One example that is very programming / technical focused is Lobsters. Here’s a mobile screenshot:

Hacker News is another example, where the main focus is the lengthy comment threads around a like.

Pinboard is a social-but-personal tool, in the sense that you can look at your personal links but also aggregates.

Discord as source of links

The concept is to harvest links from Discord servers and turn this into a bridge / archive / cross posting interface to be able to power server level and potentially also personal social bookmarking interfaces and archives.

This can drive discovery and search, as well as be browsable and discoverable.

Publish structured data to IPFS

The first step in going from Discord API would be to publish structured data to IPFS as a long term archive and source that can drive multiple front ends.

This might be an existing format like JSON Feed, or a defined JSON schema and layout of files.

This IR can then be used to generate or power a variety of front ends.

Discord to Discourse

Fission heavily uses this Discourse forum for both internal and community purposes.

But, we share the majority of links through Discord chat, with only a few of them making it to a full write up here on the forum.

  • Pick a default category in Discourse to post links to (we might just default to #discussion)
  • Reactji could get mapped to Discourse reactji and/or tags
  • Pick a default Discord channel that always posts links to the forum
  • Pick a reactji in Discord that will post a link to a specific category in the forum

Stretch

Cross post comments in Discourse forum back to the links channel? Auto create a Discord thread per link?

Discord to Web App

Build a web app that looks like a group social bookmarking app. Drive it using static data that is exported from Discord.

Categories and/or tags based on Discord channels and/or reactji.

Voting can be reactji based, and updates over time.

Stretch

Search index

  • including storing a copy of the HTML source in atJSON

Take a screenshot of the link and store it.

If it’s a link to a PDF, add it to IPFS

  • convert / store as atJSON

This is still a work in progress, but I intend to post a bounty about this. Please leave a comment if you’d like to work on this.

I’ve been exploring the space of possible constellations that would allow overweb integration and continuation of conversations in an interpersonal evergreen setting overweb, on the margin providing deep integration of social annotation tools like Hypothesis and interest based open commons based autonomous social networks. Getting close to actully delivering something of true value. I was annotating a page talking about cross-posting run indy-annotation-search prototype and it surfaced this message.

I would be interested to learn if you would be interested in even possibly retroactive funding in collaboration with others like https://www.collaborative.tech/
work that could

  1. produce immediate value
  2. delivered via fission
  3. also integrating with Fission WNFS as one of the early adopted storage layers for the IndyWeb
  4. but actually starts the process of open commons-based peer production of constellations that could Flip the prevailing paradigm for the Web

https://dev-indy0.fission.app/hyp/facet/aiframe.html?params={"user"%3A"gyuri"%2C"group"%3A"all"%2C"url"%3A""%2C"wildcard_uri"%3A""%2C"tag"%3A""%2C"any"%3A"cross%2Bposting"%2C"max"%3A"50"%2C"service"%3A"https%3A%2F%2Fhypothes.is"%2C"exactTagSearch"%3A"false"%2C"expanded"%3A"false"%2C"addQuoteContext"%3A"false"%2C"format"%3A"html"%2C"_separate_replies"%3A"false"%2C"groupName"%3A"All"%2C"sortBy"%3A"recency"}

Now I understand why is it that startups ship when the money runs out.

That pressure increases the urgency of hustling potential users/collaborators

Fission should be part of that Alliance of course

1 Like

Memex has ended up building this Memex

We haven’t set it up yet in our Discord.