Self-rendering Markdown docs - Library Research

As we have content that can be kept online with IPFS and our Webnative File System, we can start thinking about how to skip content management systems and just write and publish Markdown docs that are rendered automatically / dynamically (so, no “static” generation).

This might be a candidate for inclusion in Drive — especially alongside a basic text editor — but also can be a simple write up on how people can do this.

Let’s document existing libraries for this.

Would also be useful to see which Markdown features are supported. Footnotes would be a big one.

TeXMe

TeXMe is a lightweight JavaScript utility to create self-rendering Markdown + LaTeX documents.

Showdown

Showdown is a Javascript Markdown to HTML converter, based on the original works by John Gruber. Showdown can be used client side (in the browser) or server side (with NodeJs).

markdown-it

Markdown parser, done right. 100% CommonMark support, extensions, syntax plugins & high speed

MarkedJS

A markdown parser and compiler. Built for speed.

Note: doesn’t sanitize output HTML.

CommonMarkJS

CommonMark is a rationalized version of Markdown syntax, with a spec and BSD-licensed reference implementations in C and JavaScript.

RemarkJS

remark is a Markdown processor powered by plugins part of the unified collective.

Khan SimpleMarkdown

Most markdown-like parsers aim for speed or edge case handling. simple-markdown aims for extensibility and simplicity.

What does this mean? Many websites using markdown-like languages have custom extensions, such as @ mentions or issue number linking. Unfortunately, most markdown-like parsers don’t allow extension without forking, and can be difficult to modify even when forked. simple-markdown is designed to allow simple addition of custom extensions without needing to be forked.

At Khan Academy, we use simple-markdown to format over half of our math exercises, because we need markdown extensions for math text and interactive widgets.

Markdown WASM

Markdown parser and HTML generator implemented in WebAssembly, based on md4c

3 Likes

I was thinking about pandoc, I used it once it was very easy, but lacks some good functions and encoding issues

This specifically needs to run in the browser – so JavaScript or WebAssembly only. I don’t think there is a js version of Pandoc. I did some searching, and Pandoc is also a spec / set of rules for transforming content, so there may be some Pandoc-flavoured JS engines.

I wonder if one could compile Pandoc with GHCJS or via LLVM to Wasm :thinking:

that said, it would probably be HUGE :laughing:

We at https://twitter.com/TrailMarks has always looked at prevailing ways and sought to question them. since much of computing, as I see it and Alan Kay maintains too, is Pop Culture and Gargo Cult.
10 years ago we struggled with the cognitive dissonance when we seen the beginning of being able to do much better than the Semantic Web, and how come others haven’t discovered it before, and still going with its awfull limitations
10 years on we have RDF*, but it is just the wrong idea in the first place?
The idea of making tripples as constitutive of a semantic graph is just
so limiting. But that’s another post.

In 2003 took close look at XML and XSD even learned to use Xalan.
Quickly realized that it was just a hopeless technology. Nearly 20 years on, people voted with their feet and opted for JSON and anything but XML.

But I digress. MarkUp was a necessity at the early days of Web 2 as content-editable or anything based on the browser was horrendously brittle and awful. Writely that became Google Docs was a herculean accomplishment. The price was to generate utterly mixed up html. the meaning of nested list got lost in a mass of kludges.

Today, content-editable for a reasonably complete set of HMTL is pretty usable.
List editing mostly generates usable HTML.
For the past decade or so we explored the alternative path. Treat HTML itself as a HyperMedia Format for data exchange,

If we really want we can generate markup. The result is a drastic reduction of complexity. But better still you can have self-rendering presentation of content, which defines an underlying MindGraph, which can be resued, re-interpreted in any way you would like. so anyt presentation of any data is ipso facto interoperable and can be reused in any context.

The editor in this forum can take copy paste of HTML too and convert it on the file to MarkDown at the point of pasting.
Very useful, and sanityzing.
Ten years ago I seen an HTML to markdown converter that use regexp instead of
DOM selectors. How stupid one can get , really?

May 24, 2020

Created by refactoring instance first plumbing example. Generalizing to work for post and update with data being passed in

May 24, 2020

Content of Blog Post

Hello world Gyuri

  • one
  • two

again 9:58

1 Like

Thats actually a very cool feature.
I just posted a podcast on Anchor and saw something alike embbeded to it

I understand and agree with the desire not to have to run offline processes to build your websites, but I’d challenge you to consider the relationship between code and built artifacts in your system. Why should every client have to convert Markdown into HTML at runtime?

Yep! Some of this comes down to the nature of it, with bundlers and webpack etc etc. I’d like to carry around both the source code and the build artifacts (aka what gets published) – both of which serve different audiences.

For non-developers, being able to modify / extend the artifacts (app, website, etc.) should be built in, say by dropping in a theme.css file in their preferences.

Yeah, not thinking about this system wide at all. This is a small experiment to make extremely portable Markdown files. I did this already like 6 months ago, now just researching these libraries directly. This has all sorts of issues, like headers, footers, themes, etc. etc.

A Markdown-to-HTML Baking app could be logged into with Fission, pointed at a directory of Markdown files in the user file system, and build / generate actual HTML that is written to a (new or existing) app, or to the public/ tree of the user file system.

This could be re-used by any app that wants to use Markdown as part of a path towards publishing.

I imagine this as a great first step, but I’m thinking, ultimately, it’d be great to give users a kind of “automation” tool. Once the markdown files change, the app converting them to html automatically gets run. I think using build systems like make/just/shake as inspiration and building an accessible UI could go a long way.

1 Like

Yes definitely — events / callbacks in the filesystem will allow us to get there.

We’d have to get specific about the app and needs to come up with more requirements. Especially for publishing — mostly users want to press the button, see the output, tweak, and publish for real.

I know is not too much related, but Docable is a very interesting tool