Hey everyone, welcome to our new WNDB forum!
We had our first public research and design session last week. For those who may have missed it, we have posted the full video:
A huge thank you to everyone who participated!
I’ve tried to compile some high level notes (feel free to add anything I may have missed in the comments!):
- As we had predicted, the group was a mix of people who are interested in using WNDB as app developers (who have feature and API design interests) and those interested in the protocol / “plumbing” of WNDB
- There were not a lot of strong feelings about the nature of the API - rather, WNDB needs features and documentation on how to implement them
- There is a lot of interest / consensus around our initial event sourcce-based design ideas. @expede highlighted datomic and datascript.
- A lot of good (early) discussion around schema management: developers should be able to define them (but also share and re-use), they should be applied as “views”, and lots of interest in project cambria
Moving forward, we will most likely split sessions between the API level and the protocol level. Keep an eye on #events for announcements. This forum will include any discussion and announcements as we go.
Here’s the chat log from the session as well:
Chat Log
Here’s the full chat log from the event as well. There were several good links shared:
00:08:40 Samuel Andert: GraphQL
00:10:44 Marc-Antoine Parent: I think we can have both...
00:17:51 Gyuri Lajos: can we have the whole screen to show people?
00:19:16 Gyuri Lajos: I meant to write please
00:20:01 Brooklyn Zelenka: Guyri: not sure I follow. Do you want one of the slides?
00:20:54 Boris Mann: @brookly — Gyuri wanted the screen share gone so we can see the entire gallery, which James already did
00:23:48 Marc-Antoine Parent: underlay
00:23:51 Helder S Ribeiro: Hadn’t heard of it
00:23:55 Helder S Ribeiro: Thanks for the ref!
00:24:36 ben: sorry, I had to jump out for a call, back
00:25:24 Helder S Ribeiro: https://www.underlay.org/
00:26:02 Marc-Antoine Parent: my own work (but the site is out of date): https://hyperknowledge.org
00:27:42 Helder S Ribeiro: 👀
00:29:19 Helder S Ribeiro: https://www.inkandswitch.com/cambria.html
00:29:37 Gyuri Lajos: Grakn: The Knowledge Graph
00:29:38 Gyuri Lajos: https://grakn.ai/
00:29:51 Gyuri Lajos: for inspiration
00:30:29 Marc-Antoine Parent: Schema as a view
00:31:35 Marc-Antoine Parent: Underlay gets into the process of translating free-form RDF-ish into algebraic types, which could correspond to schemas
00:32:28 Marc-Antoine Parent: The dynamic of collective data means having to handle social data conflicts
00:34:38 Marc-Antoine Parent: Note: even with schema as view, we can let people use the schema for write operations, as long as the write is treated as an event, and decomposed into schema-independent microclaims
00:35:05 Brooklyn Zelenka: Yeah Marc-Antoine, this very much follows my current thinking
00:35:46 Marc-Antoine Parent: Brooklyn: Glad to hear it!
00:35:53 Brooklyn Zelenka: Thought currently more thinking of EAVT than events (though we had started at event sourcing)
00:36:12 Helder S Ribeiro: Related https://typedefs.com/about/
00:36:13 ben: it is a great idea, instant twitter! :)
00:37:10 Helder S Ribeiro: What’s EAVT?
00:37:24 Brooklyn Zelenka: https://tonsky.me/blog/unofficial-guide-to-datomic-internals/
00:37:28 Marc-Antoine Parent: I'm seeing EAV(T?) micro-claims as bundled in events...
00:37:46 Brooklyn Zelenka: Yep, me too
00:37:55 Helder S Ribeiro: Nice, thanks!
00:38:03 Brooklyn Zelenka: Assertion and refutations only, extended with temporal logic (make claims about the future)
00:38:17 Brooklyn Zelenka: But that’s fundamentally “schemaless”
00:38:24 Brooklyn Zelenka: Anyway, I’ll get into this all later I’m sure ;)
00:38:49 Marc-Antoine Parent: Sounds great. Are the AEVT micro-claims accessible for you? I think that's extremely important.
00:39:11 Marc-Antoine Parent: (sorry I meant addressable)
00:39:18 Brooklyn Zelenka: Yep all addressable
00:42:56 Marc-Antoine Parent: That is it, right? https://github.com/feathersjs/feathers
00:43:12 Boris Mann: Yep!
00:43:20 Brooklyn Zelenka: Yeah; here’s the homepage https://feathersjs.com/
00:44:51 Gyuri Lajos: IPLD does some amazing magic in the space of schemas must check it out
00:49:56 Samuel Andert: Just listening and learning today
00:50:45 Marc-Antoine Parent: micro claims are like addressable triples...
00:51:26 Marc-Antoine Parent: Custom schemas should also include custom indices, agreed
00:52:48 Gyuri Lajos: https://twitter.com/TrailHub1/status/1347137546904547328
00:52:50 Marc-Antoine Parent: schema views are a good layer on which to anchor subscriptions
00:52:54 Helder S Ribeiro: One thing I don’t know much about but haven’t heard mentioned and would just want to throw the wrench in there: atomic transactions?
00:53:28 Brooklyn Zelenka: Yes, we’re doing STM even in the filesystem. Atomicity required for sure
00:53:47 Steven Vandevelde: Sorry, currently eating ;)
00:54:02 Helder S Ribeiro: STM 👍
00:55:54 Marc-Antoine Parent: When you have branch fork-merge semantics, atomicity may not be quite as important (unless you cherry-pick).
00:56:28 Brooklyn Zelenka: Or, it comes automatically
00:58:54 Marc-Antoine Parent: Event queue slices are important for this
00:59:06 Marc-Antoine Parent: (slice defined by search, if you will)
01:01:29 Marc-Antoine Parent: calendly: which of the 4 topics?
01:01:47 Marc-Antoine Parent: Open Source Devs & Community?
01:03:13 Marc-Antoine Parent: I have plenty of opinions, just did not get there ;-p
01:05:51 Gyuri Lajos: CRDT?
01:07:01 James Walker: @Marc-Antoine Open Source Devs is perfect, yes :)
01:07:37 Boris Mann: @Marc-Antoine that’s me just trying to get people to self classify ;)
01:08:18 James Walker: Yeah, we’re a small enough team they all go (mostly) to the same place
01:09:27 Marc-Antoine Parent: Downside of datomic itself: 1. non FOSS, 2. the tuple is not addressable. (Can be solved with IPLD magic)
01:10:04 Marc-Antoine Parent: Two timelines actually: assertion time, target time
01:11:08 Helder S Ribeiro: Datomic Scuttlebutt
01:11:11 Marc-Antoine Parent: One "cell" at a time, to follow the table metaphor
01:12:28 Marc-Antoine Parent: downside is when you want ordered structure... but less frequent than it needs
01:12:31 Marc-Antoine Parent: seems
01:13:50 Gyuri Lajos: https://gitlab.com/american-space-software/orbit-db-mfsstore#annotations:yE2vAlDXEeu0iM_Vu2YEGw
01:14:01 Helder S Ribeiro: Highest-level API: steal from Firebase, AppSync, etc.? Maybe actually directly swappable?
01:14:51 James Walker: @Helder - we (I) started there. Interestingly, in some of our early interviews … lots of front-end devs haven’t actually used Firebase so don’t know / don’t care about it being the same
01:15:06 James Walker: (Obviously we’re still early in that process, but I found it surprising)
01:15:38 Helder S Ribeiro: Yeah, I guess it’s more like “whatever has easy-to-use libraries and lots of answers about it on SO”
01:15:57 Marc-Antoine Parent: GraphQL is not very graph-like in reality, it's just another way to consume tables
01:16:14 Marc-Antoine Parent: (No path queries)
01:16:40 David Luecke: II learned that people don’t care as much about familiarity (or even compatibility) as you’d think if the alternative makes sense and is easy to learn.
01:16:53 Marc-Antoine Parent: Interesting, David
01:16:56 James Walker: 👆
01:16:59 Gyuri Lajos: make search and sync across apps and across users pluggable at the API level
01:17:16 Gyuri Lajos: a market for these solutions too
01:17:48 Marc-Antoine Parent: +1 Gyuri
01:17:53 daniel: App developer API wise: people seem to like firebase very much
01:18:19 Samuel Andert: I am thinking about coda / air table like Ui interfaces:
01:18:20 Samuel Andert: https://coda.io/welcome
01:18:28 Samuel Andert: v
01:18:29 Helder S Ribeiro: “Ship your mocks” is an awesome tagline
01:18:30 Samuel Andert: https://airtable.com
01:19:51 Boris Mann: @Samuel — yeah for sure, also inspired by Parse, which got bought by FB and shut down
01:20:01 Helder S Ribeiro: “And 3. there’s people to help me when it gets over my head”
01:20:17 Boris Mann: _Most_ people can work with a spreadsheet like interface
01:20:27 Boris Mann: Joins is where this gets tricky
01:20:57 Marc-Antoine Parent: I believe all joins need to be materialized
01:21:37 Boris Mann: I’m already using insider talk here. Someone who can only reason about a spreadsheet has challenges making the leap to “database’, This is where the curse of knowledge of having a long SQL background is a problem :P
01:21:49 Brooklyn Zelenka: 100%
01:24:29 Marc-Antoine Parent: yes, middleware access must be optional
01:26:28 Helder S Ribeiro: One possible ideal API is no API: hook into this.setState() on React “and it just works!” https://reactjs.org/docs/state-and-lifecycle.html
01:28:38 Boris Mann: @helder definitely thinking about direct React support in different ways that just works / feels natural
01:28:40 Marc-Antoine Parent: That's one of my few issues with Underlay; lens resolution is fully dynamic, which I think is wasted recalculations, I would cache
01:28:43 David Luecke: Fun fact: Facebook does not do migrations, only append-only schema changes :)
01:29:15 Boris Mann: Yeah “migrations” in air quotes as it is the term that people vaguely know
01:31:20 Marc-Antoine Parent: 100% on operation-based CRDT
01:33:03 Helder S Ribeiro: Sqlite3 over WNFS would be a good first swing if I wanted to build an e2ee single-user app before WNDB exists
01:33:16 Boris Mann: @helder — you’re always like one step ahead of me
01:33:20 Boris Mann: Had the same thought :)
01:33:33 Brooklyn Zelenka: Actual Budget is doing something similar
01:33:33 Helder S Ribeiro: ;)
01:33:33 Marc-Antoine Parent: or you could be doing sync a lower level in libp2p?
01:33:47 Brooklyn Zelenka: https://actualbudget.com/blog/porting-local-app-web
01:34:49 daniel: SQLite in WNFS sounds nice :)
01:35:43 Marc-Antoine Parent: I'm told hypercore is closer to real-time.
01:36:49 Boris Mann: @marc-antoine libp2p is sort of “inside/underneath” js-ipfs, but mostly all of the security and versions and everything else happens at the IPFS/WNFS layer where
01:37:06 Marc-Antoine Parent: fair