Hey everyone, welcome to our new WNDB forum!
We had our first public research and design session last week. For those who may have missed it, we have posted the full video:
A huge thank you to everyone who participated!
I’ve tried to compile some high level notes (feel free to add anything I may have missed in the comments!):
- As we had predicted, the group was a mix of people who are interested in using WNDB as app developers (who have feature and API design interests) and those interested in the protocol / “plumbing” of WNDB
- There were not a lot of strong feelings about the nature of the API - rather, WNDB needs features and documentation on how to implement them
- There is a lot of interest / consensus around our initial event sourcce-based design ideas. @expede highlighted datomic and datascript.
- A lot of good (early) discussion around schema management: developers should be able to define them (but also share and re-use), they should be applied as “views”, and lots of interest in project cambria
Moving forward, we will most likely split sessions between the API level and the protocol level. Keep an eye on #events for announcements. This forum will include any discussion and announcements as we go.
Here’s the chat log from the session as well:
Here’s the full chat log from the event as well. There were several good links shared:
00:08:40 Samuel Andert: GraphQL 00:10:44 Marc-Antoine Parent: I think we can have both... 00:17:51 Gyuri Lajos: can we have the whole screen to show people? 00:19:16 Gyuri Lajos: I meant to write please 00:20:01 Brooklyn Zelenka: Guyri: not sure I follow. Do you want one of the slides? 00:20:54 Boris Mann: @brookly — Gyuri wanted the screen share gone so we can see the entire gallery, which James already did 00:23:48 Marc-Antoine Parent: underlay 00:23:51 Helder S Ribeiro: Hadn’t heard of it 00:23:55 Helder S Ribeiro: Thanks for the ref! 00:24:36 ben: sorry, I had to jump out for a call, back 00:25:24 Helder S Ribeiro: https://www.underlay.org/ 00:26:02 Marc-Antoine Parent: my own work (but the site is out of date): https://hyperknowledge.org 00:27:42 Helder S Ribeiro: 👀 00:29:19 Helder S Ribeiro: https://www.inkandswitch.com/cambria.html 00:29:37 Gyuri Lajos: Grakn: The Knowledge Graph 00:29:38 Gyuri Lajos: https://grakn.ai/ 00:29:51 Gyuri Lajos: for inspiration 00:30:29 Marc-Antoine Parent: Schema as a view 00:31:35 Marc-Antoine Parent: Underlay gets into the process of translating free-form RDF-ish into algebraic types, which could correspond to schemas 00:32:28 Marc-Antoine Parent: The dynamic of collective data means having to handle social data conflicts 00:34:38 Marc-Antoine Parent: Note: even with schema as view, we can let people use the schema for write operations, as long as the write is treated as an event, and decomposed into schema-independent microclaims 00:35:05 Brooklyn Zelenka: Yeah Marc-Antoine, this very much follows my current thinking 00:35:46 Marc-Antoine Parent: Brooklyn: Glad to hear it! 00:35:53 Brooklyn Zelenka: Thought currently more thinking of EAVT than events (though we had started at event sourcing) 00:36:12 Helder S Ribeiro: Related https://typedefs.com/about/ 00:36:13 ben: it is a great idea, instant twitter! :) 00:37:10 Helder S Ribeiro: What’s EAVT? 00:37:24 Brooklyn Zelenka: https://tonsky.me/blog/unofficial-guide-to-datomic-internals/ 00:37:28 Marc-Antoine Parent: I'm seeing EAV(T?) micro-claims as bundled in events... 00:37:46 Brooklyn Zelenka: Yep, me too 00:37:55 Helder S Ribeiro: Nice, thanks! 00:38:03 Brooklyn Zelenka: Assertion and refutations only, extended with temporal logic (make claims about the future) 00:38:17 Brooklyn Zelenka: But that’s fundamentally “schemaless” 00:38:24 Brooklyn Zelenka: Anyway, I’ll get into this all later I’m sure ;) 00:38:49 Marc-Antoine Parent: Sounds great. Are the AEVT micro-claims accessible for you? I think that's extremely important. 00:39:11 Marc-Antoine Parent: (sorry I meant addressable) 00:39:18 Brooklyn Zelenka: Yep all addressable 00:42:56 Marc-Antoine Parent: That is it, right? https://github.com/feathersjs/feathers 00:43:12 Boris Mann: Yep! 00:43:20 Brooklyn Zelenka: Yeah; here’s the homepage https://feathersjs.com/ 00:44:51 Gyuri Lajos: IPLD does some amazing magic in the space of schemas must check it out 00:49:56 Samuel Andert: Just listening and learning today 00:50:45 Marc-Antoine Parent: micro claims are like addressable triples... 00:51:26 Marc-Antoine Parent: Custom schemas should also include custom indices, agreed 00:52:48 Gyuri Lajos: https://twitter.com/TrailHub1/status/1347137546904547328 00:52:50 Marc-Antoine Parent: schema views are a good layer on which to anchor subscriptions 00:52:54 Helder S Ribeiro: One thing I don’t know much about but haven’t heard mentioned and would just want to throw the wrench in there: atomic transactions? 00:53:28 Brooklyn Zelenka: Yes, we’re doing STM even in the filesystem. Atomicity required for sure 00:53:47 Steven Vandevelde: Sorry, currently eating ;) 00:54:02 Helder S Ribeiro: STM 👍 00:55:54 Marc-Antoine Parent: When you have branch fork-merge semantics, atomicity may not be quite as important (unless you cherry-pick). 00:56:28 Brooklyn Zelenka: Or, it comes automatically 00:58:54 Marc-Antoine Parent: Event queue slices are important for this 00:59:06 Marc-Antoine Parent: (slice defined by search, if you will) 01:01:29 Marc-Antoine Parent: calendly: which of the 4 topics? 01:01:47 Marc-Antoine Parent: Open Source Devs & Community? 01:03:13 Marc-Antoine Parent: I have plenty of opinions, just did not get there ;-p 01:05:51 Gyuri Lajos: CRDT? 01:07:01 James Walker: @Marc-Antoine Open Source Devs is perfect, yes :) 01:07:37 Boris Mann: @Marc-Antoine that’s me just trying to get people to self classify ;) 01:08:18 James Walker: Yeah, we’re a small enough team they all go (mostly) to the same place 01:09:27 Marc-Antoine Parent: Downside of datomic itself: 1. non FOSS, 2. the tuple is not addressable. (Can be solved with IPLD magic) 01:10:04 Marc-Antoine Parent: Two timelines actually: assertion time, target time 01:11:08 Helder S Ribeiro: Datomic Scuttlebutt 01:11:11 Marc-Antoine Parent: One "cell" at a time, to follow the table metaphor 01:12:28 Marc-Antoine Parent: downside is when you want ordered structure... but less frequent than it needs 01:12:31 Marc-Antoine Parent: seems 01:13:50 Gyuri Lajos: https://gitlab.com/american-space-software/orbit-db-mfsstore#annotations:yE2vAlDXEeu0iM_Vu2YEGw 01:14:01 Helder S Ribeiro: Highest-level API: steal from Firebase, AppSync, etc.? Maybe actually directly swappable? 01:14:51 James Walker: @Helder - we (I) started there. Interestingly, in some of our early interviews … lots of front-end devs haven’t actually used Firebase so don’t know / don’t care about it being the same 01:15:06 James Walker: (Obviously we’re still early in that process, but I found it surprising) 01:15:38 Helder S Ribeiro: Yeah, I guess it’s more like “whatever has easy-to-use libraries and lots of answers about it on SO” 01:15:57 Marc-Antoine Parent: GraphQL is not very graph-like in reality, it's just another way to consume tables 01:16:14 Marc-Antoine Parent: (No path queries) 01:16:40 David Luecke: II learned that people don’t care as much about familiarity (or even compatibility) as you’d think if the alternative makes sense and is easy to learn. 01:16:53 Marc-Antoine Parent: Interesting, David 01:16:56 James Walker: 👆 01:16:59 Gyuri Lajos: make search and sync across apps and across users pluggable at the API level 01:17:16 Gyuri Lajos: a market for these solutions too 01:17:48 Marc-Antoine Parent: +1 Gyuri 01:17:53 daniel: App developer API wise: people seem to like firebase very much 01:18:19 Samuel Andert: I am thinking about coda / air table like Ui interfaces: 01:18:20 Samuel Andert: https://coda.io/welcome 01:18:28 Samuel Andert: v 01:18:29 Helder S Ribeiro: “Ship your mocks” is an awesome tagline 01:18:30 Samuel Andert: https://airtable.com 01:19:51 Boris Mann: @Samuel — yeah for sure, also inspired by Parse, which got bought by FB and shut down 01:20:01 Helder S Ribeiro: “And 3. there’s people to help me when it gets over my head” 01:20:17 Boris Mann: _Most_ people can work with a spreadsheet like interface 01:20:27 Boris Mann: Joins is where this gets tricky 01:20:57 Marc-Antoine Parent: I believe all joins need to be materialized 01:21:37 Boris Mann: I’m already using insider talk here. Someone who can only reason about a spreadsheet has challenges making the leap to “database’, This is where the curse of knowledge of having a long SQL background is a problem :P 01:21:49 Brooklyn Zelenka: 100% 01:24:29 Marc-Antoine Parent: yes, middleware access must be optional 01:26:28 Helder S Ribeiro: One possible ideal API is no API: hook into this.setState() on React “and it just works!” https://reactjs.org/docs/state-and-lifecycle.html 01:28:38 Boris Mann: @helder definitely thinking about direct React support in different ways that just works / feels natural 01:28:40 Marc-Antoine Parent: That's one of my few issues with Underlay; lens resolution is fully dynamic, which I think is wasted recalculations, I would cache 01:28:43 David Luecke: Fun fact: Facebook does not do migrations, only append-only schema changes :) 01:29:15 Boris Mann: Yeah “migrations” in air quotes as it is the term that people vaguely know 01:31:20 Marc-Antoine Parent: 100% on operation-based CRDT 01:33:03 Helder S Ribeiro: Sqlite3 over WNFS would be a good first swing if I wanted to build an e2ee single-user app before WNDB exists 01:33:16 Boris Mann: @helder — you’re always like one step ahead of me 01:33:20 Boris Mann: Had the same thought :) 01:33:33 Brooklyn Zelenka: Actual Budget is doing something similar 01:33:33 Helder S Ribeiro: ;) 01:33:33 Marc-Antoine Parent: or you could be doing sync a lower level in libp2p? 01:33:47 Brooklyn Zelenka: https://actualbudget.com/blog/porting-local-app-web 01:34:49 daniel: SQLite in WNFS sounds nice :) 01:35:43 Marc-Antoine Parent: I'm told hypercore is closer to real-time. 01:36:49 Boris Mann: @marc-antoine libp2p is sort of “inside/underneath” js-ipfs, but mostly all of the security and versions and everything else happens at the IPFS/WNFS layer where 01:37:06 Marc-Antoine Parent: fair