|
| yewenjie wrote:
| Is this supposed to be run on the server? Then how does it really
| solve the frontend side of issues, I'm just trying to understand.
| ranting-moth wrote:
| I sometimes see interesting links on HN but when I click on
| them and skim through, I still have no idea what exactly it
| does. This is one of them.
| hk__2 wrote:
| It's not exactly clear in the article, but there is a client
| part:
| https://github.com/orbitinghail/sqlsync/blob/main/GUIDE.md
|
| > Step 2: Install and configure the React library
| jokethrowaway wrote:
| Not involved with the project but - this is a database which
| run client side and sync with a database on the server
| carlsverre wrote:
| Precisely! The same database (SQLite) is running on both the
| client and the server. SQLSync provides a custom storage
| layer to SQLite that keeps everything in sync. As changes are
| made locally (optimistically) they are synced into the cloud
| where they are eventually applied to the primary db replica.
| The clients subscribe to changes from the primary db and then
| rebase themselves to stay in sync.
|
| I really need to write up a detailed overview of how this
| works! Thanks for the feedback!
| HatchedLake721 wrote:
| Unless I misunderstood, feels like I've been doing this with
| Ember Data since ~2013.
|
| https://guides.emberjs.com/release/models/
|
| There's also https://orbitjs.com/
| no_wizard wrote:
| Ember.js has had more innovations contributed to modern
| framework ideas than any other framework, really.
|
| EmberData, Ember Routing, Ember Multi Applications (can't
| remember what its called, but its a precursor to
| microfrontends) all in one CLI tooling etc.
|
| I could never understand what holds Ember back from being more
| used. I think it used to be performance but I think they
| addressed that with Glimmer many years ago.
| robocat wrote:
| I think you have misunderstood?
|
| The article is responding to the pattern of yet another custom
| data model and custom data API (a la Ember).
|
| Instead provide an SQL database (the well proven SQLite) within
| the front end and use SQL to interact. And sync data from-to
| the backend DB.
|
| Which one could then slap on a model or ORM layer on top of -
| should that be one's bent.
|
| It isn't clear how they manage the subscription to data
| updates/inserts/deletions - it mentions supporting triggers,
| but that feels icky to me.
| carlsverre wrote:
| First, thanks!
|
| > It isn't clear how they manage the subscription to data
| updates/inserts/deletions - it mentions supporting triggers,
| but that feels icky to me.
|
| Architecture post coming soon. In the meantime, I want to
| clarify that SQLSync does not use triggers for sync. Instead,
| I hijack SQLites page storage and added page replication to
| it. Writes are consolidated through an API I call the
| "reducer" which allows SQLSync to keep track of which logical
| writes correspond to which sets of page changes. The actual
| sync is pretty dumb: we run the reducer on both the client
| and the server. The client replicates down server pages, and
| then periodically throws out local changes, resets to the
| server state, and then replays any mutations that haven't yet
| been acked on the server.
| frenchman99 wrote:
| You say things like "X is pretty dumb" and then go on
| saying stuff I don't understand. Pretty annoying if you ask
| me.
|
| And that's despite me having worked with Cassandra, Kafka,
| Postgres and a variety of programming languages, DevOps
| tools, having worked with Vuejs and React.
| mst wrote:
| Could you clarify which of:
|
| - page storage
|
| - reducers
|
| - replaying mutations
|
| - acks
|
| you're unclear on?
| matlin wrote:
| I'm currently writing a very similar article about "full-stack
| databases" which highlights the same pattern where many apps end
| recreating the logic of our backend and database in the frontend
| client code. The solution we're promoting is to choose a database
| that can run on both the server and in the client and then sync
| between them.
|
| The reason we aren't using Sqlite for our product is because Sql
| is frankly not the right tool for querying data for an
| application. It doesn't easily map to the data-structures you
| want in your client code and nearly all SQL databases have no way
| to subscribe to changes to a query without polling the query
| repeatedly.
|
| So if you like the idea of having a complete database on your
| client but also want deep integration with Typescript/Javascript
| check out what we're building at https://github.com/aspen-
| cloud/triplit
| johnny22 wrote:
| postgres has some capability to do that, but does need a
| server.
| matlin wrote:
| Yeah you can subscribe to overall changes to the data on a
| row by row basis but can't subscribe to an actual query. Many
| apps and libraries imitate reactive queries by just
| refetching all queries from Postgres when any data changes or
| just repeatedly polling the query every 10 seconds or so but
| this puts a lot of strain on the database. You can just
| subscribe to the replication stream but then you're left
| trying to reconstruct your queries in your application code
| which is extremely error prone and painful
| culi wrote:
| I like the implications of this to a "local first" architecture
| pqdbr wrote:
| Can someone explain me how it's syncing the state between two
| different devices without any activity in the Network tab in
| DevTools, not even WS traffic?
|
| I get that you can sync state between browser tabs, but I'm
| trying on two different devices (iPhone and Desktop).
|
| And as far as I can tell, the Wasm layer can't perform network
| requests directly.
|
| UPDATE: In the console tab I can see 'coordinatorUrl:
| 'wss://sqlsync.orbitinghail.workers.dev', but I was expecting to
| see this Websockets connection in the Network tab, and it isn't.
| jasonjmcghee wrote:
| Websockets tracking in the browser can be weird. Try refreshing
| the page while you have WS selected in the Network tab
| pqdbr wrote:
| I did that!
| carlsverre wrote:
| Good catch! SQLSync runs in a shared worker to enable cross-tab
| reactivity and centralise both local storage and the
| replication system. You can inspect the worker at the magic
| url: chrome://inspect/#workers
| tootie wrote:
| The thing I've used for this kind of problem is fusejs which is a
| lightweight search index. You can load it with a list of JSON
| documents and do structured or fuzzy string searches. I find it
| pretty well-suited to the kind of frontend experiences I need a
| lot of data for.
| carlsverre wrote:
| This is cool! Thanks for sharing. Sounds like Fuse would be a
| great solution for a read-only index. But what if you want to
| collaborate on the data with other people?
|
| FWIW check out SQLite's full text search extension:
| https://www.sqlite.org/fts5.html
| recusive_story wrote:
| Local to a webpage, do you feel building wrapper over indexedDB
| instead of sqlite would be better idea?
| carlsverre wrote:
| That's a great way to accomplish local storage, but requires a
| bit of gymnastics to build sync. By controlling the database
| entirely, SQLSync can provide very ergonomic sync to the
| developer and performance for the user.
|
| So it's not that one is better than the other. Just the
| capabilities, performance, and test-ability differs.
| bob1029 wrote:
| Trying to synchronize state between client & server is a cursed
| problem.
|
| You can sidestep it altogether if you make mild UX sacrifices and
| revert to something more akin to the PHP/SSR model. SPA is nice,
| but multipart form posts still work. Just the tiniest amount of
| javascript can smooth out most of the remaining rough edges.
|
| Our latest web products utilize the following client-side state:
| 3rd party IdP claims for auth 1st party session id in query
| args The current document
|
| For the first item, I genuinely don't even know where this is
| stored. It's Microsoft's problem, not ours. All other state lives
| on the server. We treat the client more or less like a dumb
| terminal that punches |