I developed an open-source task management software based on CRDT with a local-first approach. The motivation was that I primarily manage personal tasks without needing collaboration features, and tools like Linear are overly complex for my use case.
This architecture offers several advantages:
1. Data is stored locally, resulting in extremely fast software response times
2. Supports convenient full database export and import
3. Server-side logic is lightweight, requiring minimal performance overhead and development complexity, with all business logic implemented on the client
4. Simplified feature development, requiring only local logic operations
There are also some limitations:
1. Only suitable for text data storage; object storage services are recommended for images and large files
2. Synchronization-related code requires extra caution in development, as bugs could have serious consequences
3. Implementing collaborative features with end-to-end encryption is relatively complex
The technical architecture is designed as follows:
1. Built on the Loro CRDT open-source library, allowing me to focus on business logic development
2. Data processing flow:
User operations trigger CRDT model updates, which export JSON state to update the UI. Simultaneously, data is written to the local database and synchronized with the server.
3. The local storage layer is abstracted through three unified interfaces (list, save, read), using platform-appropriate storage solutions: IndexedDB for browsers, file system for Electron desktop, and Capacitor Filesystem for iOS and Android.
4. Implemented end-to-end encryption and incremental synchronization. Before syncing, the system calculates differences based on server and client versions, encrypts data using AES before uploading. The server maintains a base version with its content and incremental patches between versions. When accumulated patches reach a certain size, the system uploads an encrypted full database as the new base version, keeping subsequent patches lightweight.
> For the uninitiated, Linear is a project management tool that feels impossibly fast. Click an issue, it opens instantly. Update a status and watch in a second browser, it updates almost as fast as the source. No loading states, no page refreshes - just instant, interactions.
How garbage the web has become for a low-latency click action being qualified as "impossibly fast". This is ridiculous.
lwansbrough · 18m ago
Trite remark. The author was referring to behaviour that has nothing to do with “how the web has become.”
It is specifically to do with behaviour that is enabled by using shared resources (like IndexedDB across multiple tabs), which is not simple HTML.
To do something similar over the network, you have until the next frame deadline. That’s 8-16ms. RTT. So 4ms out and back, with 0ms budget for processing. Good luck!
jitl · 1h ago
A web request to a data center even with a very fast backend server will struggle to beat 8ms (120hz display) or even 16ms (60hz display), the budget for next frame painting a navigation. You need to have the data local to the device and ideally already in memory to hit 8ms navigation.
ahofmann · 1h ago
This is not the point, or other numbers matter more, then yours.
In 2005 we wrote entire games for browsers without any frontend framework (jQuery wasn't invented yet) and managed to generate responses in under 80 ms in PHP. Most users had their first bytes in 200 ms and it felt instant to them, because browsers are incredibly fast, when treated right.
So the Internet was indeed much faster then, as opposed to now. Just look at GitHub. They used to be fast. Now they rewrite their frontend in react and it feels sluggish and slow.
DanielHB · 24m ago
Unless you are running some really complicated globally distributed backend your roundtrip will always be higher than 80ms for all users outside your immediate geographical area. And the techniques to "fix" this usually only mitigate the problem in read-scenarios.
The techniques Linear uses are not so much about backend performance and can be applicable for any client-server setup really. Not a JS/web specific problem.
ahofmann · 13m ago
My take is, that a performant backend gets you so much runway, that you can reduce a lot of complexity in the frontend. And yes, sometimes that means to have globally distributed databases.
But the industry is going the other way. Building frontends that try to hide slow backends and while doing that handling so much state (and visual fluff), that they get fatter and slower every day.
Zanfa · 53m ago
> Now they rewrite their frontend in react and it feels sluggish and slow.
And decided to drop legacy features such as <a> tags and broke browser navigation in their new code viewer. Right click on a file to open in a new tab doesn’t work.
mentalgear · 59m ago
Local-First & Sync-Engines are the future. Here's a great filterable datatable overview of the local-first framework landscape:
https://www.localfirst.fm/landscape
My favorite so far is Triplit.dev (which can also be combined with TanStack DB); 2 more I like to explore are PowerSync and NextGraph. Also, the recent LocalFirst Conf has some great videos, currently watching the NextGraph one (https://www.youtube.com/watch?v=gaadDmZWIzE).
petralithic · 1h ago
ElectricSQL and TanStack DB are great, but I wonder why they focus so much on local first for the web over other platforms, as in, I see mobile being the primary local first use case since you may not always have internet. In contrast, typically if you're using a web browser to any capacity, you'll have internet.
Also the former technologies are local first in theory but without conflict resolution they can break down easily. This has been from my experience making mobile apps that need to be local first, which led me to using CRDTs for that use case.
jitl · 1h ago
Because building local first with web technologies is like infinity harder than building local first with native app toolkits.
Native app is installed and available offline by default. Website needs a bunch of weird shenanigans to use AppManifest or ServiceWorker which is more like a bunch of parts you can maybe use to build available offline.
Native apps can just… make files, read and write from files with whatever 30 year old C code, and the files will be there on your storage. Web you have to fuck around with IndexedDB (total pain in the ass), localStorage (completely insufficient for any serious scale, will drop concurrent writes), or OriginPrivateFileSystem. User needs to visit regularly (at least once a month?) or Apple will erase all the local browser state. You can use JavaScript or hit C code with a wrench until it builds for WASM w/ Emscripten, and even then struggle to make sync C deal with waiting on async web APIs.
Apple has offered CoreData + CloudKit since 2015, a completed first party solution for local apps that sync, no backend required. I’m not a Google enthusiast, maybe Firebase is their equivalent? Idk.
agos · 36m ago
and if you didn't like or cared to learn CoreData? just jam a sqlite db in your application and read from it, it's just C. This was already working before Angular or even Backbone
terencege · 59m ago
I'm also building a local first editor and rolling my own CRDTs. There are enormous challenges to make it work. For example the storage size issue mentioned in the blog, I end up using with yjs' approach which only increase the clock for upsertion, and for deletion remove the content and only remain deleted item ids which can be efficiently compressed since most ids are continuous.
jddj · 46m ago
In case you missed it and it's relevant, there was an automerge v3 announcement posted the other day here which claimed some nice compression numbers as well
terencege · 34m ago
As far as I know, automerge is using DAG history log and garbage collecting by comparing the version clock heads of 2 clients. That is different than yjs. I have not followed their compression approach in v3 yet, will check if having time.
mkarliner · 1h ago
Meteor was/is a very similar technology. And I did some fairly major projects with it.
mentalgear · 55m ago
Meteor was amazing, I don't understand why it never got sustainable traction.
mbaranturkmen · 1h ago
How is this approach better than using react-query to persist storage which periodically sync the local storage and the server storage? Perhaps I am missing something.
petralithic · 1h ago
That approach is precisely what the new TanStack DB does, which if you don't know already has the same creator as React Query. The former extends the latter's principles to syncing, via ElectricSQL, both organizations have a partnership with each other.
Gravityloss · 2h ago
Some problem on the site. Too much traffic?
Secure Connection Failed
An error occurred during a connection to bytemash.net. PR_END_OF_FILE_ERROR
Error code: PR_END_OF_FILE_ERROR
jcusch · 2h ago
It looks like I was missing a www subdomain CNAME for the underlying github pages site. I think it's fixed now.
Gravityloss · 1h ago
I still see the same error
Gravityloss · 43m ago
Ok, it works, problem was probably on my end.
ivape · 3h ago
This is an ad.
dewey · 2h ago
It’s a developer writing about a tool they like. If you’d call word of mouth an “ad” the I guess it’s one.
jhy · 2h ago
Well, perhaps it's an AI writing about a tool...
> No API routes. No request/response cycles. No DTOs. Just… objects that magically sync. It kind of feels like cheating.
lambdas · 42m ago
That and the paragraph above:
> What makes this powerful is that these aren’t just type definitions - they’re live, reactive objects that sync automatically.
Is what twigged my AI radar too. LLM’s seem to really love that summarisation pattern of `{X is/isn’t just Y. Pithy concluding remark}`
jcusch · 7m ago
Fair enough, I thought what I'd originally written for that section was too wordy, so I asked Claude to rewrite it. I'll go a bit lighter on the AI editing next time. Here's most of the original with the code examples omitted:
Watching Tuomas' initial talk about Linear's realtime sync, one of the most appealing aspects of their design was the reactive object graph they developed. They've essentially made it possible for frontend development to be done as if it's just operating on local state, reading/writing objects in an almost Active Record style.
The reason this is appealing is that when prototyping a new system, you typically need to write an API route or rpc operation for every new interaction your UI performs. The flow often looks like:
- Think of the API operation you want to call
- Implement that handler/controller/whatever based on your architecture/framework
- Design the request/response objects and share them between the backend/frontend code
- Potentially, write the database migration that will unlock this new feature
Jazz has the same benefits of the sync + observable object graph. Write a schema in the client code, run the Jazz sync server or use Jazz Cloud, then just work with what feel like plane js objects.
ivape · 2h ago
Why do you think marketing is not sophisticated?
Here I’ll offer my services. I’ll pretend to do a technical deep dive of your app for X amount. No one will know, I’ll just act super interested.
When the fuck did anyone ever go “omg this web app so impressive”, never, ever, never, ever.
theglenn88_ · 2h ago
Wow dude you’re being needlessly abrasive.
Did you apply for a job at Linear and get rejected or something?
I think Linear is impressive, I’m a big fan of local first, I’ve built and app that is local first so this post is actually pretty interesting.
If you don’t like it, move along.
dewey · 2h ago
It’s a choice to always see the worst in everything.
Many blog post submissions here are someone diving into something they like, hardware, software, tool etc. and it’s just because people like to share.
jcusch · 2h ago
An ad for what? I'm not associated with any of the projects mentioned.
ivape · 2h ago
An ad for Linear.
defrost · 1h ago
If it's an ad for Linear why is so much text spent on Electric SQL, Zero, and Jazz?
ivape · 1h ago
Lol. You figured it out. It’s real. Takes notes everyone, this is how your promote, add some random shit at the bottom to “balance” out the sus.
There’s two types of people in this world - Rubes and Carnies.
mrkeen · 1h ago
In that worldview, it's unthinkable that you'd come and warn us about ads out of the goodness of your own heart.
This architecture offers several advantages:
1. Data is stored locally, resulting in extremely fast software response times 2. Supports convenient full database export and import 3. Server-side logic is lightweight, requiring minimal performance overhead and development complexity, with all business logic implemented on the client 4. Simplified feature development, requiring only local logic operations
There are also some limitations:
1. Only suitable for text data storage; object storage services are recommended for images and large files 2. Synchronization-related code requires extra caution in development, as bugs could have serious consequences 3. Implementing collaborative features with end-to-end encryption is relatively complex
The technical architecture is designed as follows:
1. Built on the Loro CRDT open-source library, allowing me to focus on business logic development
2. Data processing flow: User operations trigger CRDT model updates, which export JSON state to update the UI. Simultaneously, data is written to the local database and synchronized with the server.
3. The local storage layer is abstracted through three unified interfaces (list, save, read), using platform-appropriate storage solutions: IndexedDB for browsers, file system for Electron desktop, and Capacitor Filesystem for iOS and Android.
4. Implemented end-to-end encryption and incremental synchronization. Before syncing, the system calculates differences based on server and client versions, encrypts data using AES before uploading. The server maintains a base version with its content and incremental patches between versions. When accumulated patches reach a certain size, the system uploads an encrypted full database as the new base version, keeping subsequent patches lightweight.
If you're interested in this project, please visit https://github.com/hamsterbase/tasks
How garbage the web has become for a low-latency click action being qualified as "impossibly fast". This is ridiculous.
It is specifically to do with behaviour that is enabled by using shared resources (like IndexedDB across multiple tabs), which is not simple HTML.
To do something similar over the network, you have until the next frame deadline. That’s 8-16ms. RTT. So 4ms out and back, with 0ms budget for processing. Good luck!
In 2005 we wrote entire games for browsers without any frontend framework (jQuery wasn't invented yet) and managed to generate responses in under 80 ms in PHP. Most users had their first bytes in 200 ms and it felt instant to them, because browsers are incredibly fast, when treated right.
So the Internet was indeed much faster then, as opposed to now. Just look at GitHub. They used to be fast. Now they rewrite their frontend in react and it feels sluggish and slow.
The techniques Linear uses are not so much about backend performance and can be applicable for any client-server setup really. Not a JS/web specific problem.
But the industry is going the other way. Building frontends that try to hide slow backends and while doing that handling so much state (and visual fluff), that they get fatter and slower every day.
And decided to drop legacy features such as <a> tags and broke browser navigation in their new code viewer. Right click on a file to open in a new tab doesn’t work.
My favorite so far is Triplit.dev (which can also be combined with TanStack DB); 2 more I like to explore are PowerSync and NextGraph. Also, the recent LocalFirst Conf has some great videos, currently watching the NextGraph one (https://www.youtube.com/watch?v=gaadDmZWIzE).
Also the former technologies are local first in theory but without conflict resolution they can break down easily. This has been from my experience making mobile apps that need to be local first, which led me to using CRDTs for that use case.
Native app is installed and available offline by default. Website needs a bunch of weird shenanigans to use AppManifest or ServiceWorker which is more like a bunch of parts you can maybe use to build available offline.
Native apps can just… make files, read and write from files with whatever 30 year old C code, and the files will be there on your storage. Web you have to fuck around with IndexedDB (total pain in the ass), localStorage (completely insufficient for any serious scale, will drop concurrent writes), or OriginPrivateFileSystem. User needs to visit regularly (at least once a month?) or Apple will erase all the local browser state. You can use JavaScript or hit C code with a wrench until it builds for WASM w/ Emscripten, and even then struggle to make sync C deal with waiting on async web APIs.
Apple has offered CoreData + CloudKit since 2015, a completed first party solution for local apps that sync, no backend required. I’m not a Google enthusiast, maybe Firebase is their equivalent? Idk.
> No API routes. No request/response cycles. No DTOs. Just… objects that magically sync. It kind of feels like cheating.
> What makes this powerful is that these aren’t just type definitions - they’re live, reactive objects that sync automatically.
Is what twigged my AI radar too. LLM’s seem to really love that summarisation pattern of `{X is/isn’t just Y. Pithy concluding remark}`
Watching Tuomas' initial talk about Linear's realtime sync, one of the most appealing aspects of their design was the reactive object graph they developed. They've essentially made it possible for frontend development to be done as if it's just operating on local state, reading/writing objects in an almost Active Record style.
The reason this is appealing is that when prototyping a new system, you typically need to write an API route or rpc operation for every new interaction your UI performs. The flow often looks like: - Think of the API operation you want to call - Implement that handler/controller/whatever based on your architecture/framework - Design the request/response objects and share them between the backend/frontend code - Potentially, write the database migration that will unlock this new feature
Jazz has the same benefits of the sync + observable object graph. Write a schema in the client code, run the Jazz sync server or use Jazz Cloud, then just work with what feel like plane js objects.
Here I’ll offer my services. I’ll pretend to do a technical deep dive of your app for X amount. No one will know, I’ll just act super interested.
When the fuck did anyone ever go “omg this web app so impressive”, never, ever, never, ever.
Did you apply for a job at Linear and get rejected or something?
I think Linear is impressive, I’m a big fan of local first, I’ve built and app that is local first so this post is actually pretty interesting.
If you don’t like it, move along.
Many blog post submissions here are someone diving into something they like, hardware, software, tool etc. and it’s just because people like to share.
There’s two types of people in this world - Rubes and Carnies.
Who paid you for these comments? Atlassian?