OpenFront: Realtime Risk-like multiplayer game in the browser (openfront.io)

(I evaluated semantic diff tools for use in Brokk but I ultimately went with standard textual diff; the main hangup that I couldn't get past is that semantic diff understandably works very poorly when you have a syntactically invalid file due to an in-progress edit.)

affyboi · 12h ago

Note that diffsitter isn’t abandoned or anything. I took a year off working and just started a new job so I’ve been busy. I’ve got a laundry list of stuff I want to do with this project that will get done (at some point)

aiiizzz · 2h ago

This is cool, I really think this kind of thing integrated with LLMs for code editing will be wonderful. Days of manually typing code are coming to an end. I was looking for something better than meld, and this might be it.

ilyagr · 13h ago

In case anybody happens to be interested in testing `gumtree` with https://github.com/jj-vcs/jj, I think I got them to work together. See https://github.com/GumTreeDiff/gumtree/wiki/VCS-Integration#... (assumes Docker).

pfdietz · 17h ago

The interesting problem here would be how do you produce a robust parse tree for invalid inputs, in the sense of stably parsing large sections of the text in ways that don't change too much. The tree would have to be an extension of an actual parse tree, with nodes indicating sections that couldn't be fully parsed or had errors. The diff algorithm would have to also be robust in the face of such error nodes.

For the parsing problem, maybe something like Early's algorithm that tries to minimize an error term?

You need this kind of robust parser for languages with preprocessors.

o11c · 12h ago

Unfortunately, this depends on making good decisions during language design; it's not something you can retrofit with a new lexer and parser.

One very important rule is: no token can span more than one (possibly backslash-extended) line. This means having neither delimited comments (use multiple single-line comments; if your editor is too dumb for this you really need a new editor) nor multi-line strings (but you can do implicit concatenation of a string literal flavor that implicitly includes the newline; as a side-effect this fixes the indentation problem).

If you don't follow this rule, you might as well give up on robustness, because how else are you going to ever resynchronize after an error?

For parsing you can generally just aggressively pop on mismatched parens, unexpected semicolons, or on keywords only allowed in a top-ish level context. Of course, if your language is insane (like C typedefs), you might not be able to parse the next top-level function/class anyway. GNU statement-expressions, by contrast, are an actually useful thing that requires some thought. But again, language design choices can mitigate this (such as making classes values, template argument equivalent to array indexing, and statements expressions).

pfdietz · 12h ago

> how else are you going to ever resynchronize after an error?

An error-cost-minimizing dynamic programming parser could do this.

o11c · 10h ago

That fundamentally misunderstands the problem in multiple ways:

* this is still during lexing, not yet to parsing

* there are multiple valid token sequences that vary only with a single character at the start of the file. This is very common with Python multi-line strings in particular, since they are widely used as docstrings.

pests · 18h ago

I watched a video long ago about how the Roslyn C# compiler handled this but I forget the details.

ilyagr · 13h ago

https://github.com/Wilfred/difftastic/wiki/Structural-Diffs is a nice list of alternatives.

Difftastic itself is great as well! The author wrote up nice posts about its design: https://www.wilfred.me.uk/blog/2022/09/06/difftastic-the-fan..., https://difftastic.wilfred.me.uk/diffing.html.

jacobr · 1h ago

Could the next-gen version control system just store ASTs? Does this already exist?

Every user gets their own preferred formatting, and linters and tools could operate on already-parsed trees

haradion · 37m ago

The Unison programming language is built around that idea: https://www.unison-lang.org/docs/the-big-idea/

vrm · 15h ago

This is neat! I think in general there are really deep connections between semantically meaningful diffs (across modalities) and supervision of AI models. You might imagine a human-in-the-loop workflow where the human makes edits to a particular generation and then those edits are used as supervision for a future implementation of that thing. We did some related work here: https://www.tensorzero.com/blog/automatically-evaluating-ai-... on the coding use case but I'm interested in all the different approaches to the problem and especially on less structured domains.

esafak · 16h ago

Some make a semantic diff splitter please! Break up big commits into small, atomic, meaningful ones.

ethan_smith · 13h ago

Check out git-imerge or git-absorb which can help with this problem by intelligently splitting or absorbing changes into the right commits.

alwillis · 11h ago

First time I used absorb was in Mercurial back in the day: https://gregoryszorc.com/blog/2018/11/05/absorbing-commit-ch...

0x457 · 15h ago

Well, that's what git-patch is: https://patch-diff.githubusercontent.com/raw/denoland/deno/p...

esafak · 14h ago

I can't make sense of that link. How many parts was the diff split up into, and along what lines?

0x457 · 14h ago

Yeah, I don't know why I linked that as an example. Wanted to show structure of a patch. Each commit of a patch already has everything ready to be processed and chunked IF you keep them - small, atomic, semantically meaningful. As in do smaller commits.

mdaniel · 6h ago

> > Some make a semantic diff splitter please! Break up big commits into small, atomic, meaningful ones.

> Each commit of a patch already has everything ready to be processed and chunked IF you keep them - small, atomic, semantically meaningful. As in do smaller commits.

Reads like:

User1: I need help with my colleagues who do not make independent, small, semantically intact commits

User2: well, have you tried making smaller, more independent, semantically intact commits?

---

My interpretation of the wish is to convert this, where they have intermixed two semantically independent changes in one diff:

    +++ a/alpha.py
    --- b/alpha.py

     def doit():
    -    awesome = 3.14
    +    awesome = 4.56

    -    print("my dog is fluffy")
    +    print("my cat is fluffy")

into this

    +++ a/alpha.py
    --- b/alpha.py

     def doit():
    -    awesome = 3.14
    +    awesome = 4.56

         print("my dog is fluffy")

    +++ a/alpha.py
    --- b/alpha.py

     def doit():
         awesome = 3.14

    -    print("my dog is fluffy")
    +    print("my cat is fluffy")

where each one could be cherry-picked at will because they don't semantically collide

The semantics part would be knowing that this one could not be split in that manner, because the cherry-pick would change more than just a few lines, it would change the behavior

    +++ a/alpha.py
    --- b/alpha.py

     def doit():
    -    the_weight = 3.14
    +    the_weight = 4.56

    -    print("my dog weighs %f", the_weight)
    +    print("my cat weighs %f", the_weight)

I'm sure these are very contrived examples, but it's the smallest one I could whip up offhand

fjfaase · 22h ago

Discussed before on https://news.ycombinator.com/item?id=27875333

koozz · 18h ago

I thought I’ve seen it before. I use Difftastic myself, amazing diffs. https://github.com/Wilfred/difftastic

alwillis · 11h ago

Same.

john_max_1 · 3h ago

How does it compare to diffmerge?

mertleee · 8h ago

This is really cool.

Although - for more exotic applications parsing structural data I've found langium is far more capable as a platform. Typescript is also a pleasant departure from common AST tools.

1-more · 13h ago

pmkary · 15h ago

What a genius idea.

affyboi · 12h ago

Nah I think most people could make something like this in a weekend

dcre · 15h ago

See also https://mergiraf.org/ for a tool that uses ASTs to resolve (some) merge conflicts.

the__alchemist · 17h ago

Is there an anti-tree-sitter version too?

davepeck · 17h ago

yes, although it's sort of the same as Context-Free-Typing-sitter

Iwan-Zotow · 15h ago

integration to VSCODE?