One thing that is interesting about this is that I wasn't able to get good results from smaller/faster models like claude haiku, so I opted to use a larger model instead. I found that the small delay of about a second was worth it for more consistent results.
This was pretty interesting to implement!
- I used an [lsp server](https://github.com/dlants/magenta.nvim/pull/162/files#diff-3...) to track opened files and aggregate text changes to get a stream of diffs.
- I then feed that along with the [context surrounding the cursor](https://github.com/dlants/magenta.nvim/pull/162/files#diff-1...), and a [system prompt](https://github.com/dlants/magenta.nvim/pull/162/files#diff-a...) into an LLM, [forcing a tool use for a find/replace within the context window](https://github.com/dlants/magenta.nvim/pull/162/files#diff-1...)
- Finally, I show the find/replace in the buffer using [virtual text extmarks](https://github.com/dlants/magenta.nvim/pull/162/files#diff-1...), applying a comment effect to the added sections, and a strikethrough to the removed sections
One thing that is interesting about this is that I wasn't able to get good results from smaller/faster models like claude haiku, so I opted to use a larger model instead. I found that the small delay of about a second was worth it for more consistent results.
I also opted to have this be manually triggered (Shift-Ctrl-l by default in insert or normal mode). This is a lot [less distracting](https://unstable.systems/@sop/114898566686215926).
One cool thing is that you can use a plugin parameter, or a project-level parameter to [append to the system prompt](https://github.com/dlants/magenta.nvim/blob/main/node/option...). I think by providing additional [examples of how you want it to behave](https://github.com/dlants/magenta.nvim/blob/main/node/provid...), you can have it be a lot more useful for your specific use-case.