This is a troll, right? The strace for "ls" on my system consists of 1) checking if selinux is enabled (which their re-implementation does not do), 2) openat -> fstat -> getdents64, 3) writing out the information those syscalls gave you.
Jumping from "oh my gosh you can (incorrectly) reimplement ls from a trace of syscalls" to "this means the end of binary blobs" is a level of detachment from reality I've most recently seen from Geoff Lewis. Never mind that the author mixes up eBPF tracing and strace.
shric · 17h ago
This is a weird post as the generated program doesn’t even output results.
I tried a similar test using Python and Claude Code and it generated something that outputted a sorted list of directory contents, like ls.
However, unlike the author, this doesn’t lead me to think this is AGI. How many posts are there out there that these models would be trained on have people explaining strace outputs and comparing code to them?
h4ny · 17h ago
Why is this impossible? If the LLM has effectively seen all the code that could lead to that trace, all it has to do is to pick the bits out that has the highest chance to map to it right?
Is it a new marketing strategy to start by saying you're "incredibly cynical" about something that you're going to say the exact opposite about, perhaps to mask arguments with little rigor?
I think you can significantly save on your claude tokens if you just give the llm `head -n 1 TRACE`
execve("/usr/bin/ls", ["ls"], 0x7fff7e969540 /* 110 vars */)
Some how it figures out how to implement `ls`. That indeed shouldn't be possible. It puts an AI skeptic into believing we're in post-AGI territory here.
Edit: tbf, I did not notice they replaced `ls` with `lol` in the trace file to avoid this, but still. To think the strace of ls was never part of the training data of the LLM then to think that is enough to somehow reverse engineer what binary blob is is insane thinking https://github.com/ghuntley/strace-to-application/blob/trunk...
treetalker · 17h ago
If you would, please explain why it shouldn't be possible (I am not a software engineer).
eddythompson80 · 17h ago
I was being sarcastic. The author asked an LLM to implement `ls`* and was so surprised when it did that he thought maybe he can use that insight into reverse engineering firmware blobs. Complete delusion, but chatbots have that effect on some people.
* if you don't know what `ls` is, just think like implementing the simplest possible application that's 2 or 3 lines of code.
bloppe · 17h ago
To be fair, the author claims he removed any mention of "ls" from the file. But also to be fair, the post is annoyingly click-baity. I'd be impressed if the LLM could generate more than 2 lines of code from a strace.
plorkyeran · 14h ago
The LLM also failed to implement it. The rust code produced doesn’t actually result in the same syscalls as the strace contained.
queenkjuul · 1h ago
It doesn't even print the results lmao
egamirorrim · 15h ago
This guy's such a weird writer. Speaking like he discovered YOLO mode and called it the Ralph Wiggum method.
This was underwhelming, but I like his websites styling.
satisfice · 17h ago
Why does this guy keep using words like “skeptical” and then demonstrating the opposite attitude?
Jumping from "oh my gosh you can (incorrectly) reimplement ls from a trace of syscalls" to "this means the end of binary blobs" is a level of detachment from reality I've most recently seen from Geoff Lewis. Never mind that the author mixes up eBPF tracing and strace.
I tried a similar test using Python and Claude Code and it generated something that outputted a sorted list of directory contents, like ls.
However, unlike the author, this doesn’t lead me to think this is AGI. How many posts are there out there that these models would be trained on have people explaining strace outputs and comparing code to them?
Is it a new marketing strategy to start by saying you're "incredibly cynical" about something that you're going to say the exact opposite about, perhaps to mask arguments with little rigor?
Edit: tbf, I did not notice they replaced `ls` with `lol` in the trace file to avoid this, but still. To think the strace of ls was never part of the training data of the LLM then to think that is enough to somehow reverse engineer what binary blob is is insane thinking https://github.com/ghuntley/strace-to-application/blob/trunk...
* if you don't know what `ls` is, just think like implementing the simplest possible application that's 2 or 3 lines of code.
This was underwhelming, but I like his websites styling.
Do I have to do all the skepticism myself? FFS!