Show HN: Run automated ML experiments using Claude Code
The goal was largely to demonstrate that this is possible, specifically to:
* encourage to people who want to run some ML experiment but don't have time t code it to actually give it a try * provide evidence that LLM recursive self-improvement is not "science fiction"
The template is bare bones, it does not come with niceties for monitoring experiments, conduct experiments at scale, etc.
The script assumes that CUDA, Python, PyTorch are already set up. This is quite easy if you rent an instance from https://lambda.ai/ - that's pre-installed. You'd only need to install Claude Code (which itself requires npm) to get it going.
As I mentioned in the README, the most advanced experiment I tried so far is injection of sentence-embedding memory into a pre-trained transformer.
The timeline on https://ai-2027.com/ assumes that we'll only be able to get AI coding agents which can do ML experiments in 2026, but it seems like it is already possible now. (I spent only few hours on this, obviously proper AI labs can spend whole days on infrastructure, scaffolding, prompting, fine-tuning, etc.)
* fist iterate on the idea with o3 (best choice) or other big model (Opus 4, Gemini 2.5 Pro, Grok 3) -- ask it whether it was done before, how to improve it, what is the expected outcome, etc. o3 is really smart, it can explain intuition between different choices, etc. * Python packages are hard. Using virtual environment (venv) is recommended. `uv` is probably the modern way to manage venv, but installing torch with CUDA support via uv is pain, what I found works is: * `uv pip install torch --torch-backend=cu126` (uv pip uninstall torch) * lambda.ai provides high-quality environment, but it might lack cheaper GPU options. * as I mentioned in README, there's no sandboxing, Claude can do pretty much arbitrary stuff...