Fine-tuning LLM Agents without Fine-tuning LLMs