Show HN: LLMuxer – Find the cheapest LLM that keeps your accuracy
1 mihir_ahuja 2 8/15/2025, 6:32:44 PM github.com ↗
I built LLMuxer because I kept defaulting to GPT-4o for everything, even simple tasks where a smaller, cheaper model would have done just fine.
It runs your prompts or dataset (currently for classification tasks) across multiple models, compares performance vs. cost, and recommends the best value so you’re not wasting tokens or budget.
Would love feedback and ideas for features you’d want before using it in your own workflow!
Comments (2)
mihir_ahuja · 8h ago
LLMuxer automates cost–accuracy trade-offs across LLM providers through OpenRouter, so you can easily benchmark and compare dozens of models with one API key.
mihir_ahuja · 8h ago
!pip install llmuxer
llmuxer run --dataset banking77_test.json --models openai:gpt-4o-mini openai:gpt-4o claude:claude-3-5