My AI skeptic friends are all nuts (fly.io)

Back when web development was taking off, there was always a go-to stack — something like Postgres + Django + jQuery, or .NET + Bootstrap, SQLITE. Over the years we had proven tech and proven patterns like : MVC, SPA etc...

Now that local LLMs are gaining traction, I’m wondering what the equivalent stack looks like today.

Models, Runtime, hardware and other tools.

That could rival the Claudes, ChatGPTs or Geminis, etc

Thanks

Comments (6)

bluejay2387 · 1d ago

2x 3090's running Ollama and VLLM... Ollama for most stuff and VLLM for the few models that I need to test that don't run on Ollama. Open Web UI as my primary interface. I just moved to Devstral for coding using the Continue plugin in VSCode. I use Qwen 3 32b for creative stuff and Flux Dev for images. Gemma 3 27b for most everything else (slightly less smart than Qwen, but its faster). Mixed Bread for embeddings (though apparently NV-Embed-v2 is better?). Pydantic as my main utility library. This is all for personal stuff. My stack at work is completely different and driven more by our Legal teams than technical decisions.

fazlerocks · 1d ago

Running Llama 3.1 70B on 2x4090s with vLLM. Memory is a pain but works decent for most stuff.

Tbh for coding I just use the smaller ones like CodeQwen 7B. way faster and good enough for autocomplete. Only fire up the big model when I actually need it to think.

The annoying part is keeping everything updated, new model drops every week and half don't work with whatever you're already running.

runjake · 1d ago

Ollama + M3 Max 36GB Mac. Usually with Python + SQLite3.

The models vary depending on the task. DeepSeek distilled has been a favorite for the past several months.

I use various smaller (~3B) models for simpler tasks.

v5v3 · 19h ago

Ollama on a M1 MacBook pro but will be moving to a Nvidia GPU setup.

gabriel_dev · 1d ago

Ollama + mac mini 24gb (inference)

xyc · 1d ago

recurse.chat + M2 max Mac

My AI skeptic friends are all nuts (fly.io)

Self-hosting your own media considered harmful according to YouTube (jeffgeerling.com)

Photos taken inside musical instruments (dpreview.com)

OpenAI slams court order to save all ChatGPT logs, including deleted chats (arstechnica.com)

FFmpeg merges WebRTC support (git.ffmpeg.org)

Cloudlflare builds OAuth with Claude and publishes all the prompts (github.com)

Root shell on a credit card terminal (stefan-gloor.ch)

If you are useful, it doesn't mean you are valued (betterthanrandom.substack.com)

Square Theory (aaronson.org)

The Who Cares Era (dansinker.com)

IRS Direct File on GitHub (chrisgiven.com)

WeatherStar 4000+: Weather Channel Simulator (weatherstar.netbymatt.com)

The ‘white-collar bloodbath’ is all part of the AI hype machine (cnn.com)

Quarkdown: A modern Markdown-based typesetting system (github.com)

A proposal to restrict sites from accessing a users’ local network (github.com)

How to post when no one is reading (jeetmehta.com)

Human coders are still better than LLMs (antirez.com)

Merlin Bird ID (merlin.allaboutbirds.org)

Why I wrote the BEAM book (happihacking.com)

Cursor 1.0 (cursor.com)

Show HN: I rewrote my Mac Electron app in Rust (desktopdocs.com)

Deep learning gets the glory, deep fact checking gets ignored (rachel.fast.ai)

US Trade Court finds Trump tariffs illegal (bloomberg.com)

Progressive JSON (overreacted.io)

My website is ugly because I made it (goodinternetmagazine.com)

The impossible predicament of the death newts (crookedtimber.org)

Precision Clock Mk IV (mitxela.com)

The time bomb in the tax code that's fueling mass tech layoffs (qz.com)

FLUX.1 Kontext (bfl.ai)

Show HN: My LLM CLI tool can run tools now, from Python code or plugins (simonwillison.net)

Covert web-to-app tracking via localhost on Android (localmess.github.io)

The Visual World of 'Samurai Jack' (animationobsessive.substack.com)

Tesla seeks to guard crash data from public disclosure (reuters.com)

Show HN: Kan.bn – An open-source alterative to Trello (github.com)

EU Commission refuses to disclose authors behind its mass surveillance proposal (old.reddit.com)

Japan Post launches 'digital address' system (japantimes.co.jp)

As a developer, my most important tools are a pen and a notebook (hamatti.org)

Show HN: Air Lab – A portable and open air quality measuring device (networkedartifacts.com)

Show HN: I wrote a modern Command Line Handbook (commandline.stribny.name)

Deepseek R1-0528 (huggingface.co)

Cinematography of “Andor” (pushing-pixels.org)

Prompt engineering playbook for programmers (addyo.substack.com)

Meta: Shut down your invasive AI Discover feed (mozillafoundation.org)

The radix 2^51 trick (2017) (chosenplaintext.ca)

The Right to Repair Is Law in Washington State (eff.org)

Gurus of 90s Web Design: Zeldman, Siegel, Nielsen (cybercultural.com)

Figma Slides Is a Beautiful Disaster (allenpike.com)

Surprisingly fast AI-generated kernels we didn't mean to publish yet (crfm.stanford.edu)

Google restricts Android sideloading (puri.sm)

(On | No) Syntactic Support for Error Handling (go.dev)

Ask HN: What Does Your Self-Hosted LLM Stack Look Like in 2025?

Comments (6)