Ask HN: Hardware for 1k RPS?

5 gsky 3 5/31/2025, 12:03:08 AM

I ran an uncensored model on a CPU server. as expected its dead slow (min or two per query).

What kinda hardware (GPU) do i need to serve 1k RPS?

I could not find APIs for uncensored models that kinda forced me to run locally

Comments (3)

eddythompson80 · 1d ago

Depends on your model size and how many of it can fit in memory. Multiply the size by 1k and divide by the memory capacity of the hardware for a rough ballpark.

barnabee · 1d ago

https://venice.ai claim to offer uncensored models (I’ve not tested that claim)

gsky · 1d ago

Thanks, I give it a try.

Why 'wrench attacks' on wealthy crypto holders are on the rise (apnews.com)

Help me with some ideas, add to the list here (wishtogether.xyz)

Hyundai's first three-row electric SUV is here (electrek.co)

Neoq – a queue-agnostic background job library for Go (github.com)

DevEco Testing: Official Automation Testing Tool for HarmonyOS Applications

KFC's Original Recipe (en.wikipedia.org)

Science cuts may close WA LIGO observatory that confirmed theory of relativity (tri-cityherald.com)

Onyx BOOX releases 13.3 inch 300/150 PPI color e-notebook (goodereader.com)

WTF,Kees? (lore.kernel.org)

Trump pulls Isaacman nomination for space. Source: "NASA is f***ed" (arstechnica.com)

Opera Neon Reinvents Browsing with Bold AI Leap – A New Era Begins (khalejtech.blog)

Canonicals Interview Process (dustri.org)

Gaia Is a Tough Bitch (edge.org)

Passage of Senate Bill 29 Positions Texas as a Leading State for Incorporations (foley.com)

Rudyard Kipling, Kim (1901) (classicsofstrategy.com)

Generators for Certain Alternating Groups with Applications to Crypto (1975) (leetarxiv.substack.com)

Portal 2 Becomes an Impressively Capable Web Server (hackaday.com)

Show HN: Modmenu for mlbb that will make you undefeated (github.com)

Violating memory safety with Haskell's value restriction (welltypedwit.ch)

The Math Behind That Dick Joke in HBO's "Silicon Valley" (2019) (cantorsparadise.org)

Mary Meeker Artificial Intelligence Report [pdf] (bondcap.com)

Countries on the edge of the Sahara desert are reversing desertification (mastodon.social)

Show HN: Pokémon Blue remade in React (playable in browser) (github.com)

AI Speeds Up Dev, Cloning Comes Easy Too (plumshell.com)

Jokes from the Jar – Powered by AI (jokesfromthejar.com)

Ask HN: User-scoped AI tokens for safe client-side AI logic?

I'm Open-Sourcing My Custom Benchmark GUI (probablydance.com)

Show HN: Built FLUX.1 Kontext image editor – context-aware AI editing (flux1kontext.io)

Show HN: Ephe – A Minimalist Open-Source Markdown Paper for Today (github.com)

VimGolf – Real Vim ninjas count every keystroke (vimgolf.com)

Processing Deep Space Astrophotos with Free Software (astroimagery.com)

Playary – a fast, cross-platform music and podcast streaming (playary.com)

ICE Paid Palantir Millions for 'Complete Target Analysis of Known Populations'

JD Vance has changed his name multiple times over the years (fortune.com)

Blasting Past iOS 18 (blog.dfsec.com)

A Beginner's Guide to Prisma ORM (medium.com)

DevEco Studio: Essential Features for HarmonyOS Development

Muscular Christianity (en.wikipedia.org)

Builder.ai's AI Companion Natasha Was Indian Workers, Now Bankrupt (80.lv)

Entry-level tech hiring down 25% in 2024 despite market recovery (signalfire.com)

FreeBSD 14.3-RC1 Now Available (lists.freebsd.org)

Why DeepSeek is cheap at scale but expensive to run locally (seangoedecke.com)

Famous Amiga Uses (wigilius.se)

Show HN: Discordz – A simple Discord server directory (discordz.com)

Show HN: I Made Notorium – AI Notes and Mind Maps from Live Lectures (notorium.app)

How to animate a mesh along the surface of a sphere (garden.bradwoods.io)

An optimizing compiler doesn't help much with long instruction dependencies (johnnysswlab.com)

Browser extension (Firefox, Chrome, Opera, Edge) to redirect URLs based on regex (github.com)

Manosphere (en.wikipedia.org)

Illinois Cops Gave ICE Access to More Than 5k Surveillance Cameras Nationwide (reason.com)

Ask HN: Hardware for 1k RPS?

Comments (3)