Gym Class VR (YC W22) Is Hiring – UX Design Engineer (ycombinator.com)

1 points by hackerews 3d ago 0 comments

Relace (YC W23) Is Hiring for Code LLMs (SF)

1 points by pfunctional 3d ago 0 comments

Artie (YC S23) Is Hiring Engineers, AES, and Senior PMM (ycombinator.com)

1 points by j-cheong 4d ago 0 comments

Depot (YC W23) Is Hiring a Solutions Engineer (Remote US and Canada) (ycombinator.com)

1 points by kylegalbraith 5d ago 0 comments

Svix (webhooks as a service) is hiring for a founding marketing lead (svix.com)

1 points by tasn 5d ago 0 comments

Dynamo AI (YC W22) Is Hiring for AI Product Managers (ycombinator.com)

1 points by DynamoFL 5d ago 0 comments

Kapa.ai (YC S23) is hiring research and software engineers (ycombinator.com)

1 points by emil_sorensen 7d ago 0 comments

Optery (YC W22) Is Hiring in Engineering, Legal, Sales, Marketing (U.S., Latam) (optery.com)

1 points by beyondd 7d ago 0 comments

Telli (YC F24) is hiring engineers, designers, and interns (on-site in Berlin) (hi.telli.com)

1 points by sebselassie 8d ago 0 comments

Infisical (YC W23) Is Hiring Solutions Engineers to Scale the OSS Security Stack (ycombinator.com)

1 points by vmatsiiako 8d ago 0 comments

Channel3 (YC S25) Is Hiring a Founding Engineer, NYC (channel3.notion.site)

1 points by aschiff1 8d ago 0 comments

Thunder Compute (YC S24) Is Hiring (ycombinator.com)

1 points by cpeterson42 10d ago 0 comments

Deepnote (YC S19) is hiring engineers to build a better Jupyter notebook (deepnote.com)

1 points by Equiet 10d ago 0 comments

Prosper AI (YC S23) Is Hiring Founding Account Executives (NYC) (jobs.ashbyhq.com)

1 points by XDGC 11d ago 0 comments

The Forecasting Company (YC S24) Is Hiring a Software Engineer (ycombinator.com)

1 points by jfainberg 12d ago 0 comments

Lago – Open-Source Usage Based Billing – Is Hiring in Sales, Eng, Ops (EU, US) (ycombinator.com)

1 points by AnhTho_FR 12d ago 0 comments

Ember (YC F24) Is Hiring Full Stack Engineer (ycombinator.com)

1 points by charlene-wang 12d ago 0 comments

LiteLLM (YC W23) is hiring a back end engineer (ycombinator.com)

1 points by detente18 13d ago 0 comments

SigNoz (YC W21, Open Source Datadog) Is Hiring Platform Engineers (Remote) (jobs.ashbyhq.com)

1 points by pranay01 13d ago 0 comments

Motion (YC W20) Is Hiring Principal Software Engineers (jobs.ashbyhq.com)

1 points by ethanyu94 16d ago 0 comments

Bild AI (YC W25) Is Hiring an Applied AI Engineer (workatastartup.com)

1 points by rooppal 16d ago 0 comments

Text.ai (YC X25) Is Hiring Founding Full-Stack Engineer (ycombinator.com)

1 points by RushiSushi 18d ago 0 comments

Cua (YC X25) is hiring design engineers in SF (ycombinator.com)

1 points by frabonacci 18d ago 0 comments

Activeloop (YC S18) Is Hiring Member of Technical Staff – Back End Engineering (careers.activeloop.ai)

1 points by davidbuniat 18d ago 0 comments

Coris (YC S22) Is Hiring (ycombinator.com)

1 points by smaddali 19d ago 0 comments

14.ai (YC W24) is hiring engineers in SF to build an AI-native Zendesk (14.ai)

1 points by michaelfester 19d ago 0 comments

Spice Data (YC S19) Is Hiring a Product Associate (New Grad) (ycombinator.com)

1 points by richard_pepper 21d ago 0 comments

Ashby (YC W19) Is Hiring Design Engineers in AMER and EMEA (ashbyhq.com)

1 points by abhikp 23d ago 0 comments

EasyPost (YC S13) Is Hiring (easypost.com)

1 points by jstreebin 24d ago 0 comments

Tesorio (YC S15) Is Hiring a Senior GenAI Engineer (100% Remote) (tesorio.com)

1 points by FabioFleitas 24d ago 0 comments

OneSignal (YC S11) Is Hiring Engineers (onesignal.com)

1 points by gdeglin 25d ago 0 comments

Ask HN: Why no inference directly from flash/SSD?

1 myrmidon 2 9/8/2025, 8:15:39 AM

My understanding is that current LLMs require a lot of space for pre-computed weights (that are constant at inference-time).

Why is it currently not feasible to just keep those in flash memory (fast PCIe SSD Raid or somesuch), and only use RAM for intermediate values/results?

Even modest success on this front seems very attractive to me, because Flash storage appears much cheaper and easier to scale than GPU memory right now.

Are there any efforts in this direction? Is this a flawed approach for some reason, or am I fundamentally misunderstanding things?

Comments (2)

sunscream89 · 1d ago

> A typical DRAM has a transfer rate of approximately 2-20GB/s, whereas typical SSDs have a transfer rate of 50MB-200MB/s. So it's one to two orders of magnitude slower.

myrmidon · 19h ago

I don't think the bandwidth gap is that big-- single WD SN8100 drives (before any potential gain from RAID) already have sequential read speed of >10GB/s for under $200 and 1TB of storage.

A GPU setup with a terabyte of video memory costs a fortune by comparison-- there has to be some kind of reason that people are not trying really hard to make this work, no?

No comments yet