High performance client for Baseten.co

Comments (1)

mich5632 · 11h ago

We wrote a rust py03 client for OpenAI embeddings compatible servers (openai.com, or infinity, TEI, vllm, sglang). Most server-side ML infrastructure auto-scales based on the workload. On embedding workloads, this is no longer the bottleneck and has shifted to the client. In Python, the client is blocked by the global interpreter lock. With the performance package, we release the gil during requests, so you have available resources to query your VectorDB again.

The State of React and the Community in 2025 (blog.isquaredsoftware.com)

Memoir: Lifelong Model Editing with Minimal Overwrite Informed Retention for LLM (arxiv.org)

DIYRE: DIY Audio Projects (diyrecordingequipment.com)

Can you hear a 51% duty cycle (youtube.com)

Institutional Books by Institutional Data Initiative (institutionaldatainitiative.org)

Green Tea Garbage Collector (github.com)

What happened to Air India 171 (youtube.com)

For All That Is Good About Humankind, Ban Smartphones (jacobin.com)

Elicitation (june.kim)

Tailscale Founder Talks Future IPO as Revenue Surges on AI Adoption (bloomberg.com)

On the Usability of Editable Software (flak.tedunangst.com)

Plotform – Product Hunt but for your book launches (plotform.cc)

$100 Hamburger (en.wikipedia.org)

Pip's Quake (blitter.net)

Exploring the Dangers of AI in Mental Health Care (hai.stanford.edu)

Review: 'Print the Legend' gives form to 3-D printer companies' history (2014) (latimes.com)

SIMD-friendly algorithms for substring searching (0x80.pl)

After 18 Years of Infertility, an AI Tool Let a Couple Conceive (today.com)

Building a WordPress MCP Server for Claude: Automating Blog Posts with AI (val.demar.in)

Chebfun: Open-source package for computing with functions to 15-digit accuracy (chebfun.org)

Let's Play some Glider 4.0 with John Calhoun [video] (youtube.com)

Show HN: I created an AI form builder and it's free (minform.io)

Experimental Spacetime Distortion: Generating Gravitational Waves in the Lab (ej-eng.org)

UK unis to cough up to £10M on Java to keep Oracle off their backs (theregister.com)

The secret fast track for animal drugs (worksinprogress.co)

Comment on the Illusion of Thinking (arxiv.org)

Premium accounts to fund the matrix.org homeserver (matrix.org)

Anne Wojcicki to buy back 23andMe and its data for $305M (cnbc.com)

DHS is using CBP Home Mobile App to incentivize the voluntary self-deportation (dhs.gov)

Filedb: Disk Based Key-Value Store Inspired by Bitcask (github.com)

Venusian pancake dome likely formed due to elastic lithosphere and dense lava (phys.org)

Culinary Ocean that Separates the US and Europe: innards (1993) (nytimes.com)

Why do French men pee on the street [video] (bbc.com)

Baking the Y Combinator from scratch (again) (the-nerve-blog.ghost.io)

ArkFlow and Python: Easy Real-Time AI (arkflow-rs.com)

Regenerate Your Land (gramagrasslivestock.com)

Show HN: Hack to Save Any Videos from YouTube (github.com)

The Tech Job Meltdown (professoraxelrod.com)

Fastmigrate: Database Migrations for SQLite (github.com)

When Lee Miller Took a Bath in Hitler's Tub (newyorker.com)

Show HN: QuickGenAI – Turn Any Link into AI Lead Magnet (quickgen.ai)

Signed Git pushes for Linux kernel development server (2020) (people.kernel.org)

Solving Life's Annoyances with Constraint Programming [video] (youtube.com)

Propagation via lazy clause generation (2009) (link.springer.com)

MLB removes A's 'reverse boycott' game broadcasts from digital archive (sfgate.com)

There Aren't Enough Cables to Meet Growing Electricity Demand (bloomberg.com)

Show HN: Dual Elliptic Curve Math Backdoor in Python (leetarxiv.substack.com)

Rethinking Losses for Diffusion Bridge Samplers (arxiv.org)

The twom database format (fastmail.com)

The Case for Code‑Centric System Dynamics (SD) (teachyourselfsystems.com)

High performance client for Baseten.co

Comments (1)