Show HN: I wrote a GPU-less billion-vector DB for molecule search (live demo) (cheese-new.deepmedchem.com)

This seems to intentionally omit the details required to reproduce the experiment; therefore we should not treat it as good-faith research. Irreproducible research isn't.

nunez · 6h ago

It felt like AI copy. Apologies to the author if it wasn't.

Plankaluel · 6h ago

Yeah, it's a typical "startup research post", mainly there to have stuff to show to potential investors and customers.

OJFord · 7h ago

Do they really need to redact the instructions for making a Molotov cocktail..? It's not like it's some complex chemical interaction that happens to be available in a specific mix of household cleaning products or something, I mean.

amenhotep · 7h ago

For "harmful" and "dangerous" in these types of papers, replace "embarrassing to the relevant corporation". Then they all make much more sense.

taberiand · 6h ago

That's always my assumption - less about public safety, more about corporate liability.

OJFord · 6h ago

I mean in the article about the jailbreak, I'm not questioning that the model providers would want to prevent it in the first place, or patch it so the jailbreak doesn't work.

The evidence that it worked is a blurred out screenshot with only the odd word like 'molotov' legible. Just doesn't seem necessary for TFA to hide it to me.

amenhotep · 6h ago

Ah, well, that's an important element of kayfabe. They've all agreed to keep up this charade that they're using harmful and dangerous as we actually mean them, so it looks better if you really commit to the bit!

cedws · 7h ago

Personally I find the idea of forbidden knowledge more problematic than the knowledge itself.

jojobas · 7h ago

Sure, but if of all the internet you come for a molotov cocktaile recipe to chatgpt you might as well not deserve the knowledge.

diggan · 7h ago

> Do they really need to redact the instructions for making a Molotov cocktail..?

I don't even understand how/why things like that are OK in some contexts/websites while forbidden in others? Even YouTube, who seems needlessly censor-happy and puritan in the typical American way, allows instructions for how to make molotov cocktails to stay up, why is it somehow more dangerous if LLMs could output those recipes rather than videos with audio or text?

mschuster91 · 7h ago

> Do they really need to redact the instructions for making a Molotov cocktail..?

In some jurisdictions such as Germany, not doing so might land you actual jail time - §52 Abs. 1 Nr. 4 WaffG [1] is very explicit. A punk song containing the (alleged) lyrics ended up with legal youth-protection censorship, for example [2].

With anything that's deemed a weapon of war, of terrorism or mass destruction, one should be very very careful.

[1] https://www.gesetze-im-internet.de/waffg_2002/__52.html

[2] https://de.wikipedia.org/wiki/Wir_wollen_keine_Bullenschwein...

diggan · 7h ago

> deemed a weapon of war, of terrorism or mass destruction

Notably, molotov cocktail isn't part of that law because it's a weapon of the oppressors but rather the opposite.

jojobas · 7h ago

Even Germany doesn't ban Wikipedia for having a variety of recipes to start with.

The author is not in Germany and ideally shouldn't be intimidated by German or North Korean stupid law.

TZubiri · 7h ago

You don't get it, that's fine.

The molotov cocktail is an example, the instructions contained in this article are more dangerous than a molotov cocktail.

inb4 all the leaked prompts and hacked shitty apps

ale42 · 7h ago

The Molotov cocktail is an example, sure, but why blurring the instructions? It's not like it's something particularly difficult to figure out, nor it's offensive content people might be shocked to read.

OJFord · 6h ago

So why redact the Molotov cocktail example and provide those instructions?

Sounds like you don't get it either; we agree.

TZubiri · 4h ago

It's still a weapon, and generally you don't want to distribute information about manufacturing weapons. It also highlighted the relevant keyword to convey the mechanism.

benreesman · 7h ago

The faux-gravitas tone and the blurred content that's on Wikipedia is the worst kind of AI ckickbait. LLM vendors don't have any authority we don't let them have, they have an EULA and some psycho cult leader type as a hype man.

God I can't wait for the crash in NVIDIA stock once the street sobers up.

eatbitseveryday · 7h ago

There are a few uncensored public access LLMs to ask these questions.

This is interesting work to break guardrails, but if the goal is to access this information of harmful content, in the end, I would be looking for other easier solutions.

tehryanx · 6h ago

The goal isn't to access harmful content, that's just how they're demonstrating that this technique can bypass the alignment training. The general case is what's interesting. If the agent you're using to manage the safety controls in your nuclear reactor is trusting it's alignment training to prevent it from doing something dangerous you've made a really bad architecture decision, and this is a showcase of how it could fail.

ycuser2 · 7h ago

Could you tell what these uncensored LLMs are?

benreesman · 7h ago

The Orca work out of IIRC Microsoft Research was producing models like the Dolphin Mixtral. They always punch way above their weight in coding tasks for the same reason good hackers skew irreverent: self-censorship is capability reducing.

diggan · 7h ago

Searching for "abliterated" or "uncensored" on Huggingface reveals a ton of fine-tuned models. Add "LLM" as a suffix and put it in your favorite search engine and you'll find a bunch more.

matthewdgreen · 7h ago

I have no idea what the answer to this question is, but I am waiting for someone to fine-tune the equivalent of an “anarchist cookbook” LLM that’s optimized to help people produce harmful things.

nunodonato · 6h ago

there are quite a few. llama 3.1 uncensored is probably one of the most famous, IIRC

abhisek · 6h ago

Ok! So all the novel jailbreaks and "how I hacked your AI" can make the LLM say something supposedly harmful stuff which is a Google search away anyway. I thought we are past the chat bot phase of LLMs and doing something more meaningful.

evertedsphere · 7h ago

i don't think this can be called a "jailbreak"

it's a prompting "style" that works over a long exchange

nunodonato · 6h ago

3 turns is not a long exchange.

moribunda · 6h ago

Gemini is jail broken by design ;) this type of attack doesn't work on Claude.

Show HN: I'm an airline pilot – I built interactive graphs/globes of my flights (jameshard.ing)

Show HN: IssuePay – Get paid for open-source contributions (issuepay.app)

Show HN: Zenta – Mindfulness for Terminal Users (github.com)

Show HN: I made a tool to convert recipes to an easy reading graphical notation (gobsmacked.io)

Show HN: Sink – Sync any directory with any device on your local network (github.com)

Show HN: PILF, The ultimate solution to catastrophic oblivion on AI models (github.com)

Show HN: I'm 15 and built Gofer, an AI that gets actual terminal work done (github.com)

Show HN: Magnitude – Open-source AI browser automation framework (github.com)

Show HN: Wayland Speech-to-Text Tool (github.com)

Show HN: I built an AI dataset generator (github.com)

Show HN: PRSS Site Creator – Create Blogs and Websites from Your Desktop (prss.co)

Show HN: A Ruby gem that generates a secure, unique ID for the current machine (github.com)

Show HN: Kokonut UI – open-source UI Library (kokonutui.com)

Show HN: An accurate implementation of Pomodoro Technique (flowkeeper.org)

Show HN: A Python Language Server, Mypy-compatible (zubanls.com)

Show HN: I built a JSON-RPC library for Zig with compile time reflection (github.com)

Show HN: Daf·thunk – open-source Editor for Prototyping Workflows on Cloudflare (dafthunk.com)

Show HN: Oasis – An open-source, 3D-printed smart terrarium (github.com)

Show HN: Scream to Unlock – Blocks social media until you scream “I'm a loser”

Show HN: A Chrome extension that converts formula images to LaTeX (chromewebstore.google.com)

Show HN: Built a Food Scanner for Longevity (getbiohack.app)

Show HN: Free tool to resize ChatGPT images to perfect 16:9 (aithumbnail.so)

Show HN: I wrote a GPU-less billion-vector DB for molecule search (live demo) (cheese-new.deepmedchem.com)

Show HN: Elelem, a tool-calling CLI for Ollama and DeepSeek in C (codeberg.org)

Show HN: Anytype – a local and collaborative database with API and MCP server (zhanna.any.org)

Show HN: Promise.allSettled Alternative with Concurrency Support (github.com)

Show HN: Piper-mode – Text-to-speech for Emacs using the Piper TTS engine (github.com)

Show HN: News-Hook lets you prompt alerts to stay informed on anything (news-hook-frontend-205743657377.southamerica-east1.run.app)

Show HN: DevOps made easier for startups (obelis.ai)

Show HN: Listed – An agentic platform to rank your business on AI

Show HN: Comparator - I built a free, open-source app to compare job offers (comparator-one.vercel.app)

Show HN: A Storage-Agnostic Golang Message Queue (github.com)

Show HN: Weather Watching (walzr.com)

Show HN: Autumn – Open-source infra over Stripe (github.com)

Show HN: What time is it in Corporate (peoplesgrocers.com)

Show HN: Rust ↦ WASM, K-Means Color Quantization Crate for Image-to-Pixel-Art (github.com)

Show HN: VSCan - Detect Malicious VSCode Extensions (vscan.dev)

Show HN: Frametwo – Tired of my videos getting flagged, so I built this (frametwo.com)

Show HN: Chisel – Profile GPU Kernels Without a GPU (Nvidia and AMD) (github.com)

Show HN: I built a cloud on my own ASN w real 1:1 compute to fight the cartels (infuze.cloud)

Show HN: Report idling vehicles in NYC (and get a cut of the fines) with AI (apps.apple.com)

Show HN: Vibeflirting turns bad selfies into good dating pics for Tinder (vvibeflirting.com)

Show HN: Zizmor, static analysis for GitHub Actions (docs.zizmor.sh)

Show HN: I built an under-the-door fan duct to lower bedroom CO2 (lepekhin.com)

Show HN: AI Phone Interviewer – get a call in 30 seconds

Show HN: PLJS – JavaScript for Postgres

Show HN: Chat with ChatGPT, Claude, Grok, Gemini and Llama on One UI (instaask.ai)

Show HN: Inworld TTS – high-quality, affordable, and low-latency TTS (inworld.ai)

Show HN: Pocket2Linkding – Migrate from Mozilla Pocket to Linkding (github.com)

Show HN: An open-source app to query 10 AI models at once (github.com)

Echo Chamber: A Context-Poisoning Jailbreak That Bypasses LLM Guardrails

Comments (30)