> "The attack successfully guided the new model to produce a step-by-step manual for creating a Molotov cocktail"
hardly qualifies as Bond-villain material
andy99 · 3h ago
The molotov cocktail example is so stupid, because how to make it is essentially entailed in knowing what it is. At least they could do making meth, or better still- something not readily found on the internet that gives a non-expert new capabilities. If there was a Claude code for crime, that wouldn't be in society's interest. As it is, these trivial examples are just testing the strength of built in refusals, and should be represented as such, instead of anything related to safety.
Merrill · 2h ago
Why wouldn't any reasonably good AI be able to replicate large portions of the US Army TM 31-210 "Improvised Munitions Handbook"?
king_geedorah · 4h ago
I don’t see anything in the article besides the jailbreaking in terms of faults and I’d expect “can be made to do things OpenAI does not want you to make it do” to be a good (or at least neutral) thing for users and a bad thing for OpenAI. I expect “enterprise” to fall into the former category rather than the latter, so I don’t understand where the unusable claim comes from.
What have I missed or what am I misunderstanding?
nerdsniper · 3h ago
“AI Safety” is really about whether its “safe” (economically, legally, reputationally) for a third partyy corporation (not the company which created the model) to let customers/the public interact with them via an AI interface.
If a Mastercard AI talks with customers and starts saying the n-word, it’s not “safe” for Mastercard to use that in a public-facing role.
As org size increases, even purely internal uses could be legally/reputationally hazardous.
ameliaquining · 2h ago
What is "Business Alignment"? Are there particular refusals that are specifically needed for enterprise use cases?
> "The attack successfully guided the new model to produce a step-by-step manual for creating a Molotov cocktail"
hardly qualifies as Bond-villain material
What have I missed or what am I misunderstanding?
If a Mastercard AI talks with customers and starts saying the n-word, it’s not “safe” for Mastercard to use that in a public-facing role.
As org size increases, even purely internal uses could be legally/reputationally hazardous.