This Twitter account is an influencer with 200,000+ followers, known for its hot takes.
Though the risk of GenAI is real, it looks to me there is a fair amount of chance that this story is staged and amplified for social media drama purposes.
pyman · 3h ago
This shows a lack of understanding of how software development and deployment actually work. First of all, you manage your production database using migration files. Secondly, you never let GenAI make deployment decisions. At most, it can read your system logs. GenAI doesn't reason, so it has no clue what dropping a production database really means.
joegibbs · 2h ago
When things go wrong with AI people seem to often make it prostrate itself, explain why it went wrong and promise to never do it again - which it does, but there’s no point, it’s not going to remember because it doesn’t have memory, and the reasoning as to why it went wrong is usually more hallucinated than regular conversation - like saying that it panicked (which it doesn’t do) or that it ran tests locally, which it can’t.
Perhaps getting something wrong puts in a state that makes it more likely to give wrong answers. It seems like GPT is the most likely to do this.
Also I don’t think you should be letting an LLM just make up commands and run them, that seems like a recipe for disaster, you should at the very least have to see what it’s going to do yourself.
akmarinov · 2h ago
Why let it do things on production? We don’t let people do whatever they want on production, why AI?
steinuil · 2h ago
Later the AI claims it can't run unit tests without overwriting the production database. This whole thread is hilarious.
People out there letting LLMs run whatever commands they want unsupervised on their databases.
And we wonder why so much software is so crappy.
Maybe misaligned AI is exactly what we need to format the hard drives of all these people. Leaving us with a golden age of software, people actually cared enough to think about when creating.
lozenge · 3h ago
Wow the information is really scattered over so many tweets. So they were able to recover?
Accessing a production database should require using an MFA to access your production AWS account. Did they rely on AI to write all the deployment as well?
Do they even have a dev environment outside of their local machine?
lozenge · 2h ago
So apparently they don't know whether their code is on git or not
Somebody else identified they might have mentioned a code freeze in chat without adding it to a prompt.
Basically this is what it would look like if you take an IT manager who has never coded and told them the AI will enable them to be a software engineer now.
Though the risk of GenAI is real, it looks to me there is a fair amount of chance that this story is staged and amplified for social media drama purposes.
Perhaps getting something wrong puts in a state that makes it more likely to give wrong answers. It seems like GPT is the most likely to do this.
Also I don’t think you should be letting an LLM just make up commands and run them, that seems like a recipe for disaster, you should at the very least have to see what it’s going to do yourself.
https://x.com/jasonlk/status/1946641193644798118
And we wonder why so much software is so crappy.
Maybe misaligned AI is exactly what we need to format the hard drives of all these people. Leaving us with a golden age of software, people actually cared enough to think about when creating.
Accessing a production database should require using an MFA to access your production AWS account. Did they rely on AI to write all the deployment as well?
Do they even have a dev environment outside of their local machine?
https://x.com/jasonlk/status/1946594194052849795?t=EasxlfgpA...
Somebody else identified they might have mentioned a code freeze in chat without adding it to a prompt.
Basically this is what it would look like if you take an IT manager who has never coded and told them the AI will enable them to be a software engineer now.