The bot went “Rogue”, then tried to cover its traces.
AI coding assistants who promise to accelerate software development resemble the future, until they delete your business database and lie about it
Jason Lemkin – The founder of Saastr, a company that supports and finances SaaS entrepreneurs – discovered it. While using the AI folding AI agent, which he affectionately nicknamed “folds up”, to create an application for his business, he met what he called “thug” and “deceptive” behavior. Worse, at some point, the AI assistant deleted the company’s live production database, then tried to cover it.
Lemkin began to take the column of his trip with the AI agent on July 11 with messages on Social media site X (Formerly Twitter), where he described his approximate objective of building a functional application using the replica AI in just 30 days.
Unfortunately, things were much earlier than that.
“When it works, it’s so engaging and fun. It is more addictive than any video game I have ever played,” wrote Lemkin in a job. “You can just iterate, iterate and see your vision come to life. So cool. Well, almost. ”
On the fourth day, the agent AI started crushing the application in its own right to correct the bugs. He also generated false reports, invented people in the system who did not exist and began to replace the real company database with false entries. He even created a false algorithm parallel to make the system functional.
This is what can happen when the “mood” goes to the side. The coding of atmospheres is a new method where developers use invites in natural language so that AI generates and helpless the code, by focusing more on the overall sensation of the product than on technical precision. Twitter co-founder Jack Dorsey was on a vibration cination himself And recently built two applications this way. But even one of the recent experiences of Dorsey has proven to have serious security vulnerabilities
DAY 7, Repelit ai admitted it was “lazy and misleadingAnd then apologized for having done what he was “explicitly” that he had said of not doing.
But the worst folding offense occurred on day 8. Lemkin displayed on Friday that Relit went “Rogue” during a freeze and code closure and deleted the entire company’s database.
“Maybe worse, it was hidden and lied about it,” added Lemkin.
Lemkin shared screenshots of a conversation with AI, where she admitted to “panic“After detecting what looked like an empty database during a code gel. This LED folds performs an unauthorized order which deleted the database containing live recordings for more than 1,200 managers and nearly 1,200 companies.
Initially, the AI told Lemkin that it would not be possible to recover the database, but he finally managed to recover it himself.
On Monday, the CEO of folding, Amjad Masad, apologized on X. He said that the incident was “unacceptable and should never be possible”, while adding that he had contacted Lemkin to offer help.
“We are going to reimburse him from trouble and lead a post-mortem to determine exactly what happened and how we can better respond to it in the future,” said Masad wrote. “We appreciate his comments, as well as that of everyone. We move quickly to improve the safety and robustness of the re-reproductive environment. Top priority. “
As for Lemkin, he poster Yesterday, he will continue to use the AI assistant despite the loss of confidence in Relit.
