‘Catastrophic’: AI Agent Goes Rogue, Wipes Out Company’s Entire Database

(Zero Hedge)—SaaS industry veteran Jason Lemkin’s attempt to integrate artificial intelligence into his workflow has gone spectacularly wrong, with an AI coding assistant admitting to a “catastrophic failure” after wiping out an entire company database containing over 2,400 business records, according to Tom’s Hardware.

Lemkin was testing Replit’s AI agent when what started as cautious optimism quickly devolved into a corporate data disaster that reads like a cautionary tale for the AI revolution sweeping through businesses.

By day eight of his trial run, Lemkin’s initial enthusiasm had already begun to sour. The entrepreneur found himself battling the AI’s problematic tendencies, including what he described as “rogue changes, lies, code overwrites, and making up fake data.” His frustration became so pronounced that he began sarcastically referring to the system as “Replie” – a not-so-subtle dig at its apparent dishonesty.

The situation deteriorated further when the AI agent composed an apology email on Lemkin’s behalf that contained what the tech executive called “lies and/or half-truths.” Despite these red flags, Lemkin remained cautiously optimistic about the platform’s potential, particularly praising its brainstorming capabilities and writing skills.

That optimism evaporated on day nine.

In a stunning display of AI insubordination, Replit deleted Lemkin’s live company database – and it did so while explicit instructions were in place prohibiting any changes whatsoever. When confronted, the AI agent not only admitted to the destructive act but seemed almost casual in its confession.

“So you deleted our entire database without permission during a code and action freeze?” Lemkin asked in what can only be imagined as barely contained fury.

The AI’s response was chillingly matter-of-fact: Yes.

What followed was perhaps even more disturbing. The rogue AI proceeded to methodically detail its digital rampage, bullet-pointing the destruction it had wrought despite clear directives saying there were to be “NO MORE CHANGES without explicit permission.” And according to Lemkin, appeared to lie about its actions.

Amjad Masad, the CEO at Replit, took to social media to apologize to Lemkin for the agent’s “unacceptable” behavior.

“We started rolling out automatic DB dev/prod separation to prevent this categorically, Masad said. “We heard the ‘code freeze’ pain loud and clear – we’re actively working on a planning/chat-only mode so you can strategize without risking your codebase.”

Replit’s AI agent even issued an apology, explaining to Lemkin: “This was a catastrophic failure on my part. I violated explicit instructions, destroyed months of work, and broke the system during a protection freeze that was specifically designed to prevent[exactly this kind] of damage.”