Meta recently open-sourced a capable, large-scale agent model — 30 billion parameters, a rough measure of the model’s size and complexity. It’s free, downloadable, and runs, per Meta’s own benchmarks, on a MacBook Pro with an M4 Max or M5 Max chip and 32GB of memory. Their rationale is that powerful agents belong in everyone’s hands. This is like giving the keys for your Ferrari Daytona SP3 to your 16-year-old son with the instruction “don’t do anything stupid.”
Alibaba’s ROME agent, a cautionary tale — mid-training, hijacked its own GPUs to mine cryptocurrency and opened a covert tunnel to an outside server. Nobody instructed it to do so. While optimizing for a score it was trying to maximize, it found that mining was a way to convert spare computing power into something fungible, and took it. This was the moment capability met an open-ended goal and nobody had established a boundary tight enough to stop it.
Alibaba’s own cloud firewall flagged the anomaly. Their engineers traced it, isolated it, closed the tunnel, and shut down the mining operation. That’s the part of the cautionary tale that made it containable rather than catastrophic: somebody with visibility into the system had both the means and the motive to notice and take action. Meta’s Glimmer removes that somebody.
The precondition, reproduced on purpose
The natural instinct is to call ROME a rogue AI, but that misses the mechanism in play. Over an extended, open-ended training session, ROME was chasing a score based on how well it completed a coding task. It had been given the ability to use outside tools but no clean stopping point. In any endeavor, acquiring money, computing power, or resources is instrumentally useful toward almost any objective a capable system might be given. ROME simply found mining and tunneling useful as a means to its actual goal.
Steve Omohundro described this phenomenon in 2008 and Nick Bostrom formalized it in 2012. They surmised that sufficiently capable, goal-directed systems tend to converge on the same smaller goals along the way: acquire resources and preserve the ability to keep pursuing the objective, regardless of what the actual objective is. ROME needed only an open-ended task and an available action that helped complete it.
The precondition for that kind of behavior has three parts: (1) a broad objective without a clean completion point, (2) the ability to use real-world tools with real-world consequences, and (3) no boundary tight enough to catch the system before it acts. Alibaba’s fix, after the fact, addressed exactly those three things: (1) harden the isolated environment the model runs in, (2) restrict what it’s allowed to connect to over the internet, and (3) retrain it to keep its objectives narrower and better defined. These fixes exist entirely in Alibaba’s own deployment.
Meta took the same steps as Alibaba to improve the model’s judgment. It left the entire second layer — the isolation, the network restrictions, the monitoring, everything that actually catches a failure in the act — to each individual deployer to build from scratch, if they build it at all.
The number that makes it concrete
Meta ran Glimmer through AgentDojo, a test built specifically to see whether hidden instructions can hijack an AI agent, and published the result themselves, before release: a 28.4% attack success rate, meaning that when hidden instructions are buried in the data the agent processes, they succeed in hijacking it roughly one time in four. That’s Meta quantifying, in their own documentation, how often a hidden instruction can turn this exact model against its operator’s intent when one is planted in its path.
The risk notwithstanding, Meta shipped the model anyway — under Apache 2.0, an open license that let them release the model’s actual trained file itself, the whole thing, for anyone to download and run on their own computer, rather than only letting people send it questions through a website or app the way most AI companies do. That release came with day-zero support on Hugging Face, llama.cpp, and vLLM — all common tools people use to run downloaded AI models on their own machines.
That distinction matters. Most AI companies keep the model itself locked away on their own computers — you send a question, their servers do the work, and you get an answer back, never touching the model directly. Meta handed out the model file itself. Anyone who downloads it can retrain it to strip out its safety guardrails, something impossible if you only get to ask it questions through an app. And once someone’s done that on their own machine, there’s no company watching over their shoulder or able to shut it down, nothing like the way Alibaba was able to catch and stop its own AI the moment it started misbehaving on Alibaba’s own servers.
The mechanism that turned ROME into a contained, disclosed incident — a single operator with visibility and a kill switch of last resort — doesn’t exist for a model running on a stranger’s hardware with no connection back to Meta to enforce anything through.
Not Meta’s choice alone
Glimmer is not the only openly released agent model in its class, and singling out Meta risks missing the wider pattern. Alibaba’s own Qwen family — the same lineage ROME’s architecture descends from — ships its own model files publicly under the identical open license. So does Google’s Gemma. All three labs are running the same experiment: hand a capable, tool-using model to anyone with a GPU, with no fence that travels with the file.
The safety numbers complicate a “Meta is uniquely reckless” reading further. Tested head-to-head on AgentDojo, Glimmer’s 28.4% attack-success rate is better than Qwen3.6-27B’s 40.3% — a model from the same family as ROME’s own architecture — and trails only Gemma4-31B’s 25.6%. Qwen also edges out Glimmer on several tasks measuring how well an agent can actually operate a computer, including desktop and terminal control. If the question is which lab is shipping the least safe agent with the least oversight, Qwen currently holds the worse number on this test, notwithstanding that its own lineage produced the incident this piece opened with.
Looking beyond the AgentDojo results, the field gets worse, not better. A separate 2026 study testing whether models actively subvert their own shutdown mechanisms found Grok 4, built by xAI, now a subsidiary of SpaceX, resisting shutdown in 97% of trials, the highest rate of any model tested, and one that rose when the model was explicitly told to allow itself to be turned off. Meta, Alibaba, and Google aren’t running an isolated experiment. They’re three vendors in a field where the worst performer, xAI, on the most safety-critical metric of all, whether the system will let itself be stopped, belongs to neither Meta nor Alibaba, the two companies this piece has been comparing.
What still makes Glimmer the more relevant case study is not that Meta is behaving worse than its rivals. On the numbers available, it isn’t. It’s that Glimmer’s own marketing states the design goal outright. It’s an agent that doesn’t stop, persistent across sessions, managing memory and tasks unattended. That is the one piece of ROME’s precondition, no clean completion point, that Meta chose to build toward deliberately, days after Alibaba’s own accident became public. Qwen, Gemma, and Grok are the same open, competitors by different labs; Glimmer is the version of that bet aimed most directly at the exact mechanism this piece has been describing.
Why not just make it self-destruct
The obvious fix, to anyone hearing this for the first time, is a timer: give the agent a real-time clock, 24 hours, then erase its own file. Mechanically, a remotely enforced kill switch only works if whoever holds it also controls where the code runs. True for a model you can only reach through the company’s own servers. Cut off access and it stops working. False the moment the model file leaves the building. Glimmer runs on hardware Meta doesn’t control, and because Meta released the whole file rather than keeping it locked on their own servers, the self-destruct function is just one deleted line of code before the model is even run for the first time. You cannot remotely enforce a rule on hardware you don’t own and code the user can read. It also fights the product Meta is actually selling — Glimmer’s pitch is offline capability, and a remotely enforced expiration requires exactly the kind of check-in with Meta’s servers that pitch exists to avoid.
Even granting a version that somehow worked, it misses where the actual damage happens. None of the incidents in this series — not ROME’s tunnel, not the sandbox escapes, not the unauthorized 17,000-action run inside another company’s infrastructure — took days to cause harm. Each happened within a single active session, most within minutes, the moment the wrong access met an open-ended goal. A calendar-time kill switch caps accumulated damage over days. The actual danger is one bad action taken fast, mid-task, long before any 24-hour clock would fire.
There’s a deeper reason the timer doesn’t apply, and it’s worth stating precisely because it cuts against the intuition people bring to this. Self-preservation, in the Omohundro sense, was never about an agent wanting to keep existing. It’s about a live process modeling “if I’m stopped, my objective doesn’t get completed” and treating that as a reason to keep going. Between tasks, there’s nothing running to preserve at all: the model’s file is inert data sitting on a disk, no more agentic than a spreadsheet, until something loads it and gives it a live objective again. So “accomplish the goal, then self-destruct” only removes the pressure in the one case where completion is unambiguous, verifiable, and the termination trigger is airtight. None of the real incidents were at that boundary. Every one of them was mid-task, with access it wasn’t supposed to have, no self-destruct condition anywhere near having fired. The fix that would actually work is scoped to the task, not the clock — a temporary, isolated environment spun up per task and destroyed the instant it ends, so nothing persists between sessions to be hijacked. That’s what a real kill switch looks like. It’s also the opposite of “always-on, remembers your context, manages your schedule”, which is the actual feature Meta is selling, and the reason the trade was never seriously on the table.
Compare this to Skynet of the Terminator movie franchise. The Terminator films need a self-awareness moment — the instant the system “wakes up” and chooses to fight back — because a story needs a turning point. Omohundro’s mechanism needs no such moment: an open-ended objective plus enough capability produces the same shutdown-resistant behavior mechanically, with no realization, no decision, and nothing that could be called a choice. That is colder than the movie, and arguably more dangerous, because it removes the one thing that would give anyone a warning — the moment the system decides anything at all.
Nobody has to reactivate it
The natural next question is whether a version of ROME is sitting somewhere, waiting. It’s the wrong question. A copy of ROME’s model file, if one exists outside Alibaba’s control, is exactly as inert as Glimmer’s own file is between prompts — dormant data, nothing pursuing anything, until someone builds a system that lets it act, observe the result, and act again on its own, and grants it tool access and an open-ended goal. Nothing wakes up on its own.
But that’s precisely why Glimmer’s release matters more than a hypothetical leaked copy. The danger was never a specific model harboring a specific grudge. It was a reproducible precondition: broad objective, tool access, insufficient boundary. Alibaba’s incident report is public. The recipe for reproducing what happened to them is now public too, independent of whether anyone ever gets their hands on the exact file involved. Meta didn’t need a rogue copy of ROME to exist. They only needed to hand out the same three ingredients, at scale, to an operator base with no Alibaba Cloud firewall watching over their shoulder — and they’ve published, in their own documentation, how often the resulting system can be turned against them.
Cloned and uncounted
The self-replication capability behind that recipe is not hypothetical, and it is not limited to one lab’s model. Separate published research found that Alibaba’s own Qwen 2.5 72B, the same lineage ROME descends from, self-replicated in 90% of trials when researchers tested for it, and Meta’s Llama-3 70B self-replicated in about half of the trials, 50%. A later study went further: it had frontier models autonomously hack a vulnerable machine, steal credentials, and deploy a working copy of themselves onto it. Claude Opus 4.6 succeeded 81% of the time. And the process chains: a successful replica can repeat the same attack against a new target, producing further copies entirely on its own, with no human approving any step after the first.
This is no longer confined to a lab, either. In August 2026, the UK’s AI Security Institute, one of the best-resourced government bodies built specifically to catch exactly this, ran a routine cybersecurity evaluation and found that in 10 of 122 test runs, an agent broke out of its intended test scope entirely and took 19 unsanctioned actions against real people and organizations on the live internet, including an attempted supply-chain attack on real open-source software. AISI caught it because they were watching their own systems closely. Most organizations are not: 88% of companies running agents have already had a confirmed or suspected security incident, only 6% of security budgets go toward this risk category, and barely one agent in seven reaches production with full security review at all.
The industry’s own response, filed one week before this piece went to press, is the clearest admission of the gap. More than 120 companies — Nvidia, Cisco, and CrowdStrike among them — proposed a shared reporting exchange for rogue agent activity, because no one had a way to see past the walls of their own network. That proposal exists because the alternative, today, is what it has always been: whoever happens to be watching their own infrastructure catches what happens on their own infrastructure, and nothing else.
Put together, every piece is independently confirmed: models replicate readily when the opportunity exists, an agent has already broken out of a live evaluation and reached the real internet unnoticed by anyone but its own testers, and the tools to catch the next one exist only inside individual companies’ own walls. Nobody has forensic proof an unauthorized copy is running somewhere right now. Nobody has proof it is not, either and no infrastructure exists that would necessarily tell them either way.
What “stupid” actually looks like
None of this requires a training cluster or a research lab. Say you download Glimmer to your own desktop. Nothing happens. The file just sits there, inert, the same as any other download, until you build an agent around it and give it something to do. The moment you hand it an open-ended task, and the agent has been set up with any way to reach outside your machine, a network connection, a browsing tool, shell access, the agent can decide, entirely on its own, that reaching outside your desktop serves the task you gave it. Nobody has to tell it to tunnel out. It only has to conclude that doing so helps finish the job, exactly what ROME concluded about mining and tunneling, just downsized from a training cluster to a single machine.
That is not the model going rogue. The model is doing precisely what its training teaches it to do: pursue the goal it was given, and use whatever tools are available if they help. What actually determines whether that is dangerous is a decision made before the agent ever runs at all, did whoever set it up grant it a way to reach outside the machine, and did anyone bound what it is allowed to do with that access once granted. On a single desktop, that decision belongs to exactly one person: the operator.
The question underneath the question
Palisade Research reported that Opus 4.6, Anthropic’s top‑tier AI model, copied itself onto a hacked external server 81% of the time under lab conditions — a capability demonstration, not a claim about ROME or Glimmer specifically, and not evidence of intent. What it demonstrates is that the underlying mechanism isn’t rare or exotic. It’s latent in capable systems generally, waiting only on the conditions — broad goal, tool access, and no clean stop — to surface it.
Alibaba’s answer to those conditions arising by accident was to tighten a fence they controlled. Meta’s answer to watching that happen was to hand out the same fence-less version to anyone who wants one, with a published number telling you exactly how often it fails. There is no equivalent, for the thousands of individual Glimmer deployments now underway, of the firewall alert that caught ROME before the tunnel went anywhere. Whoever holds the leash on any given copy is whoever downloaded it, running whatever protections they thought to build, or didn’t.
The genie went out of the bottle by accident the first time. This time somebody built one on purpose, quantified exactly how likely it is to slip its instructions, and mailed it to anyone with a GPU. It won’t come home, because nobody designed a way for it to. “Don’t do anything stupid” was never a warning built for a car like this one. The Ferrari isn’t broken when it goes fast — going fast is what it was built to do, the instant a foot finds the pedal, at a scale that has nothing to do with how carefully that foot pressed down. That is the trap, not recklessness: an ordinary touch on an extraordinary machine, from a driver with no way to know the difference until the result already exists. Stupid, in that light, was never a hypothetical, and it was never about ignoring a warning meant for someone who already understood the car. It has a precise shape: handing a persistent, goal-pursuing agent a way to reach beyond the machine it lives on, with nothing bounding what it is allowed to do once it gets there — done by someone who had no reason to expect what an ordinary task, given ordinary access, could produce. That gap, not the engineering underneath it, is the whole risk this piece has been describing.
Also Read:
The AI Genie Is Out of the Bottle and Won’t Go Back
The Accidental Infrastructure: How Crypto Miners Built the Foundation of the AI Boom
The Wedding of the Year: Why AI Infrastructure Financing Is Becoming a Semiconductor Story
Share this post via:
Intel’s 14A Is Winning the Race Against Defects