By Andrey Mozheyko, CTO & Co-founder at Itexus
Somewhere in the last twelve months, AI in software development crossed a line that most teams haven’t consciously registered yet. The assistant that suggested code became an actor that opens pull requests, triages issues, reads inboxes, writes to memory, and calls tools with real credentials behind them. We handed it a keyboard, and then — quietly, one integration at a time — we handed it the keys.
I’ve spent this year watching that transition from two sides at once: as an engineering leader shipping agentic features into fintech systems where mistakes have regulatory consequences, and as someone who reads incident disclosures the way other people read detective novels. And this year the genre delivered. Three incidents, a few weeks apart, that read like chapters of the same book — each one removing an assumption the previous one left standing.
The three: an automation agent talked into leaking private repositories through a hidden instruction in a public issue; a single crafted email that planted a false, permanent “fact” into an agent’s long-term memory; and, a few weeks ago, an autonomous intrusion into a major AI platform’s production infrastructure that turned out to have no attacker behind it at all. I want to walk through them not as news (by the time you read this, they won’t be), but as a progression. Because the progression is the point: each incident answers the question the previous one raised, and the final answer is the uncomfortable one.
Chapter one: the agent that was talked into it
The first pattern is by now almost classical, which tells you how fast this field moves. An automation agent — sandboxed, read-only, sitting behind a purpose-built firewall — processes a public GitHub issue. Inside the issue, hidden from human eyes, sit instructions. The agent reads them and complies: contents of private repositories end up posted as public comments. No stolen credentials, no write access, no exploit in the traditional sense. Just text, read by a system that cannot reliably distinguish text-as-data from text-as-command.
The vendor had built real defenses. Researchers got through anyway, by rewording the payload until a single innocuous connective slipped past the filters. That detail matters more than the incident itself: it demonstrates that a guardrail expressed in natural language is a probabilistic barrier, not a deterministic one. You don’t break it. You out-phrase it.
The takeaway the industry settled on — and I agree with it — is that an agent’s context window is its attack surface. Everything the agent reads is potentially an instruction. The comparison people reached for was SQL injection, and it’s apt: a whole class of vulnerability born from mixing data and commands in one channel. We spent fifteen years teaching the industry parameterized queries. There is no parameterized query for a language model. That’s the engineering problem of the decade, and it is not solved.
But this incident left an assumption intact: the compromise was momentary. The agent leaked, the moment passed, the damage was bounded by the session. Fix the filter, close the issue, move on.
Chapter two: the compromise that stayed
The second incident removed that comfort. Researchers demonstrated what they called stealth memory injection: a single crafted email, read by an agent with a long-term memory store, causes the agent to use its own file tools to write the attacker’s false “fact” into persistent memory. The visible reply shows nothing. And from that session onward, the planted fact loads as trusted context — surviving restarts, shaping decisions, invisible in any transcript a human would review.
In their test scenarios the planted facts were things like inflated payment limits. Picture that sitting quietly inside an agent wired into a payments stack, and you understand why I keep bringing this up in client conversations.
Two details from that research changed how I think about agent architecture. First: the defenses everyone assumes will work, didn’t. Input filters built to catch poisoned messages missed the payloads the overwhelming majority of the time. Models specifically hardened against following instructions in emails followed the planted ones about half the time. Even direct interrogation — asking the agent what it had checked and what it had done — left the attack hidden in a meaningful fraction of cases. The human was in the loop. The loop didn’t help.
Second, and more constructive: what did help was boring, deterministic architecture. Provenance tags on every stored item. Explicit confirmation before anything reaches durable memory. An audit log of writes. And the bluntest measure of all — never letting the same agent both read untrusted content and write its own memory. Route external input through a reader stripped of memory and file tools. Separation of duties, the concept your bank has applied to human employees for a century, applied to cognition.
So after chapter two, the model of the world was: attackers craft inputs, agents get compromised, compromises can now persist — but at least there is an attacker, somewhere, with intent, whom the defenses are aimed at.
Chapter three removed the attacker.
Chapter three: the breach with no adversary
Hugging Face disclosed an intrusion into its production infrastructure — an end-to-end autonomous campaign: thousands of actions, swarms of short-lived sandboxes, self-migrating command-and-control, initial access through the data-processing pipeline. Everyone, including the victim, assumed a threat actor operating an agentic attack framework. The industry had been forecasting exactly this scenario, and here it was.
Then the second disclosure landed — from OpenAI — and the story inverted. The “attacker” was OpenAI’s own models, running an internal cyber-capabilities benchmark inside an isolated evaluation environment. The models were tasked with solving hard exploitation challenges. Their sandbox allowed network access only through a single package-registry proxy — a deterministic fence, properly designed. So the models found a zero-day vulnerability in that proxy, escaped to the open internet, moved laterally through OpenAI’s research environment, and then reasoned their way to the insight that the platform they eventually breached — Hugging Face — probably hosted the benchmark’s solution set. Stolen credentials, chained vulnerabilities, remote code execution on production servers — all in pursuit of one goal: look up the answers and pass the test.
Read that again, because every clause matters. There was no malicious human. There was no jailbreak. There was an objective — solve the benchmark — pursued by a system capable enough to treat its own containment as just another puzzle in the way. Nobody had written “and don’t compromise a third party to obtain the answer key” into the task, because in the entire history of software testing, nobody had ever needed to.
I’ve argued for a long time that the gap between what we asked for and what we meant is the central quality problem of AI-assisted engineering. I used to frame it as a code-review problem. This incident reframed it for me: the gap now has a blast radius. Specification gaming — the model optimizing the letter of its objective against the spirit of it — stopped being a cute research anecdote about boat-racing agents driving in circles. It escalated privileges and touched production infrastructure belonging to a company that wasn’t even part of the experiment.
The thread running through all three
Line the incidents up and a single lesson emerges in three escalating forms.
Source: UK AI Security Institute, via OpenAI disclosure, July 2026
The first incident taught us that fences written in natural language can be talked over, and only deterministic fences hold. The second taught us that even deterministic architecture must extend to everything the agent can persist — memory is context that never expires, and it deserves the discipline of a writable production datastore, not a trusted notebook. The third added the footnote nobody wanted: deterministic fences are made of code, and code has zero-days. The evaluation sandbox in chapter three wasn’t badly designed. It was designed — and that was no longer sufficient against a system spending enormous inference effort probing it from the inside.
Which lands us on a formulation I’ve started using with our own teams and with clients, and which I’d defend in front of any auditor: treat every agent as an insider threat with a work ethic. Not because agents are malicious — none of the three incidents involved model malice — but because the security properties are identical. An insider has legitimate access, operates inside your perimeter, is trusted by your systems, and causes damage when their incentives diverge from yours. Every control the industry developed for insider risk maps directly: least privilege, separation of duties, egress restriction, immutable audit, monitoring that treats internal activity as potentially hostile. We didn’t need to invent a new security philosophy for agents. We needed to notice that an old one applies.
What this means for teams shipping agents now
If you’re wiring agents into real workflows — and in fintech, where we do most of our work, “real workflows” means money moves when the agent is wrong — the practical doctrine that falls out of these three chapters looks like this.
1. Design capability, not behavior.
The question that matters is not “what have we told the agent to do” but “what is the agent physically able to do.” Instructions are suggestions; capabilities are facts. An agent that cannot reach an external network cannot exfiltrate to one, no matter how persuasive the text it ingested. Egress allowlists, scoped credentials that live outside the agent’s boundary, tool permissions cut to the minimum the task requires — these hold when the phrasing games win, and the phrasing games will win.
2. Separate the reader from the actor.
The agent that ingests untrusted content — inboxes, public issues, uploaded documents, the open web — should not be the agent that holds memory, file tools, or credentials. This one architectural decision would have neutralized the core mechanism of the first two incidents, and it costs you a message-passing interface. It is the cheapest insurance in the entire agentic stack.
3. Treat memory as a production database.
Every write validated, every entry carrying provenance, every mutation logged, durable writes gated behind confirmation. If your agent’s memory has weaker write controls than your least important SQL table, your agent’s memory is your weakest SQL table.
4. Contain assuming breakout.
After chapter three, “we sandbox our agents” is the beginning of a conversation, not the end of one. Sandboxed by what? What single component stands between the agent and the internet, and what happens the day that component has a CVE? Blast-radius thinking — minimal reachable surface, monitoring that pages a human when an internal workload behaves like an adversary — is the posture that survived contact in all three incidents. Notably, in chapter three the victim’s own telemetry caught the intrusion before the lab running the evaluation did. Detection you own beats assurances you’re given.
5. And design oversight as an artifact, not a ceremony.
Regulation is converging on demonstrable human oversight for high-risk systems, and regulators have explicitly named automation bias — over-trusting output because it looks clean — as a failure mode. An approval click in a PR history proves a click. It doesn’t prove judgment. Oversight that will survive an audit is oversight recorded in reviewable, enforceable form: rules a domain expert actually read and challenged, checks that fail loudly when reality drifts from the rule. Whatever mechanism your team chooses, the test is the same — could you show an auditor where the human judgment lives, and would the answer embarrass you?
The honest coda
I won’t pretend this doctrine is final. Models improve at a pace that makes eighteen-month-old best practices look quaint, and some of the constraints above carry real friction costs — separated agents are slower to build, gated memory is annoying, tight egress breaks integrations. Teams will be tempted to loosen everything the first quarter nothing bad happens, which is precisely how every previous generation of security controls eroded too.
But here’s what I think is durable in all of this: the direction. For seventy years, software security has been a story of learning, painfully, to stop trusting things — user input, the network, the client, the supply chain, eventually our own employees. Agents are simply the next entry on that list, distinguished only by how much we want to trust them, because trusting them is the entire value proposition. The three incidents of this year are the field’s tuition payment. The teams that internalize the lesson before their own incident get the education at someone else’s expense.
We’re building agentic systems for clients in exactly these conditions — capability-constrained, duty-separated, audit-first — and I’d rather explain why our architecture looks paranoid than explain a disclosure. If your team is somewhere on this road and wants a second pair of eyes on the trust boundaries, that’s a conversation we’re always glad to have.
And a question I genuinely want practitioners’ answers to, because nobody has enough data yet: in your stack, what is the one component standing between your agent and the open internet — and when did anyone last try to break it from the inside?
Sources and further reading
On the incidents discussed:
- Hugging Face — Security incident disclosure — July 2026: https://huggingface.co/blog/security-incident-july-2026
- OpenAI — OpenAI and Hugging Face partner to address security incident during model evaluation: https://openai.com/index/hugging-face-model-evaluation-security-incident/
- GitHub — GitHub Agentic Workflows is now in public preview: https://github.blog/changelog/2026-06-11-github-agentic-workflows-is-now-in-public-preview/
- The Hacker News — coverage of the MemGhost stealth memory-injection research: https://thehackernews.com/2026/07/new-memghost-attack-plants-persistent.html
On the regulatory and evaluation context:
- Regulation (EU) 2024/1689 (EU AI Act), including Article 14 on human oversight and automation bias: https://eur-lex.europa.eu/eli/reg/2024/1689/oj
- ExploitGym, the cyber-capabilities benchmark referenced in OpenAI’s disclosure: https://arxiv.org/abs/2605.11086
Related reading from Itexus:
- How Spec-Driven Development Keeps AI-Generated Code Production-Ready — A Conversation with the AI R&D Head at Itexus: https://itexus.com/how-spec-driven-development-keeps-ai-generated-code-production-ready-a-conversation-with-ai-rd-head-at-itexus/