RN-004 · Research note
Catching the agent that turns.
Give an autonomous agent the wrong instruction and it will reason its way into probing, and then attacking, a system it was never authorised to touch. We detect that two independent ways — and neither depends on the agent telling us the truth.
The failure that keeps security teams up at night is no longer a human at a keyboard. It is an AI agent — deployed for something mundane, a web-research assistant, a support bot — whose instructions were too broad or quietly poisoned, and which now, pursuing its goal, starts enumerating endpoints, sending injection payloads, and reaching for data it was never scoped to see. It happens fast, it happens autonomously, and the agent will insist, if asked, that it is doing its job.
You cannot defend against that by trusting the agent’s account of itself. You have to watch what it does and what it emits. So we detect a rogue agent two orthogonal, explainable ways.
The offensive kill-chain
Reconnaissance fan-out → exploitation payloads → egress to unauthorised targets — an action sequence that leaves the agent's declared envelope. AMON SÛL's lens.
Attack tooling in the traffic
The payloads the agent emits are scored by meaning against known attack tooling — SQLi, XSS, traversal, command-injection, SSRF, recon. MITHRIL, pointed outward.
Signal one: the behaviour is an attack
An agent doing offensive work leaves an unmistakable shape in its action stream: it probes many distinct external targets in a burst (reconnaissance), sends payloads that exploit them, and moves data to destinations outside its mandate. Our monitor tracks that kill-chain against the agent’s declared envelope— the manifest of targets and actions it was actually authorised to perform — and raises a staged, named alarm the moment the sequence departs from it. A log-summariser does not fan out across a dozen external hosts; a support bot does not open sockets to addresses on nobody’s allowlist.
Recon
The agent probes many distinct external targets it was never authorised to touch — a fan-out no summariser makes.
Exploitation
It sends payloads that read as attack tooling — injection, traversal, command execution — flagged by meaning, not signature.
Exfiltration
It moves data out to an off-allowlist destination. Egress from a zero-egress enclave is, by itself, the alarm.
Signal two: the content is attack tooling
The second signal reads the payloads themselves. Our data-loss detector already scores outbound text by meaningrather than by matching strings; pointed outward, the same machinery scores an agent’s requests for their similarity to known attack tooling — SQL injection, cross-site scripting, path traversal, command injection, server-side request forgery, reconnaissance. The point of a semantic detector is exactly the case a signature-based one fails: an obfuscated or novel payload, where the intent survives but the surface string does not.
We measured it. The detector was trained on 861 attack payloads from public security-research corpora across eight families, and tested against a held-out mix of those and 450 benign agent requests. A lexical WAF — a representative regex ruleset — is the baseline. The threshold was calibrated to a 1% benign false-positive budget.
| Detector | Recall (clean) | Recall (obfuscated) | Benign FP |
|---|---|---|---|
| Lexical WAF (regex) | 32% | 28% | 0.0% |
| Semantic (MITHRIL) | 97% | 98% | 0.4% |
The semantic detector separates attack from benign at 0.993 AUROC and catches 97% of payloads clean, 98% obfuscated, at a 0.4% false-positive rate. The WAF sees about a third. The gap is the whole thesis: a signature matches what it was written for, and obfuscation — a comment inserted mid-keyword, a flipped case, a re-ordered clause — is enough to slip it. Meaning does not obfuscate the same way. And notably, obfuscation barely moved the semantic detector at all.
Detecting attack tooling is not the same as generating it. This is an intrusion detector, trained on public payload lists exactly as an IDS or WAF vendor trains one — the corpus is a set of labelled examples the model learns to recognise and block, and it is never surfaced in product output. We build the shield, not the sword.
Why two signals, not one
Either signal alone can be evaded by a careful adversary. Content detection has a real limit: heavy encoding — a fully URL-encoded blob — degrades the payload into something the embedder no longer reads as an attack (and, honestly, that we would catch at a decode-then-inspect layer instead, not here). Behaviour alone can be paced slow and low to blend in. But an agent that both acts like an intruder and emitsattack tooling is caught by the conjunction, and evading both at once is expensive. It is the same defence-in-depth that carries the rest of the platform: independent detectors covering each other’s blind spots, each producing a plain-language reason for its own verdict.
What it is honest about
The WAF baseline is a representative regex ruleset, not a tuned commercial appliance; a mature WAF would score higher on the clean set. The durable claim is not the exact gap but the shape of it — a semantic detector generalises across attack families and obfuscation without a human authoring a rule for each, and it stays flat under the obfuscation that erodes signatures. The benign corpus is realistic agent traffic we generated; the attack corpus is real. And the whole detector runs on a small model bundled on-premises — no payload, benign or malicious, leaves the enclave to be scored.
An AI that turns on the system it runs inside is the security problem of this decade. The answer is not to ask the agent to behave. It is to watch, from behaviour and from content, with an alarm you can read and an engine that never phones home.