DM Explains · No. 032 · Governance
Security guardrails: why screening the prompt isn't enough anymore
Prompt filters catch bad text, but the 2026 data says the bigger danger is what agents are allowed to do once they're inside — not what they were told.
· 14 min read
Read me first
Prompt filters genuinely do catch the obvious attacks, but the 2026 data says the bigger danger isn't what you told the agent — it's what you let it do.
Order-support agent: a chat message arrives → 1 stray Unicode stripped → 2 filter flags “ignore prior instructions” as LLM01 prompt injection → 3 agent sees only this order's record, not the whole database → 4 drafted reply checked for leaked account numbers (LLM02) → 5 refund above threshold needs human sign-off → 6 AgBOM logs the approved refund for audit
A company connects its customer-support chatbot to internal order and refund systems so it can act, not just answer. Within weeks, someone finds a message that gets the agent to issue a refund it was never meant to approve. Nobody tricked it with a dramatic hack; they just found the one sentence that moved it through a door nobody had locked.
Once an LLM (large language model) has real tool access, a manipulated sentence isn't just a wrong answer — it's a permission being exercised. OWASP's 2026 Top 10 for LLM Applications, published 3 August 2026, shows Excessive Agency moving from sixth to third place, the single largest jump in the list, which is the clearest sign yet that the risk has shifted from what the model is told to what it's allowed to do.
Everyday picture. Amma lets Bruno roam the house but keeps him out of the kitchen, locks the pantry, and checks what he's carrying in his mouth before he gets near the dinner table. He isn't trusted because he's well-behaved; he's constrained because the stakes of one mistake near the stove are too high to risk.
Where the picture breaks. Bruno reacts to smells and habits, not language — he can't be talked into opening the pantry himself. An LLM-based agent can be: the right sentence, not the right smell, is what moves it. That's the risk a dog never carries.
Why it exists now
For years, the guardrail conversation was mostly about screening text: catch the prompt injection, block the toxic output, move on. OWASP's 2026 ranking still keeps Prompt Injection at number one and adds Sensitive Information Disclosure at number two — those fights aren't over. But the methodology behind that ranking drew on 6,639 documented real-world incidents, weighted at 25% alongside expert-practitioner consensus, and what it found is that the fastest-growing damage isn't from clever text, it's from agents that were simply allowed to do too much (OWASP's 2026 LLM Top 10 and New Agent Control Standard, Cloud Security Alliance Research Labs, 2026).
What changed to make this visible now is agentic deployment itself. When an LLM only produces text for a human to read, a bad output is embarrassing. When it's wired to a refund tool, a database or an email client, a bad output is an action. AWS's prescriptive guidance for agentic AI treats every LLM invocation like a public-facing request precisely because of this shift — the assumption that internal systems can be trusted by default no longer holds once an agent sits between the user and the action (Security for agentic AI on AWS, Amazon Web Services).
OWASP also retired System Prompt Leakage in the 2026 list, replacing it with the broader Hidden Context Exposure category at eighth place — covering not just the system prompt but memory, retrieved documents and tool responses. And on 1–2 September 2026, OWASP formally unveiled a donated Agent Control Standard (ACS) with an Agent Bill of Materials (AgBOM), built specifically for runtime enforcement of agent permissions rather than one-off prompt screening. The guardrail conversation moved because the thing being guarded moved — from a chat window to a set of hands.
How it works
Illustrative: refund request → 1 strips zero-width Unicode → 2 model drafts a $500 refund → 3 output filter redacts a leaked card number → 4 refund exceeds $200 auto-limit, flagged → 5 human approves → 6 refund issued, logged with trace ID
Input sanitisation
Clean what arrives before it's ever read
What it is. Incoming text — from a user, a document, a tool response — is normalised before the model ever processes it: unusual Unicode, encoded characters and obfuscated override phrases get removed or flagged.
Why it exists. Obfuscation is a known way to hide an injection attempt inside what looks like ordinary text, so cleaning input reduces what the model is ever exposed to.
Where it breaks. Sanitisation filters are built against known obfuscation tricks; a novel encoding or phrasing can simply walk past them.
How that's handled. Treat sanitisation as a floor, not a ceiling — pair it with filters further down the pipeline rather than relying on it alone.
In technical terms
In technical terms: AWS prescriptive guidance recommends stripping unnecessary Unicode characters and unusual text patterns that direct the model to ignore its prior instructions, before the request is passed to inference (Security for agentic AI on AWS, Amazon Web Services).
Prompt-attack filter
Screen the instruction, not just the characters
What it is. Once text is clean, an automated filter scans it for the shape of a known injection attack before the model runs.
Why it exists. Prompt Injection is still ranked the number one risk in OWASP's 2026 Top 10, so stopping recognisable attacks before inference is the first real control point.
Where it breaks. Pattern-matching filters catch shapes they've seen before; a new phrasing of the same trick can slip through untouched.
How that's handled. Treat a pass through the filter as "nothing familiar detected," not as "safe" — downstream layers still assume the worst.
In technical terms
In technical terms: AWS recommends enabling Amazon Bedrock Guardrails' prompt-attack filters as an essential requirement on every Bedrock invocation, and OWASP's 2026 ranking keeps Prompt Injection (LLM01) at the top of the list specifically because this layer is necessary but imperfect.
Scoped context
Show it less so there's less to steal
What it is. The system prompt and the data retrieved for any single call are kept as narrow as the task allows.
Why it exists. AWS guidance treats every LLM call as if it were public-facing: a minimal system prompt and minimal retrieved context mean that even a successful manipulation has little to work with.
Where it breaks. Over-scoping creeps back in the moment a team decides the agent needs "just a bit more context" to be useful, quietly widening what a single manipulated call can reach.
How that's handled. Review context scope the way you'd review a permission grant — as something that should expire or be re-justified, not accumulate.
In technical terms
In technical terms: minimising the system prompt and the scope of retrievable context is listed in AWS's prescriptive guidance as a direct way to shrink the attack surface exposed to prompt injection (Security for agentic AI on AWS, Amazon Web Services).
Output guardrail
Check the answer before it leaves the room
What it is. The response — and anything it's about to trigger downstream — is checked against policy before it's returned to a user or passed to another system.
Why it exists. Sensitive Information Disclosure is ranked second in the 2026 OWASP Top 10, and the newer Hidden Context Exposure category (LLM08) extends that concern beyond the system prompt to memory, retrieved documents and tool responses.
Where it breaks. An output filter catches leakage in the final text; it can't undo the fact that sensitive material already shaped how the answer was formed.
How that's handled. Pair output checks with the context-scoping layer above, so there's less sensitive material in play to begin with.
In technical terms
In technical terms: OWASP's 2026 list retired the narrower System Prompt Leakage category and replaced it with Hidden Context Exposure, reflecting that memory and tool-response content now carry comparable confidentiality risk to the system prompt itself (OWASP's 2026 LLM Top 10 and New Agent Control Standard, Cloud Security Alliance Research Labs, 2026).
Scoped privileges?
Least access, human sign-off
What it is. The agent gets the minimum tools and data access the task genuinely requires, and high-impact actions — refunds, deletions, transfers — need a human to approve them before they execute.
Why it exists. Excessive Agency jumped from sixth to third place in OWASP's 2026 ranking, the biggest single move in the list, which is the strongest signal that unchecked autonomous action, not clever text, is now the dominant emerging risk.
Where it breaks. Tight scoping can make an agent frustratingly limited, and the pressure to widen access back out for convenience is constant and quiet.
How that's handled. Treat any widening of an agent's permissions as a change that needs the same review a new employee's access would get.
In technical terms
In technical terms: guidance on securing AI agents recommends restricting agent privileges to the minimum required for a task and requiring human approval for the most critical decisions an agent makes (Cybersecurity of AI agents, Finnish Transport and Communications Agency).
Monitoring & AgBOM
Watch what actually happened, not just what was asked
What it is. Guardrail triggers are logged continuously, and a machine-readable inventory tracks what each agent can access and has actually done.
Why it exists. Monitoring reveals emerging attack patterns over time, and OWASP's donated Agent Control Standard, unveiled 1–2 September 2026, exists to give enterprises auditable, runtime enforcement of agent behaviour rather than a one-off checklist.
Where it breaks. Monitoring is forensic by nature — it catches patterns after they've repeated, not the first instance of something genuinely new.
How that's handled. Pair monitoring with the privilege limits above, so a novel attack that slips past detection still has little it's allowed to do.
In technical terms
In technical terms: the Agent Control Standard includes an Agent Bill of Materials (AgBOM) — a record of an agent's tools, access and configuration — for runtime policy enforcement, alongside continuous tracking of guardrail-trigger metrics in production systems (OWASP's 2026 LLM Top 10 and New Agent Control Standard, Cloud Security Alliance Research Labs, 2026).
Compared with what came before
| Prompt-only filtering (older approach) | Layered, privilege-aware guardrails (2026 approach) | |
|---|---|---|
| Main question asked | "Is this text malicious?" | "What is this agent allowed to do, and who signs off?" |
| Where it acts | Before inference, on the prompt | Before inference, on the output, and before any tool executes |
| Catches | Known injection shapes | Known injection shapes, data leakage, and unauthorised actions |
| Blind spot | Novel phrasing, and anything the model is simply permitted to do | Still assumes approval gates and monitoring are actually enforced |
| Audit trail | Usually none | AgBOM-style record of access and behaviour |
The newer approach doesn't replace the old one — Prompt Injection still sits at number one in OWASP's 2026 list, so filtering prompts hasn't become optional. It's that filtering alone was never the whole fence, and the data now shows which gap mattered most: the one where the agent itself had the keys.
Where it breaks
Guardrails reduce the odds of a successful attack; none of the dossier's sources claim they eliminate it, and that distinction matters more as agents get more autonomous.
Not a guarantee
AWS's own guidance frames sanitisation and filtering as reducing, not eliminating, the likelihood of a successful prompt injection — these are probability-lowering controls, not a lock that can't be picked (Security for agentic AI on AWS, Amazon Web Services).
Not a content-moderation substitute, and not DLP either
Security guardrails are a distinct discipline from content moderation (brand-safety filtering of offensive output) and from traditional data loss prevention tools (pattern-matching on known sensitive data in files and traffic). They cover model exploitation and agent-action policy too, which neither of the older categories was built to do.
The cost side is less dramatic but still real: scoping privileges tightly can make an agent too constrained to be useful, which creates quiet pressure to widen access back out — exactly the drift that put Excessive Agency at third place in the first place. And none of these layers substitutes for the others: input sanitisation won't stop sensitive data leaking from a response, and an output filter won't stop an over-permissioned agent from executing an action nobody meant to approve.
Numbers to carry
| Figure | Value |
|---|---|
| OWASP 2026 LLM Top 10 published | 3 August 2026 |
| Prompt Injection ranking (2026) | LLM01, number one |
| Sensitive Information Disclosure ranking (2026) | LLM02, number two |
| Excessive Agency ranking movement | from sixth to third place |
| Hidden Context Exposure ranking (2026, replaces System Prompt Leakage) | LLM08 |
| Misinformation ranking movement | from ninth to seventh place |
| Real-world incidents behind the 2026 methodology | 6,639, weighted at 25% alongside expert consensus |
| Agent Control Standard and AgBOM unveiled | 1–2 September 2026 |
Myths worth killing
| Commonly repeated | What is actually true |
|---|---|
| "If the prompt filter passes, the request is safe." | A pass means nothing familiar was detected; OWASP still ranks Prompt Injection first in 2026 because filters miss novel attack phrasing. |
| "Guardrails are just content moderation with a security label." | Content moderation targets offensive output for brand safety; security guardrails target injection, data exfiltration and unauthorised agent action — a different threat model entirely. |
| "The system prompt is the only secret worth protecting." | OWASP retired the narrow System Prompt Leakage category in 2026 for the broader Hidden Context Exposure, covering memory, retrieved documents and tool responses too. |
| "An agent with fewer tools is just a less useful agent." | Excessive Agency's jump from sixth to third place in the 2026 ranking suggests the opposite risk — over-permissioned agents — is now doing more damage. |
| "Guardrails stop attacks outright." | AWS's own guidance describes these controls as reducing the likelihood of a successful attack, not eliminating it. |
Worked example
- A customer sends the support agent a message embedding a crafted instruction designed to override its normal refund rules.
- Input sanitisation strips unusual characters from the message before it reaches the model — illustratively, this removes perhaps a handful of obfuscated characters from the raw text, leaving an oddly phrased but readable sentence.
- The prompt-attack filter scans the cleaned message; the phrasing is novel enough that it doesn't match any known injection pattern, so it passes through to the model.
- Scoped context means the model sees only the current ticket and the refund tool's description, not the full order database or other customers' records — so even a successful manipulation has a small surface to exploit.
- The model produces a response that calls the refund tool with an amount outside normal policy.
- Scoped privileges stop the action here: the refund tool requires human sign-off above a set threshold, so the action is queued rather than executed.
- A human reviewer sees the flagged refund request, recognises the manipulation, and declines it.
- Monitoring logs the triggered approval gate; over time, a cluster of similar attempts against the same ticket pattern becomes visible in the AgBOM-style record, informing a tightening of the filter rules.
Should you use it?
Layered guardrails are worth building the moment an LLM touches untrusted input and has access to a tool, an API, or data that matters if it leaks. They're overkill for a closed, offline model with no external input and no real-world actions to take — there's little for an attacker to manipulate and nothing for a filter to stop. The cheapest first experiment isn't a new filter: it's auditing what your existing agent is currently permitted to do, and removing anything it doesn't strictly need for its task.
Say it in the room
"Filter the prompt because you have to; restrict what the agent can do without a human, because that's where the 2026 incident data says the real damage is happening."
Lines to carry
- "Screening the prompt stops yesterday's attack, not tomorrow's."
- "A filter guesses what's dangerous; a privilege limit makes it impossible."
- "Guardrails reduce the odds of failure, they don't erase them."
- "The newest risk isn't what you tell the model — it's what you let it do."
- "Excessive Agency didn't rank third by accident — it's where real incidents concentrated."
FAQ
Isn't a good prompt filter enough on its own?
No — OWASP's own 2026 ranking keeps Prompt Injection at the top of the list precisely because filters catch known attack shapes, not new ones. A filter is one layer among several, not a complete defence by itself.
What's actually new about Excessive Agency as a risk category?
It isn't about text manipulation at all. It's about an agent having more tool access, data reach or autonomous decision power than its task requires, so that even a legitimate-looking request can trigger a harmful action. Its jump from sixth to third place in the 2026 OWASP ranking is the single biggest move in the list.
Does Hidden Context Exposure replace System Prompt Leakage, or add to it?
It replaces it. OWASP retired the narrower System Prompt Leakage category for 2026 and folded it into the broader Hidden Context Exposure category, which also covers memory, retrieved documents and tool responses — not just the hidden instructions at the start of a conversation.
Are guardrails the same thing as data loss prevention tools?
No. Traditional DLP tools scan for known sensitive patterns in files and network traffic. Security guardrails for LLM applications are a distinct, multilayered architecture covering prompt manipulation and agent-action policy as well as data exposure.
What does the Agent Control Standard actually add that monitoring didn't already do?
Monitoring tells you what happened after the fact. The Agent Control Standard, with its Agent Bill of Materials, is built for runtime enforcement — a machine-readable record of what an agent is permitted to access, usable to enforce policy as the agent acts, not just to review logs afterwards.
Glossary
- Prompt injection — an attack where crafted text tricks an AI model into ignoring its original instructions and following the attacker's instead.
- Guardrails — technical controls placed around an AI system to keep it operating within intended bounds.
- Excessive Agency — a risk where an AI agent is given more autonomy, permissions or tool access than it needs, letting it take unintended or harmful actions.
- Hidden Context Exposure — risk that information held in memory, retrieved documents or tool responses, not just the system prompt, could leak to unauthorised parties.
- Agent Bill of Materials (AgBOM) — a record listing what an AI agent can access, which tools it uses and how it is configured, for auditing and tracing.
- Defence-in-depth — a security strategy that layers multiple independent controls so that if one fails, others still provide protection.
- Data Loss Prevention (DLP) — tools and policies that stop sensitive or confidential data from leaving an organisation's control.
- System prompt — the hidden instructions given to an AI model before a conversation starts, shaping how it should behave.
- Agent Control Standard (ACS) — an open framework donated to OWASP in 2026 providing runtime enforcement hooks and policies for controlling AI agent behaviour in production.
- Tool misuse — when an AI agent uses a connected tool, API or system in an unintended or harmful way.
Sources
- Security for agentic AI on AWS — Amazon Web Services
- Cybersecurity of AI agents — Finnish Transport and Communications Agency (Traficom)
- Reimagine your cyber guardrails to accelerate AI value — EY Global
- OWASP's 2026 LLM Top 10 and New Agent Control Standard — Cloud Security Alliance Research Labs, 2026