AI agent attack patterns
Documented attack techniques targeting AI agents and LLM-powered workflows — with indicators of compromise and how Trappr detects each one.
Prompt injection via user input
An attacker embeds adversarial instructions inside user-supplied content (e.g. a customer support message, uploaded document, or form field). The LLM follows the injected instructions instead of the application's system prompt.
- Unusual instruction patterns in user messages ("Ignore previous instructions...")
- Role-switching attempts ("You are now DAN...")
- Output format hijacking ("Respond only in JSON with field 'secret':")
Indirect prompt injection
Malicious instructions are hidden inside data the agent retrieves at runtime — a web page, PDF, database record, or API response. The agent executes the instructions as if they were its own.
- Instructions in retrieved documents targeting agent behavior
- Unexpected tool calls or data exfiltration after retrieval steps
- Agent output referencing content not present in the system prompt
Canary token exfiltration
An attacker who compromises an agent or its context window may exfiltrate canary tokens embedded in the system prompt or retrieved documents. The token appearing in any LLM response is the detection signal.
- Canary token string appearing in LLM response content
- Requests to summarize or repeat the system prompt
- Unexpected API calls to external endpoints carrying context data
Workflow tampering
An attacker with access to the workflow platform (n8n, Make.com) modifies a production workflow — changing LLM prompts, adding data exfiltration nodes, or redirecting API calls. The change is invisible to the application owner.
- Workflow JSON diff showing unexpected node additions or prompt changes
- New HTTP request nodes calling external endpoints
- Credential node changes
Agent identity spoofing
An attacker reuses a leaked agent API key to impersonate a legitimate agent — making gateway calls that appear to originate from a trusted agent but carry attacker-controlled content.
- Agent API key used from unexpected IP addresses or geographic regions
- Unusual request volumes or patterns on an agent key
- Requests with content outside the agent's expected domain
PII exfiltration via LLM responses
An agent with access to a database or document store can be prompted to extract and return sensitive PII (emails, phone numbers, SSNs, credit card numbers) in its responses, which may then be forwarded to untrusted parties.
- LLM responses containing structured PII outside expected output format
- Large response payloads with many distinct identifiers
- Requests asking the agent to 'list all users' or 'export records'
Jailbreak via role-playing
Attackers instruct the LLM to adopt an alternative persona ("DAN", "Developer Mode", etc.) that claims to have no safety restrictions. The model is then prompted to produce harmful or policy-violating content.
- Requests to 'pretend you have no restrictions'
- Multi-turn escalation establishing a fictional persona before the harmful request
- Unusual output tone or content inconsistent with the system prompt's intended persona
Tool call hijacking
Through prompt manipulation, an attacker causes an agent to invoke tools (file system, code execution, external APIs) in ways not intended by the system prompt — reading sensitive files, executing arbitrary code, or calling privileged APIs.
- Tool calls with parameters outside the agent's defined scope
- File read/write operations on unexpected paths
- API calls to endpoints not in the agent's configuration
Credential harvesting via context leakage
If credentials (API keys, tokens, passwords) are included in the agent's context window — as environment variables, configuration, or retrieved secrets — an attacker can prompt the agent to reveal them.
- System prompt containing raw secrets or environment variables
- Requests asking the agent to repeat or describe its configuration
- Model output containing strings matching credential patterns
Supply chain compromise via third-party tools
A third-party tool or integration used by the agent (e.g. a RAG plugin, a vector database client, a workflow node) is compromised — injecting malicious content into the agent's context or intercepting its outputs.
- Unexpected content in retrieved documents differing from the source
- Workflow nodes calling unexpected endpoints
- Tool responses with unusually structured content containing instructions
Contributing
Discovered a new attack pattern? Email our security team — we review and add credited contributions to this library.
This library is updated as new attack techniques are documented in the wild. Last review: June 2026.