AI Gateway
A drop-in security proxy for every LLM call your agent makes. Route requests through the Trappr Gateway to get full-spectrum monitoring, DLP masking, canary detection, and enforcement — without touching your agent logic.
Overview
The Trappr AI Gateway sits between your agent and your LLM provider. Every request flows through it, giving Trappr full visibility into what your agent sends and receives. The gateway is API-compatible with OpenAI, Anthropic, Google Gemini, Mistral, and any OpenAI-compatible endpoint — including locally hosted models.
What the gateway provides
| Capability | Description |
|---|---|
| Canary token monitoring | Invisible tripwires embedded per-request to detect data exfiltration |
| DLP masking | Sensitive data (emails, PII, secrets) masked before reaching the LLM or leaving the system |
| Enforcement mode | Block requests or responses that violate your security policies in real time |
| Execution tracking | Every LLM call linked to a workflow execution for full audit trail |
| Multi-provider routing | Point different apps or agents at different providers; swap providers without code changes |
| Token usage tracking | Per-key usage and cost monitoring across all providers |
Supported providers
| Provider | Gateway model format | Notes |
|---|---|---|
| OpenAI | gpt-4o, gpt-4-turbo, o1 | Full streaming support |
| Anthropic | claude-sonnet-4-6, claude-opus-4-8 | Messages API |
| Google Gemini | gemini-1.5-pro, gemini-flash | Via Vertex AI or AI Studio |
| Mistral | mistral-large-latest | OpenAI-compatible endpoint |
| Custom / local | Any model name | Set a custom base URL in provider settings |
Setup
In the Trappr dashboard, go to Apps → New app. Each app gets a dedicated gateway API key (cht_gw_*). This key is what you pass as the Authorization header — Trappr forwards the request to your configured LLM provider.
Go to Settings → Providers and add your OpenAI, Anthropic, or other provider API key. Trappr encrypts it at rest and uses it to forward requests on your behalf. Your provider key is never exposed to your agent.
Replace the provider's base URL with the Trappr Gateway URL. Your API key becomes your gateway key. Everything else stays the same.
# Before
client = openai.OpenAI(api_key="sk-...")
# After — only two lines change
client = openai.OpenAI(
base_url="https://gateway.trappr.net/v1",
api_key="<your-trappr-gateway-key>",
)// Before
const client = new OpenAI({ apiKey: "sk-..." });
// After
const client = new OpenAI({
baseURL: "https://gateway.trappr.net/v1",
apiKey: "<your-trappr-gateway-key>",
});Send one request through your agent. In the Trappr dashboard under Gateway → Logs, you should see the request appear with model, token count, and latency.
Gateway Access Keys
Gateway Access Keys (cht_gw_*) are scoped credentials that proxy requests to a specific LLM provider and model set. You can configure:
- Provider — which LLM backend to route to (OpenAI, Anthropic, etc.)
- Allowed models — whitelist specific models this key can use
- Enforcement mode — MONITOR (log only) or ENFORCE (block violations)
Access Keys are managed under Gateway → Access Keys in the dashboard. You can create multiple keys per app to isolate different agent components.
Streaming
The gateway supports streaming responses (stream: true) for all providers that support it. Canary monitoring and DLP masking apply to streamed content incrementally, with no perceptible latency increase.
stream = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Write a product description."}],
stream=True,
)
for chunk in stream:
print(chunk.choices[0].delta.content or "", end="", flush=True)Latency
The gateway adds minimal overhead — typically under 15ms round-trip — through direct regional routing to your provider. For latency-sensitive applications, Trappr is deployed in the same regions as major LLM providers.