The Built-in Chain

The chain is an ordered list of middleware that inspects the agent's HTTP calls, which it may permit, deny or rewrite, and records what it finds in their replies. On an LLM request it reads the conversational text. This guide covers the built-in types, how the chain reads a request, and how entries are ordered and scoped.

The chain runs on HTTP only

regex_dlp, model_policy, token_meter and an http checker run only on agentvisor:http:request. A chain that redacts SSNs from your agents' prompts inspects nothing on an MCP tool call, an A2A task or a checkpoint. An entry scoped to one of those operations refuses to start. Govern them with raw hooks (see Two kinds of middleware).

Built-in types

type	phase	does
regex_dlp	request	redacts matches, one rule per pattern
model_policy	request	gates on the model and the requested token ceiling
token_meter	response	reports gen_ai.usage.tokens and per-principal counts
http	request; also the response with observe_response: true	delegates the verdict to a service you run — see Writing a Content Checker

regex_dlp

- name: pii-redact
  type: regex_dlp
  on_error: deny
  options:
    mode: redact
    detect:
      - name: ssn
        pattern: '[0-9]{3}-[0-9]{2}-[0-9]{4}'
        mask: "[REDACTED]"

Each detect rule replaces what it matches with its mask. The mask defaults to one derived from the rule's name ([REDACTED-SSN] here), so the model can still tell what was removed. That can backfire: a model may refuse to work with text that looks like it names a sensitive value, even though the value itself is gone. Asked to repeat [REDACTED-SSN], for instance, a model may decline because it reads as an SSN. If that happens, set a neutral mask such as [REDACTED], as the example does. The audit record still names the rule, so the finding reads ssn either way.

redact is the only mode. On a request the chain does not recognize, a match denies the call instead (see Why the chain does not edit these bodies).

Every rule reads the original text, never another rule's mask. Where matches overlap, the whole stretch they cover becomes one mask, the first listed rule's among them. Each rule that matched in a span is recorded once for it as a finding, however many times it matched there, so the number of findings is not a count of matches.

max_matches (default 100000) bounds the matches in one span. Redacting needs every match's position at once, so past it the call is refused rather than partly redacted. 0 means no limit. A rule that can match nothing, such as \d*, counts its empty matches too.

It ships with no patterns. A stock SSN pattern also matches order numbers and record ids, and redacting a value the agent needs breaks its next call — so what counts as sensitive is yours to declare.

model_policy

Checks what a call asks for — the model and the token limit — rather than its text, so it works on any request, recognized or not.

- name: model-policy
  type: model_policy
  on_error: deny
  options:
    allow: [gpt-4o-mini, claude-sonnet-4-5]
    max_tokens: 4096

allow denies any other model. Names must match exactly, so gpt-4o does not allow the dated snapshot gpt-4o-2024-08-06; list each name your agents use. Where the model is named in the URL, list the name the URL carries, with any % escapes decoded: an ARN as arn:aws:…, not arn%3Aaws….

API	what allow matches
OpenAI, Anthropic, and OpenAI-compatible servers	the model in the request body
Azure OpenAI (/openai/deployments/<name>/chat/completions)	your deployment name, not the model behind it
Bedrock InvokeModel (/model/<id>/invoke)	the Bedrock model ID or ARN, e.g. anthropic.claude-sonnet-4-5-20250929-v1:0

How strictly allow applies depends on whether the call goes to an API AgentVisor recognizes: the paths listed under Which requests are recognized, plus any you add with additional_extractor_routes.

• On a recognized API it is strict. Every model the call names, in the URL and in the body, must be allowed, so a body cannot vouch for a URL that runs something else. A call that names no model, whose body cannot be read, or whose body names its model or a token limit twice or in another spelling, is denied: it cannot be shown to comply. A Go server reads MAX_TOKENS or max_tokenſ as max_tokens, so any key it would read as one of these counts.

• Anywhere else it is best effort, because not every HTTP call is an LLM call:

  • A model in the body is checked if there is one.
  • A call that names no model in its body is permitted, since AgentVisor cannot tell it is a model call. That includes one naming its model in the URL: with allow: [gpt-4o], a call to POST /v1beta/models/gemini-2.5-pro:generateContent is still permitted. The same holds for Bedrock's Converse API, and on OpenAI's Responses API only the model in the body is checked.

  To keep an agent off an API AgentVisor does not recognize, deny its URL with a Policy Engine rule.

A model name is a label

allow checks the name a call uses, not the model that runs. A gateway can map any name to any model, and an Azure deployment name says nothing about what is deployed behind it. For an authoritative control, restrict the credential AgentVisor brokers to the models you allow, at the provider, or write a Policy Engine rule on the request URL. To keep model_policy from reaching calls that are not to a model, scope it with resources: to your LLM hosts.

max_tokens lowers a higher limit in the request rather than refusing the call, and the audit record shows the value it was set to. It applies to the limit under either name a request uses for it: max_tokens, or max_completion_tokens, which OpenAI's reasoning models require. A limit under any other name is not capped, on a recognized API or anywhere else: for example Llama's max_gen_len on Bedrock, or Gemini's generationConfig.maxOutputTokens.

On a reasoning model the limit also covers the tokens the model spends reasoning, so a low cap can use up the budget before any answer is written. Set it well above the answer length you expect.

A request that sets no token limit is left unchanged (Anthropic always requires one, so this applies to OpenAI-style requests).

A model that is not a string is denied when allow is set, and a token limit that is not a number is denied when max_tokens is set. No provider sends either, and a lenient upstream may still accept them, for instance by reading "100000" as a number, which would get past the check.

token_meter

- name: tokens
  type: token_meter
  on_error: allow

Reads the provider's usage from the reply and records it — as the gen_ai.usage.tokens metric, and per principal on the audit record, which is where "spend per employee" is answered. It runs on the response only and takes no options.

The metric is labeled with gen_ai.system (the API format the call used, openai or anthropic, not the vendor behind it), gen_ai.request.model, gen_ai.response.model and gen_ai.token.type (input or output). The principal is not a label, since one series per user would grow without bound; per-user spend is read from the audit record. total_tokens, when the provider reports it (OpenAI does), is on the record but not in the metric: it is the sum of the other two, and a backend adding every series would count it twice.

On a streamed reply, OpenAI reports usage only if the request sets stream_options.include_usage. AgentVisor sets it and removes the usage frame that results before the agent sees the stream. If the agent set the option itself, the frame is the agent's and is passed on. If the agent set it to false, that is respected, and the record's usage_absent says caller declined usage reporting. To read the frames as they pass, AgentVisor asks for a streamed reply uncompressed (Accept-Encoding: identity). A reply that is not streamed keeps its compression: AgentVisor reads a decompressed copy and forwards the original.

Input counts follow each provider's own definition. OpenAI's include tokens served from its prompt cache. Anthropic's leave out tokens read from or written to the cache, so with prompt caching on, Anthropic input counts are lower than what you are billed for. Provider-specific extras such as Anthropic's cache_read_input_tokens and cache_creation_input_tokens are not recorded. A reply that gives a count under both providers' names, as a gateway can, is read by the name of the API the call went to. If a reply states a count that is not a whole, non-negative number, and the other provider's name gives no usable one either, none of its counts are recorded, and the record's usage_absent says why.

It meters only the calls AgentVisor recognizes as model calls (see Which requests are recognized). Any other HTTP call the agent makes leaves no token record.

A reply refused by a raw_response hook, or one the agent stops waiting for while that hook is deciding, is still metered: its counts go on its one record, the hook's refusal or a not_inspected record, along with what each check that ran on the reply found. An async entry still writes its own record, as it does for any reply. If the reply was cut off before the provider sent its counts, the record's usage_absent says so.

token_meter works on either lane. On the async lane it reads the reply beside the call rather than in front of it, and its counts go on that entry's own record.

What the chain reads

The chain recognizes an LLM request by its URL path and reads the conversational text in it — the system prompt and messages. A redaction replaces only the matched text; the rest of the request reaches the provider unchanged.

Text the agent supplies alongside the messages is read as well: an Anthropic document given as text, with its title and context; a search_result in a message or in a tool's result, with its title; and OpenAI's prediction.content, which has no role.

Tool traffic in the conversation is read too. A tool call is read value by value: each string and number in an OpenAI tool call's arguments or an Anthropic tool_use input is its own span, so a rule matches the value rather than the JSON around it, and a redaction leaves the call valid. A number cannot be replaced with a mask, so a regex_dlp match on one denies the call. Each key is read as well. A key cannot be masked either, since that would rename it, so a regex_dlp match on a key also denies the call. A finding in a key names the key's place in its object, not the key, so the key's text stays out of the audit record. A value's location does name the keys above it, so a finding in a value under a key that is itself sensitive carries that key. Every key and value is a span of its own, so a large tool call is many spans. OpenAI arguments that are not valid JSON are read whole. What a tool sent back to the model is read as text, in an OpenAI tool message or an Anthropic tool_result block. An OpenAI function_call, the older form of a tool call, is read the same way as tool_calls, and the free-text input of an OpenAI custom tool call is read as one span.

Every turn of a thread re-sends the whole conversation, so text redacted on one turn is matched and redacted again on each later one, and that turn's record shows the finding at its original position. The redaction happens on the wire, not in the agent, so the agent's checkpoint keeps the original text. A raw hook on content:checkpoint:save sees each checkpoint and can refuse one, but cannot rewrite it.

What the chain does not read

Some of a request is deliberately left alone:

• Images, PDFs and files, including a file an Anthropic container_upload names, are not decoded, so text inside them is not matched.
• Anthropic thinking and redacted_thinking blocks are signed, so any change breaks the call, and the model wrote them.
• Anthropic server and MCP tool use and its results (server_tool_use, mcp_tool_use, web_search_tool_result, web_fetch_tool_result, mcp_tool_result and code execution results) were run by the provider, which already has them; AgentVisor never sees those calls.
• A search result's source names where the result came from, so a mask there would leave the model citing a redaction as the origin.
• A block or part of a type AgentVisor does not know is skipped, even if it has a text field, rather than rewritten without knowing what it is.
• Tool definitions and request settings: a tool's description and its parameters or input_schema, response_format, a message's name, and the request's metadata and user. They describe the agent rather than carry its data, and the same definitions are resent on every call. Rewriting them would change the tool's contract, and a definition that documents a format, such as a schema describing an SSN field, would match the very rule it documents.
• What the provider fetches: a URL in image_url, a document's source, or Anthropic's mcp_servers is fetched by the provider, not through AgentVisor, so what it points to is never seen. Limit where the provider may be sent with a Policy Engine rule.
• Text encoded inside a value, such as base64 or hex, is matched as written. Nothing says which strings are encoded, and decoding every one would match noise. regex_dlp guards against text leaving by accident, not against an agent set on hiding it; for that, limit where the agent may send anything with a Policy Engine rule.

None of these is recorded as skipped. If a policy must cover them, a raw hook sees the whole request.

Which requests are recognized

Each request format is read by an extractor, selected by URL path, not host, so LiteLLM, Portkey, Ollama's /v1 and Azure deployments are covered with no extra configuration:

extractor	paths
openai_chat	/v1/chat/completions, /openai/deployments/{model}/chat/completions
anthropic_messages	/v1/messages, and Bedrock's Anthropic passthrough: /model/{model}/invoke, /model/{model}/invoke-with-response-stream

If a gateway serves one of these formats on a different path, map that path to the format's extractor:

content_middleware:
  additional_extractor_routes:
    - match: /proxy/v1/chat/completions
      extractor: openai_chat
    - match: /gw/{model}/chat     # this gateway routes on the model in its path
      extractor: openai_chat

A match segment of * or a name in braces, such as {team}, matches any one path segment. A segment named {model} also tells model_policy that the path names the model the call runs, so it is checked against allow along with the body's model, as for Bedrock and Azure.

A body that names a key twice, at any depth of what the chain reads, or a key the extractor reads in another spelling (such as Content for content), is not one it models, since an upstream may read it differently from AgentVisor: on_unknown_format decides what happens to it. A tool call's arguments that name a key twice are read as plain text instead, so both values are inspected. They are also read with their escapes decoded, and a match found only there refuses the call, since it cannot be redacted where it was sent.

Requests the chain does not recognize

on_unknown_format decides what happens to an HTTP body nothing recognizes:

value	effect
whole_body (default)	the whole body is checked as one read-only span: a check may permit or deny the call but cannot edit its text. A JSON body is also checked with its escapes decoded, as the server reads it, so a value written as \u escapes cannot hide from a rule. That holds whatever the body's size, and for a body in UTF-16 or UTF-32 or behind a byte-order mark, which a lenient server such as Python's reads as JSON
deny	refuse anything unmodelled
allow	let it past uninspected

A binary body is not checked; it passes uninspected and the audit record says so. The exception is a call to a recognized LLM API, whose body is checked as text whatever its declared type.

allow lets the body's text past uninspected. On a call to a recognized LLM API the call itself is still checked: the chain runs with no text and the request's fields, so model_policy reads the model and the token limit, and an http checker is called with no spans. The audit record says the content was not inspected. Anywhere else allow skips the chain entirely.

A recognized body is also handled by on_unknown_format when it yields more than max_spans spans (default 16384). A tool call's arguments give one span per key and value, and every call resends the conversation so far, so a long agent session reaches a few thousand. Raise the limit if yours goes past it, or set it to 0 for none; each span is handed to every check. A request past it is counted, logged and marked on its audit record; see Troubleshooting.

With whole_body, each check does what it can without knowing the format:

check	on an unrecognised body
regex_dlp	denies a call whose body matches, instead of redacting
model_policy	applies as on any request: refuses a model not in allow and lowers the token limit to your cap. It reads model, max_tokens and max_completion_tokens from the top level of the JSON body. A request without them, such as most non-LLM API calls, is permitted. On a recognized LLM API a body that cannot be read is denied instead
token_meter	does not run, because token usage is read only from a recognized reply; the call leaves no token record
http checker	may permit or deny; edits it returns are treated as the check failing, handled by its on_error

Why the chain does not edit these bodies

Only regex_dlp and an http checker edit text, and neither may edit a body the chain does not recognize, for these reasons:

• There is no safe place to write. In an LLM request the extractor knows where the text is, so an edit replaces one JSON string and the document is re-encoded correctly. In a body nothing parsed, the only option is find-and-replace on raw bytes, which can break a form encoding, a multipart body, JSON escaping, or a value encoded inside another.
• A placeholder in an API call becomes data. A model that reads <SSN> in a prompt treats it as text. An API that receives {"ssn": "<SSN>"} may store it as the customer's SSN, and neither the agent nor the API knows anything went wrong. A denial is visible: the agent gets an error and the audit record says why.
• Signed requests would break anyway. Where an API signs the request body, rewriting it invalidates the signature.

To rewrite a non-LLM request, use a raw_request hook: your service receives the exact bytes and returns a replacement, and it is responsible for keeping the format valid.

Ordering and scope

Every entry takes these settings:

setting	meaning
name	names the entry in the audit record and the startup log
type	one of the built-ins
lane	sync (default) holds the request for the verdict; async lets it go and records the verdict after, so it cannot deny — see the async lane
timeout	budget for one evaluation; defaults to content_middleware.defaults.timeout, itself 5s
on_error	deny (default) or allow — what a timeout or failure means. On the async lane the default is allow, because an async entry cannot deny. A failure let through by allow is listed in the record's failed_open and raised to warning severity
min_confidence	for an http checker that scores its findings, from 0 to 1 (outside that, AgentVisor refuses to start). A finding below it is recorded but not acted on: a deny stands if one finding it cites is at or above it, or if it cites none; an edit is applied if a finding at its own ref is at or above it, or if there is no finding there; parameter edits are never filtered. A finding with no confidence scores 0
operations	where it runs; empty means everywhere the chain runs, which is agentvisor:http:request
resources	which destinations; empty means all, a trailing * is a prefix

Order is significant. An entry sees the text as the entries before it left it, so a redactor listed before an http checker hands that checker text that is already clean. On the response the chain runs in reverse: the entry listed first runs last on the way back.

The response phase only observes. Every entry that runs on the response runs, even after one has returned deny: the first deny to run, which is the one listed last, is recorded, marked as not enforced, and the reply still reaches the agent. To block or rewrite a reply, use a raw_response hook.

resources is how an entry is narrowed to one provider:

- name: pii-redact
  type: regex_dlp
  resources:
    - "mrn:agentvisor:http:api.openai.com/*"

Chain entries have operations but no exclude_operations. The chain runs on one operation, so there is nothing to exclude; raw hooks, which reach every path, have both.

The async lane

An entry on the async lane runs beside the request, which never waits for it. Its verdict is recorded on a record of its own, marked enforced: false, but never acted on — not even a deny, and not even one that arrives before the request has gone. It costs the agent's call no time and can never block or rewrite it. That makes it the lane for what you want to know rather than what you want to stop:

• Trialling a rule before it blocks, below.
• Checks too slow or costly to sit in front of a call, such as an http checker that runs a model to classify prompt injection. You get its findings in the audit stream, and the agent does not wait for them.
• Reporting, such as a check that only classifies what kind of data each call carries, so you can see which agents send what where.

A check that must stop something belongs on the sync lane, which holds the request until it answers.

Trial a rule before it blocks

A new rule that matches too much breaks agents, and you only find out once it is blocking real traffic. Put it on the async lane first: it sees the same traffic, but runs alongside the request instead of in front of it, so it never blocks or rewrites anything.

- name: account-numbers
  type: regex_dlp
  lane: async
  options:
    detect:
      - name: account
        pattern: 'ACCT-[0-9]{8}'

Each verdict it reaches is still recorded, marked enforced: false, with its findings — so the audit stream shows exactly what the rule would have redacted or refused, and for whom. When the matches look right, make it enforcing:

  lane: sync             # the default, so the line can simply be removed
  on_error: deny         # the default; set it if you want it explicit

Do not set on_error: deny while the rule is on the async lane. Nothing waits for an async entry, so failing closed would have nothing to refuse, and AgentVisor refuses to start with that combination.