Skip to main content

Writing a Content Checker

The http chain type is how you bring your own control into the chain — a DLP service, a prompt-injection classifier, an allow/deny policy of your own. AgentVisor POSTs the spans of each LLM request to your URL and acts on what you return.

A checker is a chain entry, configured under content_middleware.chain like the built-in types, so it runs only where the chain runs: on the agent's HTTP requests.

A versioned contract

What AgentVisor sends your checker and what it accepts back are a published contract, versioned by the X-AgentVisor-Content-Protocol header (currently 1). The full schema is in the Content Middleware Reference.

What your checker receives​

{
  "operation": "agentvisor:http:request",
  "resource": "mrn:agentvisor:http:api.openai.com/v1/chat/completions",
  "phase": "request",
  "principal": {"claims": {"sub": "alice"}},
  "thread_id": "…", "run_id": "…", "agent": "…", "agentvisor_id": "…",
  "spans": [
    {"role": "user", "text": "my ssn is 123-45-6789", "ref": "/messages/1/content"}
  ],
  "fields": {"url": "https://api.openai.com/v1/chat/completions", "method": "POST"}
}

Each entry in spans is one piece of the request's conversational text: its role, as the provider wrote it (system, developer, user, assistant, tool or function), or empty where there is none, as for Anthropic's top-level system prompt or OpenAI's prediction.content, the text, and a ref — an RFC 6901 JSON Pointer to where it sits in the request, or for an object key a key: ref (below). Return edits against ref; you never need to interpret it beyond that prefix, and the same three fields arrive whichever provider the agent called.

A request no extractor recognizes arrives, under the default on_unknown_format: whole_body, as a span whose ref is empty ("") and whose role is empty, holding the whole body. A JSON body arrives as two such spans: the body as sent, and its text with escapes decoded. Tool-call arguments that name a key twice also arrive with a decoded span of this kind. A span with an empty ref is read-only, since an edit to it cannot be written back, so answer deny when you find something in one.

A tool call arrives as one span per value. For an Anthropic tool call the ref points into the tool_use input, as in /messages/1/content/0/input/to. For an OpenAI tool call it continues below the arguments string, as in /messages/2/tool_calls/0/function/arguments/to, because the arguments are JSON carried inside that string. Edit either like any other ref. A number arrives as its text, with "number": true, and cannot be edited, since the replacement would not be a number: an edit to one denies the call, whatever the entry's on_error, so answer deny for a number your checker would otherwise redact. Each key arrives as a span of its own just before its value. Its ref starts with key: and names the object and the key's place among its keys, as in key:/messages/1/content/0/input#2; the reference says how keys are counted. It never contains the key itself, so a finding that names it does not copy the key into the audit record. A key cannot be edited, since that would rename it: an edit to one denies the call, whatever the entry's on_error, as your checker has said the key must not go out. To stop a key that matches, return deny.

fields carries what AgentVisor knows about the call. It is kept apart from the top-level keys, so the top-level shape stays the same as more is added and a checker written today keeps working.

The reference lists every key and the phase it arrives on.

On the response phase​

A checker is called on the request only, unless you opt in to the response:

options:
  url: https://usage.internal/record
  observe_response: true

On the response, a checker is called once the reply has reached the agent. It receives an empty spans list and fields, and its verdict is recorded but not enforced: a deny or an edit there changes nothing the agent receives. The audit record marks it enforced: false, and the first time a checker returns one AgentVisor logs a warning saying so. A response-phase deny does not stop the entries after it, so token metering still runs.

Use the response phase to observe, for example to record usage your own way. To block a reply, use a raw_response hook with delivery: whole.

What your checker returns​

{
  "action": "modify",
  "reason": "pii detected",
  "edits":    [{"ref": "/messages/1/content", "text": "my ssn is <SSN>"}],
  "findings": [{"ref": "/messages/1/content", "type": "us_ssn", "confidence": 0.85}],
  "fields":   {"detector_version": "2.1", "policy_bundle": "abc"}
}
  • action — permit, deny, or modify. Anything else is refused.
  • edits — only honored on modify. A ref that matches no span is an error, and an empty ref (the whole payload) is refused. An edit or parameter edit that cannot be used counts as your checker failing, handled by its on_error; under allow, none of that verdict's edits or parameter edits are made.
  • findings — recorded whatever the verdict. A permit carrying findings is the record of something noticed and deliberately allowed.
  • parameter_edits — change a generation parameter rather than content, e.g. [{"ref": "/max_tokens", "value": 4096}]. Only honored on modify. Because a parameter is a number, the audit record shows the new value. A token limit is only ever lowered, and on a request that sets none an edit adds one. The parameters you can set, and the rules for each, are in the reference.
  • fields — your own annotations, such as a detector version. They are recorded on the audit entry as checker_fields and go nowhere else: later checks in the chain do not see them, and nor does the provider. The record holds the fields of every checker in the chain together, and a later checker's key replaces an earlier one's of the same name, so give your keys a prefix of your own (for example presidio.version). See the warning below.

You may return the whole object you were handed with fields added. Anything AgentVisor does not act on is ignored rather than rejected.

That includes the fields AgentVisor sent you, so returning them is safe. The audit record keeps the method, the endpoint (never the URL's query string) and token counts in fields of its own, and every key AgentVisor sent is dropped from your returned fields. The audit entry's checker_fields holds only the keys you added.

Change text with edits, not by changing spans

If your checker changes the spans it received and returns them, AgentVisor ignores the change. Text is changed only by edits. AgentVisor logs this once per chain entry, so a mistake shows up on the first request:

content checker returned modified spans, which are ignored; return edits instead
What your checker returns is recorded as you send it

AgentVisor never copies message text or tool-argument values into the audit record itself (see what it does record). Your checker's reason, each finding's type and ref, and your fields are recorded exactly as you return them, and a deny reason is also what the agent is told. Use them for rule names, detector versions, policy bundle ids and your own latency. Putting matched text in any of them puts it in your audit stream.

Keep it small: the fields of every checker on a record are capped together at 4 KiB once encoded. Past that the record holds {"truncated": true, "bytes": N} in place of all of them, so one large checker's annotations replace every checker's.

A reference adapter​

Roughly twenty lines, here wrapping Presidio:

@app.post("/inspect")
def inspect(req: dict = Body(...)):
    edits, findings = [], []
    for span in req["spans"]:
        results = analyzer.analyze(text=span["text"], language="en")
        if not results:
            continue
        findings += [{"ref": span["ref"],
                      "type": r.entity_type.lower(),
                      "confidence": r.score} for r in results]
        if span["ref"] == "" or span["ref"].startswith("key:") or span.get("number"):
            # The whole body, a key or a number cannot be redacted, so the
            # call is refused.
            return {"action": "deny", "reason": "sensitive data that cannot be redacted",
                    "findings": findings}
        clean = anonymizer.anonymize(text=span["text"], analyzer_results=results)
        edits.append({"ref": span["ref"], "text": clean.text})

    return {"action": "modify" if edits else "permit",
            "edits": edits, "findings": findings}

The adapter never needs to know which provider the agent called: it reads each span's text and returns edits against the span's ref, refusing the call when the match is somewhere it cannot redact. Body is FastAPI's (from fastapi import Body); without it FastAPI reads req as a query parameter.

chain:
  - name: presidio-adapter
    type: http
    on_error: deny
    timeout: 5s
    options:
      url: https://dlp.internal/inspect
      headers:
        - name: Authorization
          value_env: DLP_TOKEN
      tls:
        ca_file: /etc/ssl/internal-ca.pem

Unlike a raw hook, a checker can authenticate itself to your service with headers and verify it against a private CA with tls, so it need not run on the same host.

Put credentials in a header rather than the URL. The rules for url are in the reference.

Two entries of this type with different URLs are two middleware of one type, told apart in the audit record by their name.

What identity your checker receives​

By default, only the subject:

"principal": {"claims": {"sub": "alice"}}

That matches what the audit record keeps, and is all most checkers need — a DLP adapter needs no identity at all. Because the payload leaves the host, possibly to a vendor, the default sends as little identity as possible.

Widen it per entry when a checker needs more:

options:
  url: https://budget.internal/check
  principal_claims: [sub, groups]

Exactly the claims you name are sent. principal_claims: [] sends no principal at all. Names are matched against the claims in the caller's token, including namespaced ones like https://corp.test/roles; a name that matches nothing is left out, so a typo sends less identity, never more. When none of them matches, no principal is sent.

Failure is a denial​

Any failure — timeout, non-2xx status, a response over 4 MiB, unparseable body, unrecognised action — is an error, handled by on_error. It defaults to deny: a checker that could not answer has not approved the call. The entry's timeout bounds the whole call to your checker, and there is no separate timeout under options: setting one refuses to start.

Redirects are not followed. The url you configure is where the payload and your headers are allowed to go, so a 3xx is a failed call, and the error says to set url to the final address.

Retry is off by default and not allowed on the sync lane, because a retry uses up the timeout the agent is waiting on. To retry, move the entry to lane: async — its verdict then arrives after the request has gone, so it cannot deny.