SDK Retry Mechanisms

The AgentVisor™ Python SDK includes automatic retry mechanisms for transient failures. This ensures agents can recover from temporary network issues, rate limiting, and service unavailability without requiring custom error handling code.

Overview

All SDK operations that communicate with the AgentVisor host runtime are automatically wrapped with retry logic. When a transient gRPC error occurs, the SDK retries the operation with exponential backoff.

Transient errors that trigger retry:

gRPC Status Code	Description
`UNAVAILABLE`	Service temporarily down, network blip
`DEADLINE_EXCEEDED`	Request timeout
`RESOURCE_EXHAUSTED`	Rate limiting

Permanent errors that fail immediately:

gRPC Status Code	Description
`NOT_FOUND`	Resource doesn't exist
`INVALID_ARGUMENT`	Invalid request parameters
`PERMISSION_DENIED`	Authorization denied
`UNAUTHENTICATED`	Authentication required
All others	Permanent failures

Default Configuration

Each operation type has sensible defaults optimized for its use case:

Operation	Max Attempts	Base Delay	Notes
`checkpoint`	3	100ms	State persistence
`store`	3	100ms	Key-value storage
`mcp`	2	500ms	External tools (may be slow)
`a2a`	3	200ms	Cross-network communication
`stream`	2	50ms	Fire-and-forget
`heartbeat`	2	50ms	Fire-and-forget
`log`	1	0ms	Best effort, `enabled=False` (no retry)
`state`	3	100ms	Per-thread state operations

Configuration

Retry behavior can be configured at multiple levels with the following precedence:

Global kill switch - AGENTVISOR_SDK_RETRY_ENABLED=false disables ALL retry
Operation-specific config (programmatic or environment)
Operation-specific SDK defaults (built-in per operation)
Global default config (programmatic or environment)

note

Setting AGENTVISOR_SDK_RETRY_ENABLED=false acts as a master kill switch that disables retry for ALL operations, regardless of per-operation settings. This is useful for testing or when you want to temporarily disable all retry behavior.

Programmatic Configuration

Use configure_retry() to customize retry behavior in your agent code:

from agentvisor.retry import configure_retry, RetryConfig

# Configure global defaults for all operations
configure_retry(default=RetryConfig(max_attempts=5, base_delay_ms=200))

# Configure specific operations
configure_retry(
    checkpoint=RetryConfig(max_attempts=5),
    mcp=RetryConfig(max_attempts=2, base_delay_ms=500),
    store=RetryConfig(max_attempts=3, base_delay_ms=100),
)

# Disable retry for a specific operation
configure_retry(checkpoint=RetryConfig(enabled=False))

RetryConfig Options

Parameter	Type	Default	Description
`max_attempts`	`int`	3	Maximum number of attempts (including initial call)
`base_delay_ms`	`int`	100	Base delay in milliseconds before first retry
`max_delay_ms`	`int`	5000	Maximum delay cap in milliseconds
`exponential_base`	`float`	2.0	Base for exponential backoff calculation
`jitter`	`bool`	True	Add randomized jitter to delays
`enabled`	`bool`	True	Enable/disable retry

Environment Variables

Operators can override retry configuration via environment variables:

note

The variables below (AGENTVISOR_SDK_RETRY_*) are what the SDK reads inside the guest. An operator sets them from the host side, in agentvisor.yaml under guest.sdk.retry or via the corresponding AGENTVISOR_GUEST_SDK_RETRY_* host environment variables — the host runtime translates those into the guest-visible AGENTVISOR_SDK_RETRY_* variables shown here at sandbox startup. See the Configuration Reference for the full list of host-side variables and their YAML equivalents.

Global settings:

Variable	Default	Description
`AGENTVISOR_SDK_RETRY_ENABLED`	`true`	Master switch for retry
`AGENTVISOR_SDK_RETRY_MAX_ATTEMPTS`	`3`	Default max attempts
`AGENTVISOR_SDK_RETRY_BASE_DELAY_MS`	`100`	Default base delay
`AGENTVISOR_SDK_RETRY_MAX_DELAY_MS`	`5000`	Default max delay
`AGENTVISOR_SDK_RETRY_EXPONENTIAL_BASE`	`2.0`	Exponential backoff base
`AGENTVISOR_SDK_RETRY_JITTER`	`true`	Enable jitter

Per-operation overrides:

Replace <OPERATION> with CHECKPOINT, STORE, MCP, A2A, STREAM, HEARTBEAT, LOG, or STATE:

Variable	Description
`AGENTVISOR_SDK_RETRY_<OPERATION>_ENABLED`	Enable/disable for operation
`AGENTVISOR_SDK_RETRY_<OPERATION>_MAX_ATTEMPTS`	Max attempts for operation
`AGENTVISOR_SDK_RETRY_<OPERATION>_BASE_DELAY_MS`	Base delay for operation
`AGENTVISOR_SDK_RETRY_<OPERATION>_MAX_DELAY_MS`	Max delay for operation
`AGENTVISOR_SDK_RETRY_<OPERATION>_EXPONENTIAL_BASE`	Exponential backoff base for operation
`AGENTVISOR_SDK_RETRY_<OPERATION>_JITTER`	Enable jitter for operation

Examples:

# Increase checkpoint retry attempts
export AGENTVISOR_SDK_RETRY_CHECKPOINT_MAX_ATTEMPTS=5

# Disable retry globally
export AGENTVISOR_SDK_RETRY_ENABLED=false

# Configure MCP with longer delays (external tools may be slow)
export AGENTVISOR_SDK_RETRY_MCP_MAX_ATTEMPTS=3
export AGENTVISOR_SDK_RETRY_MCP_BASE_DELAY_MS=1000

These AGENTVISOR_SDK_RETRY_* variables apply directly when set inside the guest process itself (e.g. sandbox: none, or a custom framework.env entry). For a sandboxed guest (docker/gvisor), set the host-side equivalent instead — e.g. AGENTVISOR_GUEST_SDK_RETRY_CHECKPOINT_MAX_ATTEMPTS=5 in the host's environment or guest.sdk.retry.checkpoint.max_attempts: 5 in agentvisor.yaml — and the host runtime injects the corresponding AGENTVISOR_SDK_RETRY_* variable into the guest at startup. See the Configuration Reference.

Exponential Backoff

The SDK uses exponential backoff with optional jitter to prevent thundering herd problems:

delay = min(base_delay * exponential_base^(attempt-1), max_delay)
if jitter:
    delay = random(0, delay)  # Decorrelated jitter

Example with defaults (100ms base, 2.0 base, 5000ms max):

Attempt	Calculated Delay	With Jitter
1	100ms	0-100ms
2	200ms	0-200ms
3	400ms	0-400ms
4	800ms	0-800ms
5	1600ms	0-1600ms

Fire-and-Forget Operations

Stream and heartbeat operations use "fire-and-forget" semantics:

Retries are attempted for transient errors
Errors are suppressed after retries are exhausted
No exception is raised to the caller
Agent execution continues uninterrupted

This ensures that network hiccups during streaming don't crash your agent:

from agentvisor import stream

# Even if this fails after retries, your agent continues
stream.emit_chunk({"message": "Processing..."}, chunk_type="status")
# Agent continues execution

note

Permanent errors (like INVALID_ARGUMENT) are still raised even for fire-and-forget operations, as they indicate programming errors that should be fixed.

Error Handling

After retries are exhausted, the original exception is raised. Agents can catch and handle these if needed:

import grpc
from agentvisor import checkpoint

try:
    checkpoint.save("my-state", data)
except grpc.RpcError as e:
    if e.code() == grpc.StatusCode.UNAVAILABLE:
        # Service is down even after retries
        logger.error("Checkpoint service unavailable after retries")
        # Handle gracefully or re-raise

Cause Chain Inspection

The retry mechanism inspects the exception cause chain, so wrapped errors are handled correctly:

# If your code wraps gRPC errors, retry still works
try:
    result = grpc_call()
except grpc.RpcError as e:
    raise RuntimeError("Operation failed") from e  # Retry sees the cause

Logging

Retry attempts are logged at different levels:

DEBUG: Each retry attempt (attempt number, delay, error code)
WARNING: Final failure after retries exhausted

Enable debug logging to see retry behavior:

AGENTVISOR_LOG_LEVEL=debug,agent=debug agentvisor serve ./my-agent

Example log output:

DEBUG Retry attempt 1/3 for checkpoint after UNAVAILABLE (delay: 0.087s)
DEBUG Retry attempt 2/3 for checkpoint after UNAVAILABLE (delay: 0.156s)
WARNING Operation checkpoint failed after 3 attempts: service unavailable (UNAVAILABLE)

Best Practices

Let Retry Handle Transient Errors

Don't wrap SDK calls in try/except for transient errors - the SDK handles them automatically:

# Good: Let SDK retry handle transient errors
checkpoint.save("state", data)

# Unnecessary: SDK already retries
try:
    checkpoint.save("state", data)
except grpc.RpcError as e:
    if e.code() == grpc.StatusCode.UNAVAILABLE:
        time.sleep(1)  # SDK already does this with exponential backoff
        checkpoint.save("state", data)

Handle Only Permanent Errors

Catch permanent errors that need application-level handling:

try:
    item = store.get("config", "settings")
except grpc.RpcError as e:
    if e.code() == grpc.StatusCode.NOT_FOUND:
        # Expected case: use defaults
        item = {"theme": "light"}
    else:
        raise  # Unexpected permanent error

Tune for Your Use Case

If your agent calls slow external services via MCP, increase delays:

from agentvisor.retry import configure_retry, RetryConfig

# External database queries may take 30+ seconds
configure_retry(mcp=RetryConfig(
    max_attempts=3,
    base_delay_ms=2000,  # Start with 2 second delay
    max_delay_ms=30000,  # Cap at 30 seconds
))

Disable for Testing

In unit tests, disable retry to make tests faster and more predictable:

import os
import pytest

@pytest.fixture(autouse=True)
def disable_retry():
    os.environ["AGENTVISOR_SDK_RETRY_ENABLED"] = "false"
    yield
    del os.environ["AGENTVISOR_SDK_RETRY_ENABLED"]

API Reference

Functions

`configure_retry()`

Configure retry behavior programmatically.

def configure_retry(
    default: RetryConfig | None = None,
    *,
    checkpoint: RetryConfig | None = None,
    store: RetryConfig | None = None,
    mcp: RetryConfig | None = None,
    a2a: RetryConfig | None = None,
    stream: RetryConfig | None = None,
    heartbeat: RetryConfig | None = None,
    log: RetryConfig | None = None,
    state: RetryConfig | None = None,
) -> None

`reset_retry_settings()`

Reset retry settings to defaults and reload from environment variables. Primarily for testing.

def reset_retry_settings() -> None

`is_retryable()`

Check if an exception is retryable (transient gRPC error).

def is_retryable(error: Exception) -> bool

`get_retry_settings()`

Get the current retry settings.

def get_retry_settings() -> RetrySettings

Classes

`RetryConfig`

Immutable configuration for retry behavior.

@dataclass(frozen=True)
class RetryConfig:
    max_attempts: int = 3
    base_delay_ms: int = 100
    max_delay_ms: int = 5000
    exponential_base: float = 2.0
    jitter: bool = True
    enabled: bool = True

`RetrySettings`

Container for global and per-operation retry configurations.

@dataclass
class RetrySettings:
    default: RetryConfig
    checkpoint: RetryConfig | None
    store: RetryConfig | None
    mcp: RetryConfig | None
    a2a: RetryConfig | None
    stream: RetryConfig | None
    heartbeat: RetryConfig | None
    log: RetryConfig | None
    state: RetryConfig | None

    def for_operation(self, operation: str) -> RetryConfig

Constants

`TRANSIENT_GRPC_CODES`

The set of gRPC status codes that trigger retry:

TRANSIENT_GRPC_CODES = frozenset({
    grpc.StatusCode.UNAVAILABLE,
    grpc.StatusCode.DEADLINE_EXCEEDED,
    grpc.StatusCode.RESOURCE_EXHAUSTED,
})

Overview​

Default Configuration​

Configuration​

Programmatic Configuration​

RetryConfig Options​

Environment Variables​

Exponential Backoff​

Fire-and-Forget Operations​

Error Handling​

Cause Chain Inspection​

Logging​

Best Practices​

Let Retry Handle Transient Errors​

Handle Only Permanent Errors​

Tune for Your Use Case​

Disable for Testing​

API Reference​

Functions​

configure_retry()​

reset_retry_settings()​

is_retryable()​

get_retry_settings()​

Classes​

RetryConfig​

RetrySettings​

Constants​

TRANSIENT_GRPC_CODES​

Overview

Default Configuration

Configuration

Programmatic Configuration

RetryConfig Options

Environment Variables

Exponential Backoff

Fire-and-Forget Operations

Error Handling

Cause Chain Inspection

Logging

Best Practices

Let Retry Handle Transient Errors

Handle Only Permanent Errors

Tune for Your Use Case

Disable for Testing

API Reference

Functions

`configure_retry()`

`reset_retry_settings()`

`is_retryable()`

`get_retry_settings()`

Classes

`RetryConfig`

`RetrySettings`

Constants

`TRANSIENT_GRPC_CODES`