SDK Retry Mechanisms

The AgentVisor Python SDK includes automatic retry mechanisms for transient failures. This ensures agents can recover from temporary network issues, rate limiting, and service unavailability without requiring custom error handling code.

Overview

All SDK operations that communicate with the AgentVisor host runtime are automatically wrapped with retry logic. When a transient gRPC error occurs, the SDK retries the operation with exponential backoff.

Transient errors that trigger retry:

gRPC Status Code	Description
UNAVAILABLE	Service temporarily down, network blip
DEADLINE_EXCEEDED	Request timeout
RESOURCE_EXHAUSTED	Rate limiting

Permanent errors that fail immediately:

gRPC Status Code	Description
NOT_FOUND	Resource doesn't exist
INVALID_ARGUMENT	Invalid request parameters
PERMISSION_DENIED	Authorization denied
UNAUTHENTICATED	Authentication required
All others	Permanent failures

Default Configuration

Each operation type has sensible defaults optimized for its use case:

Operation	Max Attempts	Base Delay	Notes
checkpoint	3	100ms	State persistence
store	3	100ms	Key-value storage
mcp	2	500ms	External tools (may be slow)
a2a	3	200ms	Cross-network communication
stream	2	50ms	Fire-and-forget
heartbeat	2	50ms	Fire-and-forget
log	1	0ms	Best effort, enabled=False (no retry)
state	3	100ms	Per-thread state operations

Configuration

Retry behavior can be configured at multiple levels with the following precedence:

1. Global kill switch - AGENTVISOR_SDK_RETRY_ENABLED=false disables ALL retry
2. Operation-specific config (programmatic or environment)
3. Operation-specific SDK defaults (built-in per operation)
4. Global default config (programmatic or environment)

note

Setting AGENTVISOR_SDK_RETRY_ENABLED=false acts as a master kill switch that disables retry for ALL operations, regardless of per-operation settings. This is useful for testing or when you want to temporarily disable all retry behavior.

Programmatic Configuration

Use configure_retry() to customize retry behavior in your agent code:

from agentvisor.retry import configure_retry, RetryConfig

# Configure global defaults for all operations
configure_retry(default=RetryConfig(max_attempts=5, base_delay_ms=200))

# Configure specific operations
configure_retry(
    checkpoint=RetryConfig(max_attempts=5),
    mcp=RetryConfig(max_attempts=2, base_delay_ms=500),
    store=RetryConfig(max_attempts=3, base_delay_ms=100),
)

# Disable retry for a specific operation
configure_retry(checkpoint=RetryConfig(enabled=False))

RetryConfig Options

Parameter	Type	Default	Description
max_attempts	int	3	Maximum number of attempts (including initial call)
base_delay_ms	int	100	Base delay in milliseconds before first retry
max_delay_ms	int	5000	Maximum delay cap in milliseconds
exponential_base	float	2.0	Base for exponential backoff calculation
jitter	bool	True	Add randomized jitter to delays
enabled	bool	True	Enable/disable retry

Environment Variables

Operators can override retry configuration via environment variables:

Global settings:

Variable	Default	Description
AGENTVISOR_SDK_RETRY_ENABLED	true	Master switch for retry
AGENTVISOR_SDK_RETRY_MAX_ATTEMPTS	3	Default max attempts
AGENTVISOR_SDK_RETRY_BASE_DELAY_MS	100	Default base delay
AGENTVISOR_SDK_RETRY_MAX_DELAY_MS	5000	Default max delay
AGENTVISOR_SDK_RETRY_EXPONENTIAL_BASE	2.0	Exponential backoff base
AGENTVISOR_SDK_RETRY_JITTER	true	Enable jitter

Per-operation overrides:

Replace <OPERATION> with CHECKPOINT, STORE, MCP, A2A, STREAM, HEARTBEAT, LOG, or STATE:

Variable	Description
AGENTVISOR_SDK_RETRY_<OPERATION>_ENABLED	Enable/disable for operation
AGENTVISOR_SDK_RETRY_<OPERATION>_MAX_ATTEMPTS	Max attempts for operation
AGENTVISOR_SDK_RETRY_<OPERATION>_BASE_DELAY_MS	Base delay for operation
AGENTVISOR_SDK_RETRY_<OPERATION>_MAX_DELAY_MS	Max delay for operation
AGENTVISOR_SDK_RETRY_<OPERATION>_EXPONENTIAL_BASE	Exponential backoff base for operation
AGENTVISOR_SDK_RETRY_<OPERATION>_JITTER	Enable jitter for operation

Examples:

# Increase checkpoint retry attempts
export AGENTVISOR_SDK_RETRY_CHECKPOINT_MAX_ATTEMPTS=5

# Disable retry globally
export AGENTVISOR_SDK_RETRY_ENABLED=false

# Configure MCP with longer delays (external tools may be slow)
export AGENTVISOR_SDK_RETRY_MCP_MAX_ATTEMPTS=3
export AGENTVISOR_SDK_RETRY_MCP_BASE_DELAY_MS=1000

Exponential Backoff

The SDK uses exponential backoff with optional jitter to prevent thundering herd problems:

delay = min(base_delay * exponential_base^(attempt-1), max_delay)
if jitter:
    delay = random(0, delay)  # Decorrelated jitter

Example with defaults (100ms base, 2.0 base, 5000ms max):

Attempt	Calculated Delay	With Jitter
1	100ms	0-100ms
2	200ms	0-200ms
3	400ms	0-400ms
4	800ms	0-800ms
5	1600ms	0-1600ms

Fire-and-Forget Operations

Stream and heartbeat operations use "fire-and-forget" semantics:

• Retries are attempted for transient errors
• Errors are suppressed after retries are exhausted
• No exception is raised to the caller
• Agent execution continues uninterrupted

This ensures that network hiccups during streaming don't crash your agent:

from agentvisor import stream

# Even if this fails after retries, your agent continues
stream.emit_chunk({"message": "Processing..."}, chunk_type="status")
# Agent continues execution

note

Permanent errors (like INVALID_ARGUMENT) are still raised even for fire-and-forget operations, as they indicate programming errors that should be fixed.

Error Handling

After retries are exhausted, the original exception is raised. Agents can catch and handle these if needed:

import grpc
from agentvisor import checkpoint

try:
    checkpoint.save("my-state", data)
except grpc.RpcError as e:
    if e.code() == grpc.StatusCode.UNAVAILABLE:
        # Service is down even after retries
        logger.error("Checkpoint service unavailable after retries")
        # Handle gracefully or re-raise

Cause Chain Inspection

The retry mechanism inspects the exception cause chain, so wrapped errors are handled correctly:

# If your code wraps gRPC errors, retry still works
try:
    result = grpc_call()
except grpc.RpcError as e:
    raise RuntimeError("Operation failed") from e  # Retry sees the cause

Logging

Retry attempts are logged at different levels:

• DEBUG: Each retry attempt (attempt number, delay, error code)
• WARNING: Final failure after retries exhausted

Enable debug logging to see retry behavior:

AGENTVISOR_LOG_LEVEL=debug,agent=debug agentvisor serve ./my-agent

Example log output:

DEBUG Retry attempt 1/3 for checkpoint after UNAVAILABLE (delay: 0.087s)
DEBUG Retry attempt 2/3 for checkpoint after UNAVAILABLE (delay: 0.156s)
WARNING Operation checkpoint failed after 3 attempts: service unavailable (UNAVAILABLE)

Best Practices

Let Retry Handle Transient Errors

Don't wrap SDK calls in try/except for transient errors - the SDK handles them automatically:

# Good: Let SDK retry handle transient errors
checkpoint.save("state", data)

# Unnecessary: SDK already retries
try:
    checkpoint.save("state", data)
except grpc.RpcError as e:
    if e.code() == grpc.StatusCode.UNAVAILABLE:
        time.sleep(1)  # SDK already does this with exponential backoff
        checkpoint.save("state", data)

Handle Only Permanent Errors

Catch permanent errors that need application-level handling:

try:
    item = store.get("config", "settings")
except grpc.RpcError as e:
    if e.code() == grpc.StatusCode.NOT_FOUND:
        # Expected case: use defaults
        item = {"theme": "light"}
    else:
        raise  # Unexpected permanent error

Tune for Your Use Case

If your agent calls slow external services via MCP, increase delays:

from agentvisor.retry import configure_retry, RetryConfig

# External database queries may take 30+ seconds
configure_retry(mcp=RetryConfig(
    max_attempts=3,
    base_delay_ms=2000,  # Start with 2 second delay
    max_delay_ms=30000,  # Cap at 30 seconds
))

Disable for Testing

In unit tests, disable retry to make tests faster and more predictable:

import os
import pytest

@pytest.fixture(autouse=True)
def disable_retry():
    os.environ["AGENTVISOR_SDK_RETRY_ENABLED"] = "false"
    yield
    del os.environ["AGENTVISOR_SDK_RETRY_ENABLED"]

API Reference

Functions

configure_retry()

Configure retry behavior programmatically.

def configure_retry(
    default: RetryConfig | None = None,
    *,
    checkpoint: RetryConfig | None = None,
    store: RetryConfig | None = None,
    mcp: RetryConfig | None = None,
    a2a: RetryConfig | None = None,
    stream: RetryConfig | None = None,
    heartbeat: RetryConfig | None = None,
    log: RetryConfig | None = None,
    state: RetryConfig | None = None,
) -> None

reset_retry_settings()

Reset retry settings to defaults and reload from environment variables. Primarily for testing.

def reset_retry_settings() -> None

is_retryable()

Check if an exception is retryable (transient gRPC error).

def is_retryable(error: Exception) -> bool

get_retry_settings()

Get the current retry settings.

def get_retry_settings() -> RetrySettings

Classes

RetryConfig

Immutable configuration for retry behavior.

@dataclass(frozen=True)
class RetryConfig:
    max_attempts: int = 3
    base_delay_ms: int = 100
    max_delay_ms: int = 5000
    exponential_base: float = 2.0
    jitter: bool = True
    enabled: bool = True

RetrySettings

Container for global and per-operation retry configurations.

@dataclass
class RetrySettings:
    default: RetryConfig
    checkpoint: RetryConfig | None
    store: RetryConfig | None
    mcp: RetryConfig | None
    a2a: RetryConfig | None
    stream: RetryConfig | None
    heartbeat: RetryConfig | None
    log: RetryConfig | None
    state: RetryConfig | None

    def for_operation(self, operation: str) -> RetryConfig

Constants

TRANSIENT_GRPC_CODES

The set of gRPC status codes that trigger retry:

TRANSIENT_GRPC_CODES = frozenset({
    grpc.StatusCode.UNAVAILABLE,
    grpc.StatusCode.DEADLINE_EXCEEDED,
    grpc.StatusCode.RESOURCE_EXHAUSTED,
})