AI API Error Handling: Retry Strategies & Status Codes

July 29, 2026 · 11 min read

Every AI API call can fail. Servers go down, rate limits get hit, context windows overflow, and network packets get lost. The difference between a brittle prototype and a production-grade AI application is how it handles these failures.

This guide covers the error handling patterns we recommend for any application calling Chinese AI APIs — whether through AIWave, directly, or through other providers. We'll cover status codes, retry strategies, circuit breakers, model fallbacks, and complete code examples in Python.

Common Error Status Codes

Common Error Messages

Retry Strategy: Exponential Backoff with Jitter

The gold standard for API retries. Here's a production-ready implementation:

import time
import random
import logging
from openai import OpenAI, APITimeoutError, RateLimitError, APIConnectionError, InternalServerError

logger = logging.getLogger(__name__)

def call_with_retry(
    client: OpenAI,
    messages: list,
    model: str = "deepseek-v4-flash",
    max_retries: int = 5,
    base_delay: float = 1.0,
    max_delay: float = 30.0,
    **kwargs
):
    """Call an AI API with exponential backoff + jitter retry."""
    last_error = None
    
    for attempt in range(max_retries + 1):
        try:
            response = client.chat.completions.create(
                model=model,
                messages=messages,
                **kwargs
            )
            return response
            
        except (RateLimitError, InternalServerError, APIConnectionError, APITimeoutError) as e:
            last_error = e
            if attempt == max_retries:
                break
            
            # Calculate delay with exponential backoff + jitter
            delay = min(base_delay * (2 ** attempt), max_delay)
            jitter = random.uniform(0, 0.5)
            total_delay = delay + jitter
            
            logger.warning(
                f"API error ({type(e).__name__}), retry {attempt+1}/{max_retries} "
                f"after {total_delay:.1f}s: {e}"
            )
            time.sleep(total_delay)
    
    raise last_error  # type: ignore

# Usage
client = OpenAI(api_key="key", base_url="https://api.aiwave.live/v1")
response = call_with_retry(
    client,
    messages=[{"role": "user", "content": "Hello"}],
    model="deepseek-v4-flash",
    max_retries=3
)

Circuit Breaker Pattern

When a model is consistently failing (e.g., 503 errors), you want to stop hammering it and switch to a fallback. A circuit breaker does exactly that:

from collections import deque
import time

class CircuitBreaker:
    def __init__(self, failure_threshold=5, recovery_timeout=60):
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.failures = deque(maxlen=failure_threshold)
        self.open_until = 0
    
    def can_execute(self) -> bool:
        if time.time() > self.open_until:
            self.failures.clear()
            return True
        return len(self.failures) < self.failure_threshold
    
    def record_success(self):
        self.failures.clear()
    
    def record_failure(self):
        self.failures.append(time.time())
        if len(self.failures) >= self.failure_threshold:
            self.open_until = time.time() + self.recovery_timeout

# Per-model circuit breakers
breakers = {
    "deepseek-v4-flash": CircuitBreaker(failure_threshold=5, recovery_timeout=60),
    "glm-5": CircuitBreaker(failure_threshold=5, recovery_timeout=60),
    "qwen3": CircuitBreaker(failure_threshold=5, recovery_timeout=60),
}

FALLBACK_ORDER = ["deepseek-v4-flash", "glm-5", "qwen3"]

def call_with_fallback(client, messages, **kwargs):
    for model in FALLBACK_ORDER:
        if not breakers[model].can_execute():
            continue
        try:
            response = call_with_retry(client, messages, model=model, max_retries=2, **kwargs)
            breakers[model].record_success()
            return response, model
        except Exception as e:
            breakers[model].record_failure()
            logger.error(f"Model {model} failed: {e}")
    raise RuntimeError("All models failed")

# Usage — automatically falls back between models
response, used_model = call_with_fallback(
    client,
    messages=[{"role": "user", "content": "Summarize this article."}]
)
print(f"Response from {used_model}")

Timeout Handling

# Set per-request timeout (seconds)
response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=messages,
    timeout=30.0  # 30 second timeout
)

# For streaming, set a max wait time between chunks
import signal

def handler(signum, frame):
    raise TimeoutError("No data received for 10s")

signal.signal(signal.SIGALRM, handler)
stream = client.chat.completions.create(model="deepseek-v4-flash", messages=messages, stream=True)
for chunk in stream:
    signal.alarm(10)  # Reset alarm on each chunk
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
signal.alarm(0)

Graceful Degradation Strategies

• Cached responses: Return a cached answer for common queries when the API is down
• Queue and retry: For non-real-time tasks, queue failed requests and process later
• Model downgrade: Fall from V4 Pro → V4 Flash → smaller model to increase success probability
• Partial responses: For streaming, display whatever was received before the error

Frequently Asked Questions

What are the most common AI API errors?

The most common AI API errors are 429 (rate limit), 500 (server error), 503 (service unavailable), and 400 (bad request including invalid model name, context length exceeded, or malformed JSON).

How should I handle rate limits on Chinese AI APIs?

Implement exponential backoff with jitter: start at 1 second delay, double each retry up to a max of 30 seconds. Add random jitter (0-500ms) to prevent thundering herd. Use the Retry-After header when provided.

Should I build a circuit breaker for AI API calls?

Yes, especially if you call multiple models. A circuit breaker prevents wasting time on a model that's currently down, automatically routing to healthy alternatives. It takes ~50 lines of code and saves significant debugging time.