Build a Production-Grade OpenAI-Compatible Client: Stop Treating 200 as a Verdict

An OpenAI-compatible base URL makes integration easier. It does not make a client production-ready.

The client still needs to answer four operational questions: is this failure retryable, is a 200 response structurally complete, what did the model report for usage, and can you reconcile the request without exposing a secret? This guide builds that boundary in plain Python for AIWave's OpenAI-compatible route.

The route used here is https://aiwave.live/v1/chat/completions. Model availability is live state, so query your own authenticated /v1/models result before selecting a model.

The design: send once, classify deliberately

The recovery policy is intentionally narrow:

The important rule is not “retry until it works.” It is “retry only the failures you have classified, then stop.”

A small dependency-free wrapper

RETRYABLE = {408, 429, 500, 502, 503, 504}

def request_chat(transport, payload, credential, max_attempts=3):
    if not credential:
        raise RuntimeError("missing credential")

    headers = {
        "Authorization": f"Bearer {credential}",
        "Content-Type": "application/json",
    }

    for attempt in range(1, max_attempts + 1):
        response = transport(payload, headers)
        status = response.get("status")
        body = response.get("body") or {}

        if status == 200:
            choices = body.get("choices")
            if not isinstance(choices, list) or not choices:
                raise RuntimeError("response missing choices")
            message = choices[0].get("message") or {}
            return {
                "attempts": attempt,
                "model": body.get("model"),
                "content": message.get("content", ""),
                "usage": body.get("usage") or {},
            }

        if status not in RETRYABLE or attempt == max_attempts:
            raise RuntimeError(f"request stopped at HTTP {status}")

    raise AssertionError("unreachable")

Keep the transport separate from the recovery policy. That makes it possible to test retries and malformed successful responses without consuming production credits.

Test the unhappy paths before touching a live key

The accompanying example has four deterministic checks:

Run the tests locally before adding any real credential:

$env:PYTHONPATH = (Join-Path (Get-Location) 'api-recovery')
python -m unittest discover -s api-recovery/tests -v

For a real application, load a revocable key from your secret manager or local environment. Do not place it in source files, Markdown, screenshots, browser storage, or exception messages.

The live receipt: what we verified and what we did not

On 2026-10-19, one owner-authorized, bounded, non-streaming call was made with a revocable test key. The authenticated model list contained the selected deepseek-v4-pro ID. The Console ledger recorded 91 input tokens, 16 output tokens, 2.0 seconds elapsed, and a $0.00024 request charge.

This is evidence of one bounded route, usage, and accounting path. It is not a latency promise, a success-rate claim, a cost promise for another account group, or proof that every model and feature behaves the same way.

There was also a useful failure boundary: the assistant message parsed, but it did not match a deliberately strict literal-content assertion. The client stopped and recorded the boundary; it did not retry just to manufacture a nicer result. A later public health check received HTTP 200 but had an incomplete body transfer at the client side, which is likewise a transport condition to surface rather than silently classify as healthy.

Put the receipt beside the request

Before moving a call into a job queue or an agent loop, capture a small, sanitized record:

timestamp
endpoint and model ID
attempt count and terminal status
input/output usage fields returned by the provider
elapsed time
request-level charge or a pointer to the Console ledger
redacted error category when the call stops

This record is more useful than a generic “request succeeded” log. It lets a developer separate a model problem, an account problem, a schema problem, and a transport problem without copying secrets into a ticket.

When not to use this pattern unchanged

This wrapper is for synchronous chat-completion calls. Image editing uses multipart form data. Video creation is asynchronous: HTTP 200 only means submission, and a client must poll to a terminal state, verify persistence, and reconcile settlement before it calls the work complete. Do not apply a chat retry loop blindly to a media task.

Start with one request you can explain

Use AIWave's OpenAI-compatible base URL when it fits your client, then select a currently visible model ID and send one bounded request. Keep the returned usage and your Console receipt together. The goal is not to make every failure disappear; it is to know which failure occurred, what you safely retried, and what you must fix before you send the next request.

Read the Chat Completions documentation and create a test key in the AIWave Console when you are ready to run your own bounded check.