Gateway Success Rates Need Denominators for AI API Buyers

Why This Topic Matters Now

A gateway success rate can mislead when it hides the denominator, route boundary, traffic exclusions, or failure classes. Tier 1 buyers should ask for the test window, route ID, sample boundary, latency distribution, retry policy, and the exact distinction between observed results and a contractual SLA.

This article turns those requirements into a buyer runbook. It shows how to compare representative routes, small fixtures, and immediate failures while connecting reliability evidence to price evidence. A route that succeeds often but produces long output or repeated retries still needs cost controls before production traffic moves.

Evidence Fields to Collect

Start with a route-specific acceptance plan. Record the model ID, client configuration, region, payload shape, output cap, timeout, retry rule, and price snapshot before the first test. Without that boundary, one success percentage cannot describe the workload a buyer intends to run.

For each route, keep the outcome class, latency distribution, time to first token, output behavior, and retry cost together. Compare routes only when the payload and acceptance criteria are equivalent; otherwise the rows describe different experiments.

Treat weak or immediate-failure rows as triage hypotheses, not final blame. Classify authentication, model-not-found, provider rejection, timeout, and generation errors separately, then record the owner and next test.

Planning Matrix

A source-dated planning matrix keeps the page useful for engineers and procurement reviewers. It turns a search query into an auditable route decision instead of a loose model preference.

Implementation Pattern

The implementation pattern keeps credentials as placeholders, pins the AIWave base URL, records the model, and leaves room for route-specific controls. Production applications should move credentials into environment or secret storage.

AIWAVE_API_KEY = "YOUR_API_KEY_HERE"

def summarize_route(model, requests, successes, avg_latency_ms, avg_ttft_ms):
    if requests == 0:
        return {"model": model, "status": "no_sample"}
    success_rate = successes / requests
    denominator_band = "high" if requests >= 1000 else "moderate" if requests >= 50 else "small"
    return {
        "model": model,
        "requests": requests,
        "success_rate": round(success_rate, 6),
        "denominator_band": denominator_band,
        "avg_latency_ms": avg_latency_ms,
        "avg_ttft_ms": avg_ttft_ms,
    }

# Synthetic fixture. Replace every value with your own acceptance-test result.
print(summarize_route("YOUR_MODEL_ID", 120, 117, 840.0, 220.0))

Do Not Compare Percentages Alone

A perfect percentage from a tiny fixture should not be placed beside one from a sustained production window as if the evidence is equal. Use denominator bands: high, moderate, small, and investigation-only. A buyer should ask what created the percentage, which traffic was excluded, and whether internal tests were separated from external usage.

Separate Latency From Availability

A route can succeed and still be unsuitable for an interactive product if average latency or time to first token is high for the workload. DeepSeek V4 Pro may be acceptable for planning and review while a faster route handles chatty execution. Kimi K3 may pass a tiny sample but still show extreme completion latency in one window. Treat latency as a workload-fit metric, not a single platform grade.

Classify Immediate Zero-Success Rows

Rows with zero successes and very short latency often point to alias problems, unavailable routes, provider rejection, or authorization issues rather than slow model generation. Do not remove them from evidence. Put them in a triage table with model name, request count, average latency, likely class, owner, and next test. This is how a gateway proves operational maturity: it shows the weak rows and the plan to investigate them.

Connect Reliability to Cost

Retries, timeouts, long outputs, and fallback calls all change cost. A reliability review should therefore sit beside the pricing ledger. For DeepSeek, use dated AIWave rows and direct-provider context. For GLM, preserve Z.AI rows and tool fees separately. For any route, log retries, stop reasons, output tokens, and fallback target. A route that looks inexpensive in one table can become expensive if failures trigger repeated calls.

Make Test Exclusion Reproducible

If a success report excludes internal or synthetic traffic, the query rule should be reproducible. If exclusion cannot be reproduced from the metric source, keep that limitation visible. A buyer can evaluate a bounded method more reliably than a polished percentage that hides its inputs.

Internal Links for Trust Review

Route trust-focused readers to Trust, Pricing, Models docs, Chat Completions docs, billing-group verification, and the DeepSeek cost ledger. Reliability, billing, and route discovery should reinforce each other.

Procurement Review

Procurement should ask whether observed metrics are being presented as a contract, which sample window was used, how staff traffic was handled, which routes have high denominators, which rows require investigation, and how retries affect spend. Engineering should attach a route table with requests, successes, latency, TTFT, output caps, and fallback policy. The review should approve a workload, not an entire model catalog.

Final Checklist

A denominator-aware gateway review includes the rolling window, request count, successes, success percentage, latency, TTFT, traffic exclusions, route names, stale aliases, retry policy, and dated price rows. It separates observed performance from SLA promises and keeps weak rows visible. That is the practical way for Tier 1 API buyers to evaluate AIWave or any Chinese-model gateway without relying on broad reliability claims.