Gateway Success Rates Need Denominators for AI API Buyers

Why This Topic Matters Now

A gateway success rate can mislead when it hides the denominator. The Aug 27 AIWave model success report was useful because it did the opposite: it listed the rolling window, source limits, request counts, successes, latency, time to first token, weak rows, and zero-success aliases. That is the standard Tier 1 buyers should ask for when they evaluate any Chinese AI API gateway. A percentage without sample size is not enough.

This article turns the report into a buyer runbook. It does not treat an observed success percentage as an SLA. It shows how to compare high-volume DeepSeek rows, moderate ERNIE and GLM rows, tiny-sample rows, and immediate zero-success aliases. It also connects reliability evidence to price evidence, because a route that succeeds often but produces long output or retries heavily still needs cost controls before production traffic moves.

Source Facts Checked Today

The Aug 27 AIWave observed model success report covered the previous 30 days ending at 18:00 Asia/Shanghai. It recorded 5,698 perf_metrics requests and 5,457 recorded successes, or 95.7704 percent observed success. Average end-to-end latency was 12,119.8 ms and average time to first token was 926.6 ms. A separate request-log query excluded staff accounts and found 5,455 model calls from 51 users, so the report preserved the source mismatch instead of hiding it.

The production-relevant rows showed deepseek-v4-flash at 3,093 requests and 3,090 successes, with average latency 7,331.5 ms and average time to first token 999.7 ms. It showed deepseek-v4-pro at 1,831 requests and 1,831 successes, with average latency 19,706.9 ms and average time to first token 451.2 ms. Those denominators are materially more useful than tiny rows, even when a tiny row also reports 100 percent success.

The same report flagged weak or stale-route candidates: glm-4.7-flash at 23 requests and 5 successes, glm-4.7 at 19 requests and 13 successes, glm-5 at 15 requests and 8 successes, and several zero-success aliases with 100-400 ms average latency. The report treated those as triage hypotheses, not final blame. A buyer should copy that discipline: classify authentication, model-not-found, provider rejection, timeout, and generation errors separately.

Planning Matrix

A source-dated planning matrix keeps the page useful for engineers and procurement reviewers. It turns a search query into an auditable route decision instead of a loose model preference.

Implementation Pattern

The implementation pattern keeps credentials as placeholders, pins the AIWave base URL, records the model, and leaves room for route-specific controls. Production applications should move credentials into environment or secret storage.

AIWAVE_API_KEY = "YOUR_API_KEY_HERE"

def summarize_route(model, requests, successes, avg_latency_ms, avg_ttft_ms):
    if requests == 0:
        return {"model": model, "status": "no_sample"}
    success_rate = successes / requests
    denominator_band = "high" if requests >= 1000 else "moderate" if requests >= 50 else "small"
    return {
        "model": model,
        "requests": requests,
        "success_rate": round(success_rate, 6),
        "denominator_band": denominator_band,
        "avg_latency_ms": avg_latency_ms,
        "avg_ttft_ms": avg_ttft_ms,
    }

print(summarize_route("deepseek-v4-pro", 1831, 1831, 19706.9, 451.2))

Do Not Compare Percentages Alone

A 100 percent row on four requests should not be placed beside a 100 percent row on 1,831 requests as if the evidence is equal. Use denominator bands: high, moderate, small, and investigation-only. This keeps the sales conversation honest and helps engineers choose a first production route. A buyer should ask how many requests created the percentage, which traffic was excluded, and whether internal tests were separated from external usage.

Separate Latency From Availability

A route can succeed and still be unsuitable for an interactive product if average latency or time to first token is high for the workload. DeepSeek V4 Pro may be acceptable for planning and review while a faster route handles chatty execution. Kimi K3 may pass a tiny sample but still show extreme completion latency in one window. Treat latency as a workload-fit metric, not a single platform grade.

Classify Immediate Zero-Success Rows

Rows with zero successes and very short latency often point to alias problems, unavailable routes, provider rejection, or authorization issues rather than slow model generation. Do not remove them from evidence. Put them in a triage table with model name, request count, average latency, likely class, owner, and next test. This is how a gateway proves operational maturity: it shows the weak rows and the plan to investigate them.

Connect Reliability to Cost

Retries, timeouts, long outputs, and fallback calls all change cost. A reliability review should therefore sit beside the pricing ledger. For DeepSeek, use dated AIWave rows and direct-provider context. For GLM, preserve Z.AI rows and tool fees separately. For any route, log retries, stop reasons, output tokens, and fallback target. A route that looks inexpensive in one table can become expensive if failures trigger repeated calls.

Make Staff Exclusion Reproducible

The Aug 27 report explicitly noted that perf_metrics did not contain user identity while the request-log query could exclude staff roles. That transparency matters. If a future success report excludes staff traffic, the query rule should be reproducible. If it cannot be reproduced in the metric source, keep the mismatch visible. A buyer will trust a limited report more than a polished percentage that hides its method.

Internal Links for Trust Review

Route trust-focused readers to Trust, Pricing, Models docs, Chat Completions docs, billing-group verification, and the DeepSeek cost ledger. Reliability, billing, and route discovery should reinforce each other.

Procurement Review

Procurement should ask whether observed metrics are being presented as a contract, which sample window was used, how staff traffic was handled, which routes have high denominators, which rows require investigation, and how retries affect spend. Engineering should attach a route table with requests, successes, latency, TTFT, output caps, and fallback policy. The review should approve a workload, not an entire model catalog.

Final Checklist

A denominator-aware gateway review includes the rolling window, request count, successes, success percentage, latency, TTFT, traffic exclusions, route names, stale aliases, retry policy, and dated price rows. It separates observed performance from SLA promises and keeps weak rows visible. That is the practical way for Tier 1 API buyers to evaluate AIWave or any Chinese-model gateway without relying on broad reliability claims.