DeepSeek vs GLM vs Kimi: An API Evaluation Matrix for Production Workloads

Why This Topic Matters Now

The Sep 19 keyword report surfaced the practical query DeepSeek versus GLM versus Kimi. A production team does not need another global leaderboard. It needs an evaluation matrix that says which model passed which task, under which prompt shape, with which route and source date.

The current AIWave evidence has two distinct surfaces. The live route response shows 68 current route rows and group context. The dated public JSON shows 56 USD rows with its own checked and updated dates. A sound comparison keeps those surfaces separate, so an evaluator can discuss both availability and budget without treating internal route ratios as public dollar prices.

Source Facts Checked Today

AIWave /api/pricing was checked from production on Sep 20, 2026 and returned HTTP 200, success=true, 68 live route rows, pricing_version a42d372ccf0b5dd13ecf71203521f9d2, auto_groups=['default'], and group_ratio default=1 and vip=0.9. The public /api/v1/pricing endpoint also returned HTTP 200 with 56 dated USD rows, pricing_version 83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5, updated_at=2026-09-18, and row checked dates of 2026-09-10. Use the live endpoint for route and group evidence, and the static endpoint for dated public USD rates.

The dated public rows checked during this run list DeepSeek V4 Pro at $1.914 input, $0.0637362 cache-hit input, and $5.742 output per 1M tokens; GLM 5.1 at $2.1, $0.680001, and about $6.6; and Kimi K3 at $4.5, $0.9, and $22.5. Each row is a dated base-rate record, not a permanent quote.

The live route response lists deepseek-v4-pro, glm-5.1, and kimi-k3 with OpenAI-compatible endpoint types and separate route-governance ratios. Those ratios help explain gateway routing configuration; they are not a replacement for the dated USD table.

DeepSeek, Z.AI, and Kimi publish their own documentation and pricing concepts. Review direct-provider claims independently, then compare the exact AIWave model ID, endpoint, source date, and acceptance result used by the application.

Planning Matrix

A source-dated planning matrix keeps the page useful for engineers and procurement reviewers. It turns a search query into an auditable route decision instead of a loose model preference.

Implementation Pattern

The implementation pattern keeps credentials as placeholders, pins the AIWave base URL, records the model, and leaves room for route-specific controls. Production applications should move credentials into environment or secret storage.

from dataclasses import dataclass
from openai import OpenAI

client = OpenAI(api_key="YOUR_API_KEY_HERE", base_url="https://aiwave.live/v1")

@dataclass
class Trial: model: str; task: str; passed: bool; tokens: int; finish: str

def evaluate(model: str, task: str, prompt: str) -> Trial:
    response = client.chat.completions.create(
        model=model, messages=[{"role": "user", "content": prompt}],
        temperature=0.0, max_tokens=700,
    )
    choice = response.choices[0]
    return Trial(model, task, choice.finish_reason == "stop",
                 response.usage.total_tokens, choice.finish_reason)

Write the Decision Record First

For a DeepSeek, GLM, and Kimi evaluation matrix, record the workload, approved model IDs, endpoint, source date, output cap, retry ceiling, data class, budget owner, and fallback before traffic moves. A short decision record makes a later model or price change reviewable instead of surprising.

Keep Provider and Gateway Evidence Separate

Provider documentation describes direct-platform capabilities and billing concepts. AIWave endpoints describe the gateway's current route rows, public USD snapshot, supported endpoint type, and key-group context. A route row is not a promise that every provider feature is exposed or that a provider policy transfers automatically to the gateway.

Use a Representative Acceptance Set

Test one ordinary request, one long-context request, one malformed request, and one stop-condition request. Store model, finish reason, token usage, retry count, request identifier, checked source date, and reviewer decision. Do not promote a route because a single demo looked good.

Budget Input, Cache, and Output Independently

A single token total hides the cause of a bill. Store input tokens, cached input when exposed, output tokens, tool calls, and retries in separate fields. A route's output cap should follow the task class rather than a global default inherited by every feature.

Bound Retries and Fallbacks

Retry only errors that are safe to retry, use an attempt ceiling, and preserve the request ID. A fallback must be selected by policy, not by an unbounded loop. The receipt should show the original route, fallback route, stop reason, and whether the output was accepted or revised.

Use AIWave's Evidence Layer

Use the Models docs, dated Pricing JSON, live route pricing, and Status. Read the live route table before a rollout, the dated pricing JSON before a budget review, the status page before a launch window, and the trust page before procurement review. Keep the checked dates in the internal decision record.

Final Promotion Checklist

Promotion is ready when provider sources are dated, AIWave routes are rechecked, public USD rows carry their own checked dates, canaries pass, output and retry limits exist, privacy handling is documented, and a reviewer can reconcile the receipt without seeing a prompt or reusable key.