Building a Multi-Model AI Router in Python

Jul 26, 2026

Most applications send every request to the same model. A simple classifier that routes easy requests to cheap models and hard requests to capable ones can cut costs by 50-70% without measurable quality loss.

OpenRouter does this as a service. You can build the same pattern yourself with any OpenAI-compatible API — including AIWave, which provides all the models below through one endpoint.

The Routing Strategy

The idea: classify each request by estimated complexity, then route to the appropriate model.

TierModelUse forCost (vs GPT-4o)
1 (Cheap)DeepSeek V4 FlashSimple queries, summaries, classification94% cheaper
2 (Mid)GLM-5Chinese content, moderate reasoning94% cheaper
3 (Premium)DeepSeek V4 ProComplex coding, multi-step reasoning83% cheaper

Implementation

from openai import OpenAI

client = OpenAI(api_key="***", base_url="https://aiwave.live/v1")

def classify_complexity(prompt: str) -> str:
    """Route based on simple heuristics."""
    indicators = [
        "debug", "fix", "refactor", "optimize",
        "complex", "architecture", "multi-step",
        "analyze", "compare", "benchmark"
    ]
    if len(prompt) > 3000:
        return "premium"
    if any(w in prompt.lower() for w in indicators):
        return "premium"
    if len(prompt) > 1000:
        return "mid"
    return "cheap"

MODELS = {
    "cheap": "deepseek-chat",
    "mid": "glm-4-plus",
    "premium": "deepseek-reasoner"
}

def route_chat(prompt: str) -> str:
    tier = classify_complexity(prompt)
    response = client.chat.completions.create(
        model=MODELS[tier],
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

Cost Impact

A typical app: 10,000 API calls/month.

  • 70% simple (Tier 1): 7,000 calls × DeepSeek V4 Flash
  • 20% moderate (Tier 2): 2,000 calls × GLM-5
  • 10% complex (Tier 3): 1,000 calls × DeepSeek V4 Pro

    • vs all on GPT-4o ($2.50/$10.00):

    ApproachMonthly Cost
    All GPT-4o$400+
    All DeepSeek V4 Pro$33
    Routed (3-tier)$19

    The 3-tier router saves roughly 95% compared to GPT-4o. Even compared to using DeepSeek V4 Pro for everything, routing saves 42%.

    Production Considerations

  • Latency classification: For real-time apps, use token count or prompt length instead of keyword matching
  • Fallback: If the cheap model fails or times out, escalate to the next tier
  • Monitoring: Track which tier handles what percentage of requests — adjust thresholds over time
  • A/B testing: Compare routed vs single-model quality with a sample of requests

    • All models above are available through AIWave with a single API key and PayPal payments. The $1 free credit covers thousands of tier-1 requests to test the pattern before committing.

    Compare routing approaches? Check OpenRouter for a hosted solution, or build your own with AIWave models.

    Related Articles