AIWave API

LangChain with Chinese AI Models: DeepSeek, GLM and Kimi

Using LangChain with Chinese AI models takes one line of configuration, not a custom integration. Because DeepSeek, GLM, Kimi, Qwen and ERNIE are all served here behind an OpenAI-compatible endpoint, LangChain’s existing ChatOpenAI class talks to them directly — you point base_url at the gateway and change the model string. This guide covers the setup, the four places it usually breaks, streaming, tool calling, RAG wiring, and the cost maths that makes the swap worth doing in the first place.

Get an API key →Compare live pricing →

Why LangChain does not need a dedicated Chinese model provider

LangChain ships provider packages for dozens of vendors, and newcomers assume they need one per Chinese model. They do not. LangChain’s ChatOpenAI integration accepts an arbitrary base_url. Any endpoint that speaks the OpenAI Chat Completions wire format is therefore a first-class LangChain chat model, including every model in the AIWave directory.

That has a practical consequence worth stating plainly: chains, agents, retrievers, output parsers, callbacks and LangSmith tracing all keep working. You are not adopting a fork or a community package with its own release cadence — you are using the same class you already use, pointed somewhere cheaper.

Step 1: Install and configure

You need langchain-openai. Nothing Chinese-model-specific is required.

pip install langchain langchain-openai

Then construct the chat model. The only two lines that differ from a stock OpenAI setup are base_url and model:

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="deepseek-v4-pro",
    base_url="https://aiwave.live/v1",       # ← the whole integration
    api_key=os.environ["AIWAVE_API_KEY"],
    temperature=0.2,
)

print(llm.invoke("Explain mixture-of-experts routing in two sentences.").content)

Get a key from the console; email or GitHub signup is enough, with no Chinese phone number involved. Full endpoint reference lives in the API documentation, and authentication covers key handling in more depth.

Step 2: Pick a model deliberately

The whole point of a gateway is that model choice becomes a runtime decision rather than an architectural one. These are current rates, straight from the pricing endpoint:

A representative slice of what LangChain can address through one key:

Model IDInput / 1M tokensOutput / 1M tokens
deepseek-v4-flash$0.206$0.412
deepseek-v4-pro$1.09$2.17
glm-4.7-flashfreefree
glm-5$1.55$4.96
kimi-k2.5$0.66$3.30
qwen3-coder-480b-a35b-instruct$0.12$0.36
ernie-4.0-turbo-8k$0.0012$0.0012

Rates read from the AIWave pricing endpoint on 2026-07-26. Check live pricing before budgeting — providers revise rates.

A useful default for chains: run development and evaluation loops on a cheap fast model, then promote to a larger one only where output quality actually gates the product. Because the model name is a string, that promotion is a config change, not a refactor.

Step 3: Build a chain with LCEL

LangChain Expression Language works identically. Here is a summarisation chain with a structured prompt and a string output parser:

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a technical editor. Be concise and concrete."),
    ("human", "Summarise the following in exactly three bullet points:\n\n{text}"),
])

chain = prompt | llm | StrOutputParser()

print(chain.invoke({"text": open("release-notes.md").read()}))

Swap deepseek-v4-pro for deepseek-v4-flash and the same chain runs at a fraction of the cost. Nothing else in the code changes.

Step 4: Stream tokens to the user

Streaming is where perceived latency is won or lost. LangChain’s .stream() works because the gateway emits the same server-sent-event chunks OpenAI does — the format is documented in streaming responses.

for chunk in chain.stream({"text": "..."}):
    print(chunk, end="", flush=True)

Async, for a web backend:

import asyncio
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="deepseek-v4-flash",
                 base_url="https://aiwave.live/v1",
                 api_key=os.environ["AIWAVE_API_KEY"])

async def main():
    async for chunk in llm.astream("Write a haiku about cold starts."):
        print(chunk.content, end="", flush=True)

asyncio.run(main())

Step 5: Tool calling and agents

LangChain’s bind_tools emits OpenAI-format tool definitions, which models that implement tool calling upstream accept directly.

from langchain_core.tools import tool

@tool
def get_exchange_rate(base: str, quote: str) -> str:
    """Return the exchange rate between two currency codes."""
    return f"1 {base} = 7.3 {quote}"          # replace with a real lookup

agent_llm = llm.bind_tools([get_exchange_rate])
msg = agent_llm.invoke("What is 1 USD in CNY?")
print(msg.tool_calls)

Two cautions. First, tool-calling support varies by model — it is a property of the upstream model, not of the gateway, so verify on the model page before you depend on it. Second, agents multiply token consumption: every reasoning step is another round trip. Our write-up on AI agent cost analysis works through why agent workloads are where cheap models pay for themselves fastest. For the protocol-level view, see function calling with Chinese AI models.

Step 6: Retrieval-augmented generation

RAG is the most common reason teams reach for LangChain, and it is also where token costs concentrate — every query drags retrieved context into the prompt. A long-context model reduces how much chunking gymnastics you need.

from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings
from langchain_core.runnables import RunnablePassthrough

embeddings = OpenAIEmbeddings(
    model="text-embedding-3-small",           # any embeddings backend works
)

store = FAISS.from_texts(docs, embeddings)
retriever = store.as_retriever(search_kwargs={"k": 6})

rag_prompt = ChatPromptTemplate.from_template(
    "Answer using only the context below.\n\nContext:\n{context}\n\nQuestion: {question}"
)

rag = (
    {"context": retriever, "question": RunnablePassthrough()}
    | rag_prompt
    | llm
    | StrOutputParser()
)

print(rag.invoke("What changed in the v4 release?"))

For the end-to-end version including chunking strategy and evaluation, see building a production RAG pipeline with Chinese AI models. The embeddings endpoint documents the vector side.

Step 7: Handle failures like production code

LangChain has retry and fallback primitives; use them. A single upstream provider having a bad afternoon should not take your application down, and since every model here accepts an identical request body, a fallback is genuinely one line.

primary  = ChatOpenAI(model="deepseek-v4-pro",   base_url="https://aiwave.live/v1",
                      api_key=os.environ["AIWAVE_API_KEY"], max_retries=3)
fallback = ChatOpenAI(model="glm-5",             base_url="https://aiwave.live/v1",
                      api_key=os.environ["AIWAVE_API_KEY"])

resilient = primary.with_fallbacks([fallback])
print(resilient.invoke("Summarise this incident report.").content)

Which status codes deserve a retry and which do not is covered in error codes; rate limits has a working backoff implementation.

The four things that actually go wrong

Passing the base URL without /v1

LangChain appends the path but not the version prefix. https://aiwave.live produces 404s; https://aiwave.live/v1 is correct.

Hard-coding a model ID that gets versioned

Model IDs change as providers ship updates. Read the list at startup rather than trusting a constant you wrote six months ago:

import httpx
ids = {m["id"] for m in httpx.get(
    "https://aiwave.live/v1/models",
    headers={"Authorization": f"Bearer {os.environ['AIWAVE_API_KEY']}"},
).json()["data"]}
assert "deepseek-v4-pro" in ids, "configured model is no longer served"

Assuming every model supports every parameter

JSON mode, tool calling and vision are upstream capabilities. If a model ignores response_format, that is the provider, not LangChain. Check the model directory first.

Forgetting that streaming hides token usage

Streamed responses do not carry a usage object per chunk. If you need exact accounting, measure with a non-streaming call or reconcile in the console.

What this actually saves

Take a modest production chain: 20M input tokens and 4M output tokens a month, roughly one busy internal tool. At the rates in the table above, deepseek-v4-flash costs about $5.77 a month (20 × $0.206 + 4 × $0.412), and qwen3-coder-480b-a35b-instruct about $3.84. Those figures are arithmetic on the published rates, not a benchmark claim — run your own volumes through the same multiplication.

The strategic point is not any single number. It is that a LangChain application whose model is a configuration string can chase price and capability changes without an engineering project. That is the whole argument for putting a gateway between your chain and the model.

Structured output without a JSON-mode guarantee

Chains that feed downstream code need parseable output. LangChain offers with_structured_output(), which prefers native JSON mode when the model supports it and falls back to prompt-plus-parser when it does not. That fallback matters here, because JSON mode is an upstream capability that varies across DeepSeek, GLM, Kimi and ERNIE.

from pydantic import BaseModel, Field

class Ticket(BaseModel):
    severity: str = Field(description="one of: low, medium, high, critical")
    component: str
    summary: str

extractor = llm.with_structured_output(Ticket)
result = extractor.invoke("Checkout 500s since the 14:20 deploy, payments are down.")
print(result.severity, result.component)

If a model returns prose around the JSON, wrap the parser defensively rather than trusting it:

from langchain_core.output_parsers import JsonOutputParser
from langchain_core.prompts import ChatPromptTemplate

parser = JsonOutputParser(pydantic_object=Ticket)
prompt = ChatPromptTemplate.from_template(
    "{format_instructions}\n\nClassify this incident:\n{text}"
).partial(format_instructions=parser.get_format_instructions())

chain = prompt | llm | parser

A practical rule: validate the parsed object, and on a validation failure retry once at a lower temperature before escalating to a larger model. Retrying blindly against the same settings usually reproduces the same malformed output.

Conversation memory and message history

Multi-turn chains re-send the whole history on every call, so history length is a direct cost driver. RunnableWithMessageHistory is the current mechanism:

from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory

store = {}
def history_for(session_id: str):
    return store.setdefault(session_id, InMemoryChatMessageHistory())

conversational = RunnableWithMessageHistory(
    chain, history_for,
    input_messages_key="text",
    history_messages_key="history",
)

conversational.invoke({"text": "Hello"},
                      config={"configurable": {"session_id": "user-42"}})

Two cost controls are worth building in from the start. Trim history to a token budget rather than a message count, and summarise older turns instead of dropping them outright, so the model keeps context without paying for the full transcript every turn. On a long-running assistant this is frequently a larger saving than switching models.

LangGraph for anything with branching

Once a workflow has conditionals, retries or human review, LCEL pipes stop being the right shape and LangGraph takes over. It uses the same chat model objects, so nothing about the gateway configuration changes:

from langgraph.graph import StateGraph, END
from typing import TypedDict

class S(TypedDict):
    question: str
    draft: str
    approved: bool

def draft(state: S):
    return {"draft": llm.invoke(state["question"]).content}

def review(state: S):
    verdict = llm.invoke(
        f"Answer only YES or NO. Is this factual and complete?\n\n{state['draft']}"
    ).content
    return {"approved": verdict.strip().upper().startswith("YES")}

g = StateGraph(S)
g.add_node("draft", draft)
g.add_node("review", review)
g.set_entry_point("draft")
g.add_edge("draft", "review")
g.add_conditional_edges("review", lambda s: END if s["approved"] else "draft")
app = g.compile()

This pattern — a cheap model drafting and a stronger model reviewing — is where a multi-model gateway earns its keep. Assign deepseek-v4-flash to draft and deepseek-v4-pro to review, and you pay premium rates only on the short verification step. The same idea is developed further in how a multi-model router is built.

Evaluating a model swap before you ship it

Do not migrate on vibes. Build a small evaluation set from real traffic — fifty to a hundred representative inputs is usually enough to expose a regression — and run both models over it.

import json, statistics

cases = [json.loads(l) for l in open("eval.jsonl")]

def score(model_name):
    m = ChatOpenAI(model=model_name, base_url="https://aiwave.live/v1",
                   api_key=os.environ["AIWAVE_API_KEY"], temperature=0)
    judge = ChatOpenAI(model="deepseek-v4-pro", base_url="https://aiwave.live/v1",
                       api_key=os.environ["AIWAVE_API_KEY"], temperature=0)
    out = []
    for c in cases:
        answer = m.invoke(c["input"]).content
        verdict = judge.invoke(
            f"Reference:\n{c['expected']}\n\nCandidate:\n{answer}\n\n"
            "Reply with a single integer 1-5 for factual agreement."
        ).content
        out.append(int(re.search(r"\d", verdict).group()))
    return statistics.mean(out)

for name in ["deepseek-v4-flash", "deepseek-v4-pro", "glm-5"]:
    print(name, round(score(name), 2))

Model-as-judge is imperfect and biased toward its own style, so treat the numbers as a screening signal rather than proof. Its real value is catching the case where a cheaper model is obviously worse on your data — which is exactly the failure a price-driven migration risks.

Production checklist

Migrating an existing LangChain project

If you already have a LangChain application pointed at OpenAI, the migration is narrower than most teams expect. The chat model constructor changes; the chains, retrievers, parsers and tests do not.

# before
llm = ChatOpenAI(model="gpt-4o", api_key=os.environ["OPENAI_API_KEY"])

# after
llm = ChatOpenAI(
    model="deepseek-v4-pro",
    base_url="https://aiwave.live/v1",
    api_key=os.environ["AIWAVE_API_KEY"],
)

Run your evaluation set, watch the failure modes rather than the averages, and keep the old configuration behind an environment variable so a rollback is one deploy. The step-by-step version, including the parts that are not LangChain-specific, is in migrating an OpenAI app to Chinese models.

Frequently asked questions

Does LangChain support DeepSeek natively?

You do not need native support. ChatOpenAI with a custom base_url reaches DeepSeek, GLM, Kimi, Qwen and ERNIE through the same OpenAI-compatible interface, and every LangChain feature built on chat models continues to work.

Can I use LangSmith tracing with these models?

Yes. Tracing hooks into LangChain’s callback system, which is independent of which endpoint the chat model calls.

Do I need a Chinese phone number?

No. Accounts are created with email or GitHub, and billing is in USD. That constraint is the reason this gateway exists; see accessing Chinese AI models without a Chinese phone number.

Which model should I start with?

Start on a cheap fast model such as deepseek-v4-flash while you iterate on prompts, then compare quality against a larger model on your own evaluation set before committing. The pricing page has current rates for every option.

Get an API key →Compare live pricing →

Sources

Every claim above traces back to one of these. Pricing figures come from the AIWave pricing endpoint on 2026-07-26; capability claims come from the vendors’ own documentation.

Related Articles