Streaming LLM Responses with Chinese Models: SSE in Python & TypeScript

July 20, 2026 · 14 min read · Pricing

When a user asks an AI model a question and waits 8 seconds for the complete response, it feels slow. But when the first token appears in 200ms and text flows in gradually, it feels instant. That's the power of streaming.

This tutorial shows you how to implement Server-Sent Events (SSE) streaming with Chinese AI models using the same OpenAI SDK you already know. We'll build a Python backend and a React frontend.

Why Streaming Matters for Chinese Models

Chinese models like DeepSeek V4 Pro are hosted in Singapore via AIWave, giving low latency to APAC users. Streaming over SSE adds perceived responsiveness on top of that. The combination means your users in Tokyo, Seoul, or Singapore get responses that start in ~200ms.

Python Backend: FastAPI + SSE

"""Streaming LLM responses via SSE with Chinese AI models"""
import os
import json
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import openai

app = FastAPI()
client = openai.OpenAI(
    api_key=os.environ["AIWAVE_API_KEY"],
    base_url="https://aiwave.live/v1",
)

@app.get("/chat/stream")
async def stream_chat(query: str, model: str = "deepseek-v4-pro"):
    """SSE endpoint for streaming LLM responses."""
    
    def generate():
        stream = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": query}],
            temperature=0.7,
            max_tokens=2048,
            stream=True,
        )
        
        for chunk in stream:
            delta = chunk.choices[0].delta
            if delta.content:
                # SSE format: "data: {json}\n\n"
                data = json.dumps({"type": "content", "text": delta.content})
                yield f"data: {data}\n\n"
            
            # Send token usage when stream ends
            if chunk.choices[0].finish_reason:
                usage = chunk.usage
                data = json.dumps({
                    "type": "done",
                    "tokens": {
                        "input": usage.prompt_tokens,
                        "output": usage.completion_tokens,
                    }
                })
                yield f"data: {data}\n\n"
    
    return StreamingResponse(
        generate(),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "Connection": "keep-alive",
            "X-Accel-Buffering": "no",  # Disable nginx buffering
        },
    )

React Frontend: useChat Hook

import { useState, useEffect, useRef } from 'react';

function useChatStream() {
  const [messages, setMessages] = useState([]);
  const [isStreaming, setIsStreaming] = useState(false);
  const abortRef = useRef(null);

  const sendMessage = async (text) => {
    setIsStreaming(true);
    const userMsg = { role: 'user', content: text };
    const assistantMsg = { role: 'assistant', content: '' };
    setMessages(prev => [...prev, userMsg, assistantMsg]);

    // Create AbortController for cancellation
    abortRef.current = new AbortController();

    try {
      const response = await fetch(
        `/chat/stream?query=${encodeURIComponent(text)}&model=deepseek-v4-pro`,
        { signal: abortRef.current.signal }
      );

      const reader = response.body.getReader();
      const decoder = new TextDecoder();

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;

        const chunk = decoder.decode(value);
        // Parse SSE data lines
        const lines = chunk.split('\n').filter(l => l.startsWith('data: '));

        for (const line of lines) {
          const data = JSON.parse(line.slice(6));
          if (data.type === 'content') {
            setMessages(prev => {
              const updated = [...prev];
              updated[updated.length - 1] = {
                role: 'assistant',
                content: updated[updated.length - 1].content + data.text,
              };
              return updated;
            });
          }
        }
      }
    } catch (e) {
      if (e.name !== 'AbortError') console.error(e);
    } finally {
      setIsStreaming(false);
    }
  };

  const stopStreaming = () => abortRef.current?.abort();

  return { messages, isStreaming, sendMessage, stopStreaming };
}

Key Differences: DeepSeek vs OpenAI Streaming

Feature	OpenAI	AIWave (DeepSeek V4 Pro)
SDK	openai Python/JS	Same SDK
Base URL	api.openai.com/v1	aiwave.live/v1
Stream format	SSE (standard)	SSE (standard)
First token latency	300-500ms	200-400ms
Input price	$2.50/M	$0.48/M
Output price	$10.00/M	$3.60/M

The streaming implementation is byte-for-byte identical between OpenAI and AIWave. The only change is base_url. This is by design — OpenAI-compatible means truly compatible.

Production Tips

Disable Response Buffering

Nginx and Cloudflare buffer responses by default, which defeats streaming. In nginx:

location /chat/stream {
    proxy_pass http://127.0.0.1:8000;
    proxy_http_version 1.1;
    proxy_set_header Connection '';
    proxy_buffering off;          # Critical for SSE
    proxy_cache off;
    chunked_transfer_encoding on;
}

Timeout Handling

stream = client.chat.completions.create(
    model="deepseek-v4-pro",
    messages=messages,
    stream=True,
    timeout=60,  # Total timeout for entire stream
)

for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        yield f"data: {json.dumps({'text': content})}\n\n"

Model Fallback for Streaming

MODELS = ["deepseek-v4-pro", "deepseek-v4-flash", "glm-5-turbo"]

async def stream_with_fallback(query: str):
    for model in MODELS:
        try:
            stream = client.chat.completions.create(
                model=model, messages=[{"role": "user", "content": query}],
                stream=True, timeout=30,
            )
            async for chunk in stream:
                if chunk.choices[0].delta.content:
                    yield chunk.choices[0].delta.content
            return  # Success — don't try other models
        except Exception:
            continue
    yield "All models unavailable."