Server-Sent Events vs WebSockets for Streaming AI Responses

A user clicks "Send" on a chat interface and expects words to appear within a few hundred milliseconds. Under the hood, that experience is a streaming protocol pushing tokens from a model down to a browser. Two protocols keep coming up: Server-Sent Events and WebSockets. They look similar from the outside, but the wire format, the connection model, and the failure modes are different enough that the choice will shape your backend, your proxies, and your observability for years.

What each protocol actually does

Server-Sent Events (SSE)

SSE is a unidirectional channel over a single HTTP request. The client opens an EventSource connection; the server keeps the response open and writes data: lines separated by double newlines. The browser parses them automatically. There is no client-to-server messaging on the same connection. If you need to send the prompt, you do it over a separate POST.

WebSockets

WebSockets start as an HTTP request, then upgrade to a full-duplex binary frame protocol over a single TCP connection. After the handshake, either side can send text or binary frames at any time. The protocol is message-oriented, not stream-of-bytes.

The distinction matters because AI chat is almost a perfect fit for SSE: the server produces a stream of tokens, the client renders them, and the client only needs to send one message per turn. WebSockets add a capability you usually do not need, at a cost you usually do want to avoid.

How an LLM token stream maps onto each protocol

SSE payload shape

With SSE, each token (or token batch) becomes one event:

data: {"token": "Hello"}\n\n
data: {"token": ", "}n\n\n
data: {"token": "world"}\n\n
data: [DONE]\n\n

Client code is short:

const res = await fetch('/api/chat', {
  method: 'POST',
  body: JSON.stringify({ prompt }),
});
const reader = res.body.getReader();
const decoder = new new TextDecoder();
let buf = '';
while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  buf += decoder.decode(value, { stream: true });
  const parts = buf.split('\\n\\n');
  buf = parts.pop();
  for (const part of parts) {
    if (part.startsWith('data: ')) {
      const payload = part.slice(6);
      if (payload === '[DONE]') return;
      appendToken(JSON.parse(payload).token);
    }
  }
}

WebSocket payload shape

With WebSockets, you frame messages yourself. Most teams wrap each chunk in JSON:

const ws = new WebSocket('wss://api.example.com/chat');
ws.onopen = () => ws.send(JSON.stringify({ prompt }));
ws.onmessage = (evt) => {
  const msg = JSON.parse(evt.data);
  if (msg.done) ws.close();
  else appendToken(msg.token);
};

Both work. The WebSocket version folds the prompt and the response into one connection, which feels elegant, but you can get the same effect with SSE plus a POST and you keep the benefits of plain HTTP for the request side.

The trade-offs that decide in production

Connection overhead

A WebSocket handshake is roughly 2 to 4 KB of HTTP traffic plus a cryptographic upgrade. An SSE connection is just an HTTP response with Content-Type: text/event-stream. For a single chat turn, this is negligible. For workloads that open thousands of streams per minute (multi-tenant SaaS, IDE plugins, background agents), the difference adds up: WebSockets keep idle workers alive, SSE responses can be served by the same workers that already handle your regular API.

Reverse proxies and load balancers

This is where SSE quietly wins. Nginx, HAProxy, AWS ALB, Cloudflare, and most corporate proxies understand long-lived HTTP responses. They buffer or chunk them, they apply timeouts, they let you set headers. WebSockets require explicit upgrade handling, custom timeouts (often 60s by default), and dedicated routing rules. If your platform team has ever told you "we do not allow WebSockets in this VPC," SSE sidesteps the conversation entirely.

Reconnection and resumption

EventSource ships with automatic reconnect and a Last-Event-ID header. You get retries with exponential backoff, and you get a built-in cursor for resuming a stream after a dropped connection. WebSockets have none of this. You write the retry loop, you store the offset yourself, and you decide what "resume" means for your protocol. For long generations (a 4,000-token essay can take 30 seconds on a slow model), SSE's resumability is a genuine reliability feature, not a nice-to-have.

Bidirectional needs

If your product genuinely needs server-initiated interrupts mid-generation — "stop generating," "the model needs to call a tool and wait for the result," "cancel this batch" — WebSockets let you do that on the same connection without round-tripping through another request. SSE forces you to send those signals over a separate POST or fetch with AbortController. In practice, most chat UIs do this with a POST /cancel endpoint anyway, so the SSE cost is small. But if you are building an agent loop where the server pings the client with new context every few hundred milliseconds, WebSockets' full duplex is worth it.

Observability

SSE rides on HTTP. Every line shows up in your access logs, your APM traces, and your edge logs as a normal request with a status code. You can grep it, you can sample it, and you can correlate it with the upstream LLM call by trace ID. WebSocket frames are opaque binary blips; you need custom middleware to log them, custom metrics to count them, and custom dashboards to graph message rates per connection. The further your stack leans on standard HTTP tooling, the more painful WebSockets become.

Library support and tooling

Every language has a built-in HTTP client and an SSE parser (often one screen of code). WebSockets need a maintained client library per language and per runtime, and those libraries have a long history of subtle bugs around backpressure, fragmentation, and ping handling. In Node, the built-in WebSocket only landed in v22. In Python, websockets and aiohttp disagree on edge cases. In Go, gorilla/websocket is the de facto choice and is now in maintenance mode.

A quick decision rubric

Requirement	Pick
Server pushes tokens, client sends one prompt per turn	SSE
Need to send interrupts or tool results back on the same stream	WebSockets
Runs behind a strict corporate proxy or CDN	SSE
Need built-in reconnect and Last-Event-ID resumption	SSE
Need to multiplex many independent streams over one connection	WebSockets
Observability stack is built around HTTP access logs	SSE
Many idle streams held open for long periods	SSE
Bidirectional agent loop with frequent server pings	WebSockets

A hybrid that works in practice

A pattern we have seen work well is: SSE for the token stream, plain fetch for the prompt and the cancel signal. The browser opens EventSource after a POST /chat returns a stream ID, and cancels with POST /chat/:id/cancel using AbortController. The server side can use a single HTTP worker per stream, terminate it cleanly when the model finishes, and let the proxy reap dead connections. You get SSE's resumability, plain HTTP in every log line, and no protocol gymnastics.

If you genuinely need full duplex — think multi-agent orchestration where the server streams partial reasoning and waits for tool results — WebSockets are the right tool. Just budget for the proxy config, the custom reconnection logic, and the per-language library maintenance.

Code patterns worth stealing

Server-side SSE with backpressure

import asyncio, json
from fastapi import FastAPI
from fastapi.responses import StreamingResponse

app = FastAPI()

async def token_stream(prompt: str):
    async for tok in llm.stream(prompt):
        yield f"data: {json.dumps({'token': tok})}\\n\\n"
    yield "data: [DONE]\\n\\n"

@app.post("/chat")
async def chat(prompt: str):
    return StreamingResponse(
        token_stream(prompt),
        media_type="text/event-stream",
        headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
    )

The X-Accel-Buffering: no header is the magic line that tells nginx not to buffer the response.

Client-side cancel with SSE

const ctrl = new AbortController();
fetch('/api/chat', { method: 'POST', body, signal: ctrl.signal })
  .then(r => r.body.pipeThrough(new TextDecoderStream()).pipeTo(...));

cancelButton.onclick = () => ctrl.abort();

WebSocket ping/pong for keepalive

async with websockets.connect(uri, ping_interval=20, ping_timeout=20) as ws:
    await ws.send(json.dumps({"prompt": prompt}))
    async for msg in ws:
        handle(msg)

ping_interval is one of those settings you only learn about the hard way: without it, intermediate proxies silently drop the connection after their idle timeout.

The honest summary

For most AI chat, completion, and streaming-response products, SSE is the better default. It survives every proxy you will ever meet, it gives you resumability for free, and it shows up in your logs as an HTTP request. Reach for WebSockets when you have a concrete, load-tested requirement for full-duplex communication on the same connection — agent loops, collaborative editing, multiplayer presence. "It feels cleaner" is not a requirement.

---

If you are building a product that streams AI responses, DocPilot generates the API and streaming docs that match your actual implementation, including SSE examples, cancel endpoints, and the proxy headers that actually matter. Connect a GitHub repo and the docs stay in sync with every PR.