Guides · Streaming

Streaming.

Every chat protocol streams over Server-Sent Events — set stream: true and consume tokens as they generate. Each dialect keeps its native event shape, so official SDKs work unchanged.

One flag, three dialects

Endpoint	Stream format
`/v1/chat/completions`	`chat.completion.chunk` objects, terminated by `data: [DONE]`.
`/v1/messages`	Anthropic events: `message_start` → `content_block_delta`* → `message_stop`.
`/v1/responses`	Typed events like `response.output_text.delta`, closed by `response.completed`.
`/v1/completions`	Legacy `text_completion` chunks, terminated by `data: [DONE]`.

curl -N https://api.layerx1.in/v1/chat/completions \
  -H "authorization: Bearer $LAYERX1_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model":"lx1-gpt-oss-120b","stream":true,"messages":[{"role":"user","content":"Hi"}]}'

Raw SSE with curl (-N disables buffering)

SDK examples

stream = client.chat.completions.create(
    model="lx1-gpt-oss-120b",
    messages=[{"role": "user", "content": "Write a haiku."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Official SDKs consume the streams unchanged

Usage on streams

On Chat Completions, request the final usage totals with stream_options:

{ "stream": true, "stream_options": { "include_usage": true } }

The last chunk before [DONE] then carries usage. On Messages, usage arrives natively in message_delta. Set { "include_usage": false } if a strict client chokes on the extra chunk.

Failure semantics

A stream that cannot complete ends with a protocol-shaped error event — never a silent empty 200. Your client always learns the turn failed.
Treat a mid-stream error like a 5xx: retry the whole request with backoff (Errors).
Long gaps between tokens are normal on reasoning models — they spend time thinking before emitting text. Don't set aggressive idle timeouts on the read side.

Note

Streaming and non-streaming requests draw from the same plan meter and rate limits — there is no reason not to stream interactive traffic.