Streaming Generation

Streaming generation delivers a model's output token by token as it's produced, instead of waiting for the full response and returning it in one block. The user sees words appear almost immediately, which is why chat interfaces feel responsive even when the complete answer takes several seconds. Technically, the API returns a stream (usually server-sent events) that your app reads incrementally and renders as it arrives. For builders, streaming is mostly a UX and perceived-latency win: time-to-first-token matters more than total time for how fast a product feels, and it lets you show progress, stop early, or start post-processing before generation finishes. Practical notes: streamed responses complicate anything that needs the whole output at once — JSON parsing, moderation, token counting — so you often buffer and validate at the end anyway. Handle disconnects and partial output gracefully, and remember that streaming doesn't reduce cost or total compute; it only changes when bytes reach the user.