Stop Sequence

A stop sequence is a string that ends generation the moment the model produces it. You pass one or more of them with the request; when the decoder emits a matching span it halts, and the stop text itself is normally excluded from the returned completion. It exists because a model has no reliable sense of when a caller considers the answer finished — left alone it may keep going into an explanation, a second example, or an imagined next turn — and because tokens generated after the useful part still cost money and latency. Typical uses are narrow and mechanical. In a classification or extraction call, a newline as the stop sequence guarantees a single-line answer. In a few-shot prompt built from repeated blocks, the block delimiter is the natural stop, since the model's habit of continuing the pattern is exactly what you want to interrupt. In a chat-style completion, the marker that begins a user turn stops the model from writing both halves of the conversation. Two cautions. A stop sequence that can legitimately occur inside the content will truncate a valid answer — a bare newline is a bad stop for anything multi-line, and a closing brace is a bad stop for nested JSON. And because the model was cut off rather than finished, always check the response's finish reason: treating a truncated completion as a complete one is a common source of silently malformed downstream data.