Token

A token is the smallest unit of text an LLM's tokenizer breaks input and output into for processing — not quite a character, not quite a word, but a statistically chosen chunk that might be a whole common word ("the"), part of a longer word ("token" → "tok" + "en"), or a single punctuation mark. In English, a useful rule of thumb is that 100 tokens ≈ 75 words, though this varies by language (non-English languages, especially those using non-Latin scripts, often tokenize far less efficiently, meaning the same sentence costs more tokens). Tokens matter enormously to SaaS builders because they are the literal unit of everything that costs money and time in LLM applications: API pricing is quoted per million input/output tokens (e.g., Claude Sonnet 4.5 at roughly $3/$15 per million input/output tokens), the context window is measured in tokens (not words or characters), and generation latency scales with output token count since models generate one token at a time. A concrete example: the sentence "Please summarize this document in three bullet points." tokenizes to roughly 11 tokens. If a builder sends a 10,000-word document (~13,000 tokens) plus a 50-token instruction and gets back a 100-token summary, the API call costs roughly 13,050 input tokens and 100 output tokens — and because output tokens are typically priced 4-5x higher than input tokens, verbose responses are disproportionately expensive. This is why cost-conscious builders instruct models to be concise, cache repeated context (prompt caching), and choose smaller/cheaper models for high-volume, low-complexity tasks. Understanding tokenization also explains odd LLM behaviors, like models historically struggling to count letters in a word ("how many r's in strawberry") — because the model sees tokens, not individual characters. Builders should also know that tokens are consumed on both sides of a request — every token in your prompt (system instructions, conversation history, retrieved RAG context) counts as input tokens, and every token the model generates counts as output tokens, with most providers pricing output tokens several times higher than input tokens because generation is more computationally expensive per token than reading. This asymmetry is why instructing a model to "be concise" or capping `max_tokens` in the API call is a genuine cost-control lever, not just a UX preference, and why a chat feature that lets conversation history grow unbounded (resending the entire history as input tokens on every turn) can quietly become one of the most expensive parts of a SaaS product's AI cost structure as conversations get longer.