Few-Shot Prompting

Few-shot prompting is a technique where the prompt includes a small number (typically 2-10) of worked examples — demonstrations of the exact input format and desired output format — before presenting the model with the real task. This gives the model a concrete pattern to imitate rather than relying solely on its interpretation of a natural-language instruction, and it is one of the single most effective and reliable ways to improve output consistency in production LLM applications. Few-shot prompting works because of in-context learning: transformer-based models can infer a task's rules from examples given at inference time, without any weight updates or fine-tuning, essentially "learning" the pattern fresh within that one prompt. For SaaS builders, few-shot prompting is the go-to fix when zero-shot prompting produces outputs that are technically reasonable but inconsistent in format, tone, or edge-case handling — a common symptom is the model sometimes returning a paragraph, sometimes a bulleted list, or sometimes including a preamble like "Sure, here's the summary:" that breaks a downstream parser. Adding 3-5 examples showing the exact desired format usually eliminates this variance. The trade-off is token cost: each example consumes context window space and adds to the API bill on every single call, so teams balance the incremental accuracy gain against the added latency and cost, and often compress examples to only the essential fields. Example selection also matters — examples should cover edge cases and the range of expected inputs, not just the easy cases, and should be kept consistent in format with the actual task input. Concrete worked example: a prompt for extracting structured data from customer emails: "Extract the order number and issue type from each email. Follow the exact format shown.\n\nEmail: \"Hi, order #4471 arrived damaged, please help.\"\nOutput: {\"order_id\": \"4471\", \"issue\": \"damaged\"}\n\nEmail: \"Where is my order 8823? It's been 2 weeks.\"\nOutput: {\"order_id\": \"8823\", \"issue\": \"delayed\"}\n\nEmail: \"Can I get order #1290 in a different color?\"\nOutput:" — the model, having seen the pattern twice, reliably returns {\"order_id\": \"1290\", \"issue\": \"change_request\"} in the same strict JSON shape, even though "change_request" never appeared in the examples — demonstrating the model generalizing the pattern rather than memorizing it. A related refinement production teams adopt is dynamic few-shot selection: rather than hardcoding the same static examples into every prompt, the application retrieves the 3-5 most similar historical examples to the current input (via embedding similarity search) and inserts those instead, so the model always sees demonstrations closely matched to the case at hand — this hybrid of few-shot prompting and retrieval typically outperforms a fixed, generic example set, especially on tasks with a wide variety of input types.