Model Router

A model router is a layer that decides which model should handle each request, instead of sending everything to one model. Routing can be rule-based (short classification goes to a cheap model; a long code task goes to a reasoning model), driven by a small classifier that predicts difficulty, or steered by the prompt itself. The goal is to hit a target quality at the lowest cost and latency: most requests are easy and go to a fast, cheap model, while the hard minority get escalated to a powerful, expensive one. For SaaS builders, a router is one of the highest-leverage cost optimizations available — teams routinely cut LLM spend by half or more without users noticing, because they stop paying frontier-model prices for trivial calls. Practical concerns: you need evals proving the cheap model is actually good enough on the routed traffic, a fallback path when it fails, and monitoring so drift in your traffic mix doesn't quietly degrade quality. Several managed routing services and open-source libraries now do this for you.