Confidence Threshold

A confidence threshold is the point at which a system stops acting on its own output and does something else instead: escalate to a person, fall back to a simpler path, ask a clarifying question, or decline. It converts a continuous signal about how sure the system is into a discrete operational decision, which is what an automated workflow actually needs. The difficulty is that language models do not naturally emit a trustworthy number. Fluency is not confidence, and a model asked to rate its own certainty will often produce a plausible-sounding figure with little relationship to whether the answer is right. Token probabilities carry more information but measure the likelihood of the wording rather than the truth of the claim. Practical systems therefore build the signal from something more concrete: agreement between repeated samples, whether a retrieval step found supporting material, whether a structured output validated, whether a second check agreed with the first, or a small classifier trained on past correct and incorrect answers. Wherever the signal comes from, calibration is what makes a threshold meaningful. A calibrated score means that outputs scored around a given level are right about that often, and only then does choosing a cut-off become a business decision rather than a guess. Calibration is measured, not assumed, by scoring a labelled set and comparing predicted confidence against observed accuracy. An uncalibrated score can still rank cases usefully, but the specific number where you draw the line has no interpretation. Choosing the level is a trade between two costs that are rarely equal. Below the line, work goes to a human, which costs time and caps throughput. Above it, mistakes reach the customer, which costs trust and sometimes money. Set the threshold from those costs rather than from a round number: a system that files documents can tolerate a lower bar than one that issues refunds, and the same product often needs different thresholds per action. What happens below the threshold matters as much as the threshold itself. Escalation only works if there is somewhere to escalate to, with capacity, context and a return path for the answer. A queue that nobody staffs turns a careful design into a slower failure, and a handoff that arrives without the conversation, the retrieved documents and the reason for the escalation makes the human slower than if they had started fresh. Thresholds drift. The distribution of inputs changes, a model version changes, a prompt is edited, and the same cut-off now selects a different share of traffic. Monitor the escalation rate as a first-class metric, because a sudden drop usually means confidence rose without accuracy following, and that is the failure mode where the threshold quietly stops protecting anything.