Data Labeling

Data labeling is the work of attaching the correct answer to each example in a dataset so a model can be trained or evaluated against it: marking a ticket as billing or bug, drawing a box around a defect, tagging a transcript with intent, or rating a model's answer as acceptable. It is the least glamorous and most decisive part of most applied machine-learning projects, because a model can be no more consistent than the labels it learned from — noisy labels put a hard ceiling on measurable quality that no amount of model tuning gets past. Three things make labelling succeed or fail. A written guideline that resolves the ambiguous cases, not just the obvious ones, since disagreements are almost always about edge cases nobody wrote down. A measured agreement rate between annotators, which tells you whether the task is even well defined before you scale it. And a review loop, because label quality degrades over long sessions and drifts as the guideline is reinterpreted. In LLM work labelling has shifted rather than disappeared. Less of it goes into training data and more into evaluation sets, preference comparisons and acceptance criteria — the labelled examples that decide whether a change to a prompt or a model actually helped. Model-generated labels can bootstrap a set cheaply, but a human-checked slice is what keeps the whole set honest, since a model grading its own kind of output inherits its blind spots.