Transfer Learning

Transfer learning is reusing what a model learned on one task as the starting point for another. Instead of initialising a network randomly and teaching it everything from scratch, you take a model already trained on a large general corpus — text, images, code — and adapt it to your narrower problem. The representations it built while learning the general task turn out to carry most of what your task needs; you are only supplying the last mile. This is the single idea that made modern AI accessible to companies without research budgets. Training a capable language or vision model from zero requires data and compute that almost nobody has. Adapting one requires hundreds or thousands of examples and an afternoon. Every practical workflow you encounter — fine-tuning an open-weight model on your support transcripts, training a classifier on top of embeddings, prompting a frontier model with a handful of examples — is transfer learning in some form, differing mainly in how much of the original model you disturb. Those forms sit on a spectrum worth knowing. At the cheapest end you freeze the pretrained model entirely and train a small head on its outputs, or skip training altogether and put your examples in the prompt. In the middle sit parameter-efficient methods like LoRA, which learn a small set of additional weights and leave the base model untouched. At the expensive end you update the full network, which fits the hardest domain shifts and carries the highest risk of catastrophic forgetting — getting better at your task while quietly getting worse at general ability. Two practical cautions. Transfer works when the source and target domains share structure; a model pretrained on web English adapts well to your product's support tickets and poorly to a genuinely alien signal like raw sensor telemetry. And the pretrained model brings its own biases, licence terms, and knowledge cutoff along with its capabilities, so what you inherit is not only the useful part. Choosing the lightest form that clears your accuracy bar is nearly always the right default.