Transfer Learning
Transfer learning is a machine learning technique in which a model trained on one task or dataset is reused, often with additional fine-tuning, as the starting point for a different but related task.
Definition
Transfer learning is a machine learning technique in which a model trained on one task or dataset is reused, often with additional fine-tuning, as the starting point for a different but related task.
Overview
Training a model from scratch requires large amounts of data and compute. Transfer learning avoids this by starting with a model that has already learned general representations from a large dataset — for example, a language model pretrained on broad internet text, or an image model pretrained on millions of photographs — and then adapting it to a narrower, related task with a much smaller dataset. In practice, transfer learning often takes the form of fine-tuning: the pretrained model's weights are used as initialization, and additional training on task-specific data adjusts them for the new objective. Because the model already encodes general patterns such as grammar, object shapes, or semantic relationships, it needs far fewer new examples and far less compute to reach strong performance than training from random initialization. Transfer learning underpins the modern foundation-model paradigm: rather than building a bespoke model for every task, organizations take a large pretrained foundation model and adapt it, which is dramatically more efficient. It is closely related to few-shot learning and zero-shot learning, which represent even lighter-weight forms of leveraging a model's pretrained knowledge without full retraining. Courses like PyTorch Deep Learning and Hugging Face Transformers cover practical transfer learning workflows.
Key Concepts
- Reuses a pretrained model's learned representations for a new task
- Drastically reduces the data and compute needed compared to training from scratch
- Often implemented through fine-tuning on a smaller, task-specific dataset
- Foundational to the modern foundation-model deployment paradigm
- Applicable across vision, language, audio, and multimodal models
- Works best when the source and target tasks share underlying structure