Federated Learning
Decentralized machine learning technique
Federated learning is a machine learning technique that trains a shared model across multiple decentralized devices or servers holding local data, without the raw data ever leaving its original location.
Definition
Federated learning is a machine learning technique that trains a shared model across multiple decentralized devices or servers holding local data, without the raw data ever leaving its original location.
Overview
In conventional machine learning, training data is collected centrally onto a server or data center before a model is trained on it. Federated learning inverts this: a central coordinator sends the current version of a model out to participating devices or organizations (such as smartphones, hospitals, or banks), each of which trains the model locally on its own private data for a few steps, and then sends back only the resulting model updates (gradients or weight changes), not the underlying data itself. The coordinator aggregates these updates — commonly via an algorithm called Federated Averaging (FedAvg) — into an improved global model, and the process repeats over many rounds. This architecture was pioneered by Google around 2016 for improving predictive keyboard models (like Gboard) directly on users' phones without uploading their typed text to a server. Its core motivation is privacy and data-governance: many valuable datasets, particularly in healthcare, finance, and personal devices, cannot legally or ethically be centralized due to regulations like GDPR and HIPAA or simple user trust concerns, but federated learning allows a useful shared model to be trained collaboratively anyway. Because data never leaves its source, federated learning also reduces bandwidth costs versus shipping raw data to a central location, though it introduces its own challenges around communication efficiency, handling non-identically-distributed data across participants (statistical heterogeneity), and robustness against unreliable or even malicious participants. Federated learning is often paired with additional privacy-preserving techniques such as differential privacy (adding calibrated noise to updates) and secure aggregation (cryptographic protocols ensuring the server can only see the aggregate of updates, not any individual participant's contribution) to further reduce the risk that model updates could leak information about specific individuals' data. It has become an active research area and production technique across mobile computing, healthcare AI collaboration between hospitals, and financial fraud-detection models trained across banks that cannot share customer data directly.
Key Concepts
- Trains a shared model across decentralized devices or organizations
- Raw training data never leaves its original location
- Only model updates (gradients or weights), not data, are shared centrally
- Commonly aggregated using the Federated Averaging (FedAvg) algorithm
- Pioneered by Google for on-device keyboard prediction models
- Addresses privacy regulations like GDPR and HIPAA by design
- Often combined with differential privacy and secure aggregation
- Handles challenges of statistical heterogeneity across participants' data