Backpropagation
Backpropagation is the algorithm used to train neural networks by efficiently computing how much each parameter contributed to the model's error, so those parameters can be updated via gradient descent.
6 resources across 2 libraries
Glossary Terms(5)
Gradient Descent
Gradient descent is an optimization algorithm that iteratively adjusts a model's parameters in the direction that most reduces its error, or loss, in order to…
Backpropagation
Backpropagation is the algorithm used to train neural networks by efficiently computing how much each parameter contributed to the model's error, so those para…
Attention Mechanism
The attention mechanism is a neural network technique that allows a model to dynamically weigh the importance of different parts of its input when producing ea…
Softmax Function
The softmax function converts a vector of raw real-valued scores (logits) into a probability distribution, exponentiating each score and dividing by the sum of…
Gradient Clipping
Gradient clipping is a training technique that caps the magnitude of gradients during backpropagation, preventing the exploding-gradient problem that can desta…