Which mathematical rule forms the computational foundation of the Backpropagation algorithm in Multilayer Perceptrons?
Backpropagation computes the gradient of the loss function with respect to every network weight by recursively applying the calculus chain rule backward from the output layer.
What is the primary cause of the 'Vanishing Gradient Problem' in deep feedforward networks using Sigmoid or Tanh activations?
Because the derivative of sigmoid peaks at 0.25, multiplying these small values through successive layers causes gradients to decay exponentially toward zero, halting early layer learning.
In gradient descent optimization, what role does the 'Learning Rate' (eta) play?
The learning rate scales the magnitude of parameter updates along the negative gradient vector: w_new = w_old - eta * grad(Loss).
How does 'Stochastic Gradient Descent' (SGD) differ from traditional 'Batch Gradient Descent'?
Batch gradient descent processes the full dataset before updating parameters, whereas SGD performs frequent parameter updates per sample, introducing helpful noise that can escape local minima.
What is the primary purpose of 'Pooling Layers' (e.g., Max-Pooling) in Convolutional Neural Networks (CNNs)?
Pooling layers downsample feature map height and width, reducing computational overhead and parameters while retaining dominant features and translation invariance.