Question 91
Which of the following activation functions suffer from the VANISHING GRADIENT problem for large magnitude inputs?
Sigmoid (max derivative 0.25) and Tanh (max derivative 1.0) saturate near 0 for large inputs. ReLU variants have constant derivative 1 for z > 0.