Dropout regularization during neural network training works by:
Dropout randomly sets node activations to zero with probability p during forward training passes, preventing co-adaptation of features.
In Natural Language Processing, converting words into their root dictionary form using morphological and grammatical analysis is called:
Lemmatization uses vocabulary and morphological rules to reduce inflected word forms to their valid dictionary root (lemma), unlike crude rule-based stemming.
Batch Normalization accelerates deep network training by:
Batch Normalization normalizes activations per mini-batch, mitigating internal covariate shift and allowing higher learning rates.
In Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) units prevent vanishing gradients over long sequences using:
LSTMs introduce a constant error carousel cell state regulated by Forget, Input, and Output multiplicative gates.
Which gate in an LSTM cell determines how much of the previous cell state memory should be discarded?
The Forget Gate $f_t$ = $\sigma (W_f \cdot [h_{t-1}, x_t] + b_f) computes a value$ between 0 and 1, deciding what to erase from cell state $C_{t-1}$.