Suppose singular values of a centered data matrix X (size 101 x 5) are $\sigma = [20, 10, 0, 0, 0]$. Given $N = 101 samples, what$ is the largest eigenvalue lambda_1 of the sample covariance matrix $X^{T}$ X / (N - 1)?
$lambda_1 = sigma_1^2 / (N - 1) = 20^2 / (101 - 1) = 400 / 100 = 4.$
In Hierarchical Clustering with 50 initial data points, how many total merge operations are performed to obtain a single final cluster?
Merging 50 individual points into 1 cluster takes N - 1 = 50 - 1 = 49 merge steps.
What fundamental logical function CANNOT be solved by a Single-Layer Perceptron with a linear decision boundary?
Minsky & Papert (1969) proved that Single-Layer Perceptrons cannot represent non-linearly separable functions like XOR.
According to the Universal Approximation Theorem, a feedforward neural network with a single hidden layer can approximate any continuous function on compact subsets of $R^n$, provided:
Universal Approximation Theorem states that a single hidden layer MLP with non-linear activations can approximate any continuous function given enough hidden units.
What is the mathematical derivative of the standard Sigmoid activation function $\sigma(z) = 1 / (1 + e^{-z})$?
d/dz $\sigma(z) = \sigma(z) * (1 - \sigma(z))$. Maximum derivative value is 0.25 at $z = 0$.