In Linear Regression using Ordinary Least Squares (OLS), what is the closed-form solution for the weight vector w given feature matrix X and target vector y?
The Normal Equation for OLS regression minimizing sum of squared errors is $w = (X^{T} X)^{-1} X^{T} y$.
Why does Lasso Regression (L1 regularization) produce sparse weight vectors compared to Ridge Regression (L2 regularization)?
L1 norm constraint ||w||_1 <= t forms a diamond/octahedron with non-differentiable corners at coordinate axes where elliptical loss contours frequently intersect first.
In Logistic Regression, if the probability of belonging to class 1 is p, what is the log-odds (logit) transformation?
Logit function is log-odds = $\ln(p / (1 - p))$. The sigmoid function is its inverse $p = 1 / (1 + e^{-z})$.
In a Decision Tree, if a node has 30 positive samples and 30 negative samples, what is its Shannon Entropy H(S) in bits?
p+ = 0.5, p- = 0.5. H(S) = - 0.5 log2(0.5) - 0.5 log2(0.5) = 0.5 + 0.5 = 1.0 bit (maximum impurity for binary classification).
Which of the following splitting criteria in Decision Trees measures the probability that two randomly chosen elements from a node belong to different classes?
Gini Impurity = 1 - sum(p_i^2) represents the probability of incorrectly classifying a randomly chosen element if it were randomly labeled according to the distribution of labels in the subset.