GATE DA Supervised Learning – Data Science & AI | Set 1

Original solution basis:
Standard R^2 always increases when adding features. Adjusted R^2 penalizes non-informative features: 1 - [(1 - $R^2)(n - 1) / (n - p - 1)$].

Deep Dive
Adjusted R² corrects ordinary R² for the number of predictors. With n observations and p predictors (excluding the intercept), it is 1-[(1-R²)(n-1)/(n-p-1)]. Unlike ordinary R², adjusted R² can decrease when an added predictor does not improve fit enough to justify the lost degrees of freedom.

Key Points to Remember:
• Adjusted R² = 1-[(1-R²)(n-1)/(n-p-1)].
• p counts predictors, not the intercept.
• Adjusted R² penalizes unnecessary predictors; it can be lower than ordinary R².

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
Recall = TP / (TP + FN) and FNR = FN / (TP + FN). Therefore, Recall + FNR = (TP + FN) / (TP + FN) = 1.

Deep Dive
Recall=TP/(TP+FN), while FNR=FN/(TP+FN). Adding them gives (TP+FN)/(TP+FN)=1. Therefore FNR=1-Recall. This is analogous to FPR=1-specificity, but it is calculated over actual positive examples.

Key Points to Remember:
• FNR = FN/(TP+FN).
• Recall + FNR = 1.
• Recall is TPR; FNR is the fraction of actual positives missed.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
Validation error decreases initially but starts rising as the model begins overfitting. Early stopping halts training at minimum validation loss.

Deep Dive
During training, validation loss is monitored on data not used to update model parameters. If it stops improving or begins to rise while training loss continues to fall, the model may be overfitting. Early stopping halts training near the best validation point and can act as a form of regularization.

Key Points to Remember:
• Monitor validation, not training, loss for early stopping.
• Stop after a patience interval without validation improvement.
• Restore the checkpoint with the best validation score when appropriate.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Deep Dive
Tree algorithms differ in how they route missing values. XGBoost can learn a default direction for missing values at each split. C4.5 handles missing attribute values using fractional/probabilistic assignment, while some CART implementations use surrogate splits. Thus the general idea is that certain tree implementations can handle missing values during split evaluation, but the exact mechanism is algorithm-specific.

Key Points to Remember:
• Missing-value handling is implementation-specific.
• XGBoost learns a default branch direction for missing values.
• C4.5 uses probabilistic/fractional handling; CART variants may use surrogate splits.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
All four statements are fundamental principles of linear regression theory.

Deep Dive
A is correct under the Gauss-Markov assumptions, including a correctly specified linear model, exogeneity, and full column rank; homoscedasticity and uncorrelated errors give the minimum-variance BLUE property. B is correct because Ridge trades some bias for reduced variance. C is correct because L1 penalties can set coefficients exactly to zero. D is also correct under exogeneity: heteroscedasticity alone does not make OLS biased, although OLS is no longer BLUE under the usual Gauss-Markov variance assumptions.

Key Points to Remember:
• BLUE requires the Gauss-Markov conditions; homoscedasticity matters for efficiency, not unbiasedness.
• Ridge introduces bias but often reduces variance.
• Lasso can produce exact zero coefficients and perform feature selection.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.