GATE DA Supervised Learning – Data Science & AI | Set 1

Original solution basis:
GBM performs gradient descent in function space by fitting new trees to pseudo-residuals -dL(y, f(x))/df(x).

Deep Dive
At boosting iteration m, compute the negative gradient of the chosen loss with respect to the current predictions. These values are called pseudo-residuals. The next weak learner is trained to approximate them, and its output is added to the ensemble with a learning-rate factor. For squared-error loss, the negative gradient equals the ordinary residual y-F(x).

Key Points to Remember:
• Gradient boosting performs gradient descent in function space.
• New learners approximate negative gradients/pseudo-residuals.
• For squared-error loss, pseudo-residuals are y-current prediction.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
A confusion matrix tabulates actual vs predicted classes: $\begin{bmatrix} TP & FN \\ FP & TN \end{bmatrix}$.

Deep Dive
For binary classification, the confusion matrix compares actual and predicted labels. Its four counts are TP, FN, FP, and TN. With rows as actual classes and columns as predicted classes, one common layout is $\begin{bmatrix} TP & FN \\ FP & TN \end{bmatrix}$, although some software uses the opposite axis order; always read the labels.

Key Points to Remember:
• The four entries are TP, FP, FN, and TN.
• The layout depends on whether actual or predicted labels are placed in rows.
• Precision, recall, specificity, FPR, and accuracy are computed from these counts.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
Specificity measures proportion of actual negative instances correctly identified: TN / (TN + FP).

Deep Dive
Specificity measures how many actual negatives are correctly identified. The denominator is all actual negatives, TN+FP; the numerator is the correctly rejected negatives, TN. Therefore specificity=TN/(TN+FP). A false positive lowers specificity.

Key Points to Remember:
• Specificity = TN/(TN+FP).
• Specificity is also called true negative rate.
• It complements FPR: specificity + FPR = 1.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
FPR = FP / (TN + FP) = 1 - (TN / (TN + FP)) = 1 - Specificity.

Deep Dive
Among all actual negatives, the classifier either correctly predicts negative (TN) or incorrectly predicts positive (FP). Thus FPR=FP/(FP+TN). Specificity=TN/(TN+FP), so their numerators add to the denominator and FPR=1-specificity.

Key Points to Remember:
• FPR = FP/(FP+TN).
• Specificity = TN/(TN+FP).
• FPR + specificity = 1.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
Kernel SVM decision function f(x) = sum_i alpha_i y_i K($x_i$, x) + b is a expansion over support vectors.

Deep Dive
A kernel SVM represents its decision function as f(x)=$\Sigma ᵢ\alpha ᵢyᵢK(xᵢ,x)+b$, where the sum is over support vectors. A nonlinear kernel such as RBF makes the resulting boundary nonlinear in the original input space, even though the separator is linear in the implicit feature space.

Key Points to Remember:
• Kernel SVM uses support-vector kernel expansions.
• Nonlinear kernels can produce nonlinear boundaries in input space.
• A linear kernel gives a linear decision boundary in the original feature space.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.