GATE DA Supervised Learning – Data Science & AI | Set 1

Original solution basis:
All four statements are correct ROC/PR curve properties.

Deep Dive
ROC plots TPR against FPR, while the precision-recall curve plots precision against recall. PR curves often reveal minority-class performance more clearly when positives are rare, because precision directly reflects false positives among predicted positives. A random ranking has expected ROC-AUC 0.5.

Key Points to Remember:
• ROC: TPR versus FPR.
• PR: precision versus recall; often informative for imbalanced data.
• Random ranking has expected ROC-AUC 0.5.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
All four methods are effective strategies for high-dimensional feature spaces.

Deep Dive
Lasso can select features by shrinking some coefficients to zero. PCA projects data onto a smaller set of principal components. Random Forest feature subsampling reduces reliance on a fixed set of features and can help in wide feature spaces. Kernel SVMs can represent nonlinear boundaries in high-dimensional feature spaces, although kernel methods may be computationally expensive for very large sample counts.

Key Points to Remember:
• Lasso performs embedded feature selection.
• PCA reduces dimensionality through projection.
• Random Forest feature subsampling and kernel methods can be useful, but suitability depends on sample size and computational cost.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
All four statements are fundamental properties of regularization.

Deep Dive
L1 regularization encourages sparse coefficients; L2 regularization shrinks coefficients smoothly toward zero; and both add a complexity penalty to the training objective. Increasing \lambda strengthens the penalty, usually making the model less flexible: bias tends to increase while variance tends to decrease.

Key Points to Remember:
• L1 can set coefficients exactly to zero.
• L2 usually shrinks coefficients without exact zeros.
• Stronger regularization usually increases bias and decreases variance.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
k-NN is non-parametric (does not summarize data into fixed number of parameters). The first three are parametric models.

Deep Dive
A parametric model is described by a fixed-size set of parameters whose number does not grow directly with the number of training examples. Linear and logistic regression have coefficient vectors; Gaussian Naive Bayes stores class priors and feature-distribution parameters. k-NN instead retains training instances and is non-parametric.

Key Points to Remember:
• Linear regression and logistic regression are parametric.
• Gaussian Naive Bayes estimates a finite set of priors/means/variances.
• k-NN is non-parametric and instance-based.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
p+ = 0.4, p- = 0.6. H(S) = - 0.4 log2(0.4) - 0.6 log2(0.6) = - 0.4(-1.3219) - 0.6(-0.7370) = 0.5288 + 0.4422 = 0.971 bits ~ 0.97.

Deep Dive
The class proportions are p₊=40/100=0.4 and p₋=60/100=0.6. Substitute into H=-$\Sigma p log₂p$: H=-0.4log₂0.4-0.6log₂0.6$\approx 0.97095 bits$. Rounding to two decimal places gives 0.97 bits.

Key Points to Remember:
• Entropy uses base-2 logarithms when measured in bits.
• H=-$\Sigma pᵢlog₂pᵢ$.
• A 40:60 split has entropy close to, but slightly below, the binary maximum of 1 bit.

Exam tip:
Keep full precision during intermediate calculations and round only at the final step, as requested.