GATE DA Supervised Learning – Data Science & AI | Set 1

Original solution basis:
Increasing tree depth to infinity causes severe overfitting, while the first three techniques prevent/reduce overfitting.

Deep Dive
Pre-pruning limits growth while the tree is being built. Post-pruning removes branches after a larger tree has been grown. Random Forest reduces variance by averaging many decorrelated trees and is another way to improve generalization. Increasing depth without control usually increases complexity and can fit noise, so D is not a remedy.

Key Points to Remember:
• Pre-pruning limits growth; post-pruning trims a grown tree.
• Ensembles such as Random Forest can improve generalization.
• Uncontrolled depth can increase overfitting.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
All four statements correctly formulate the Bias-Variance Decomposition.

Deep Dive
A and B describe the common interpretation of high bias and high variance. C is the expected squared-error decomposition. D describes the usual trend as model flexibility increases: bias tends to fall while variance tends to rise. These are tendencies, not universal monotonic laws for every dataset and learning algorithm.

Key Points to Remember:
• High bias is associated with underfitting; high variance with overfitting.
• Expected MSE = bias² + variance + irreducible noise.
• Increasing complexity often trades lower bias for higher variance.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
All four options are standard metrics calculated directly from confusion matrix counts.

Deep Dive
Precision, recall, F1, and accuracy can all be calculated from TP, FP, FN, and TN. Precision measures correctness among predicted positives; recall measures coverage of actual positives; F1 combines precision and recall; accuracy measures the fraction of all predictions that are correct.

Key Points to Remember:
• Precision = TP/(TP+FP).
• Recall = TP/(TP+FN); F1 is their harmonic mean.
• Accuracy = (TP+TN)/(TP+TN+FP+FN).

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
All four statements accurately contrast Bagging and Boosting frameworks.

Deep Dive
Bagging trains base learners independently on bootstrap samples and combines their outputs. Boosting trains sequentially, with each stage responding to errors or residuals from earlier stages. Random Forest is a bagging extension that also randomizes the candidate features at splits. AdaBoost raises the relative weights of examples misclassified by the current weak learner.

Key Points to Remember:
• Bagging is generally parallel and variance-reducing.
• Boosting is sequential and focuses later learners on remaining errors.
• Random Forest adds feature subsampling to bagged trees.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
Linear, polynomial (with standard valid parameter choices), and RBF kernels are positive semidefinite kernels. A sigmoid kernel is a valid Mercer kernel only for parameter settings that make its Gram matrices positive semidefinite; it is not valid for arbitrary \alpha and c.

Deep Dive
The linear, standard polynomial, and Gaussian RBF kernels are positive-semidefinite kernels under their usual parameter conditions. A sigmoid kernel is more delicate: it is not positive semidefinite for arbitrary \alpha and c, but can be valid for restricted parameter ranges. Therefore the qualification in option D is essential.

Key Points to Remember:
• A valid kernel must generate PSD Gram matrices.
• Linear and RBF kernels are standard valid kernels; polynomial kernels require standard parameter conditions.
• The sigmoid kernel is not valid for arbitrary parameter settings.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.