GATE DA Supervised Learning – Data Science & AI | Set 1

Original solution basis:
Statements 1, 2, and 4 are correct. Logistic regression models log-odds linearly, is typically estimated by maximum likelihood, and sigmoid outputs can be interpreted as P(Y=1|X). Statement 3 is not universally true: the log-loss is convex, but strict convexity and a unique finite minimizer require additional conditions such as appropriate feature-matrix rank and absence of complete separation.

Deep Dive
A is correct because logit(P(Y=1|x))=wᵀx+b. B is correct because the usual estimator maximizes the Bernoulli likelihood, equivalently minimizing log loss. D is correct because the sigmoid output estimates the conditional class probability under the model. C is not always correct: the objective is convex, but strict convexity and a finite unique minimizer depend on design-matrix rank and separation conditions.

Key Points to Remember:
• Logistic regression is linear in log-odds, not necessarily in probability.
• MLE is equivalent to minimizing binary cross-entropy.
• Convexity does not automatically guarantee a finite unique minimizer in every dataset.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
All four statements accurately describe k-NN characteristics.

Deep Dive
A is correct: k-NN is non-parametric and instance-based. B is the usual bias-variance trend as k increases. C is correct because distance calculations depend on feature scales. D is correct when training points are distinct in feature space and each point is its own unique nearest neighbour; duplicate feature vectors with conflicting labels can create ties and invalidate the unconditional 'always' claim.

Key Points to Remember:
• k-NN is instance-based and non-parametric.
• Increasing k generally lowers variance and raises bias.
• The 1-NN zero-training-error statement assumes no conflicting duplicate feature vectors/ties.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
Naive Bayes is parametric (stores only prior and CPT distributions), so instance storage is NOT required.

Deep Dive
A states the conditional-independence assumption. B is correct because additive smoothing gives unseen categorical events nonzero estimated probability. C is correct: the assumption may be violated and the classifier can nevertheless perform well, although probabilities may be poorly calibrated. D is false because Naive Bayes stores estimated priors and likelihood parameters rather than all training instances.

Key Points to Remember:
• Naive Bayes assumes conditional independence given class.
• Laplace smoothing avoids zero likelihoods for unseen categories.
• Naive Bayes is a parametric model and does not need to retain every training example for inference.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
All four criteria are standard node splitting metrics in ID3, CART, C4.5, and Regression Trees.

Deep Dive
Information Gain and Gain Ratio are classification-tree criteria; Gini impurity is widely used in CART classification trees; and mean squared error/variance reduction is a common criterion for regression trees. The correct criterion depends on whether the target is categorical or continuous and on the particular tree algorithm.

Key Points to Remember:
• Classification criteria include entropy/information gain, Gini, and gain ratio.
• Regression trees commonly minimize squared error or variance within child nodes.
• Do not confuse a node impurity measure with the final model evaluation metric.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.

Original solution basis:
All four options are fundamental theorems in SVM theory.

Deep Dive
A is correct because hard-margin constraints require a separating hyperplane. B follows from the dual representation: only \alpha ᵢ>0 examples contribute to the decision function. C is the kernel trick. D is true for the Gaussian RBF kernel, whose feature-space representation is infinite-dimensional under the standard setting.

Key Points to Remember:
• Hard-margin SVM requires linear separability in the chosen feature space.
• Only support vectors with nonzero \alpha ᵢ contribute to the decision function.
• RBF kernels correspond to an infinite-dimensional feature space.

Exam tip:
Identify the governing definition or theorem first, write the formula, substitute the given values, and check dimensions/signs before selecting the option.