In Laplace (Add-1) smoothing for a binary feature with $k = 2 categories$ and total count N, the smoothed probability estimate for count c is:
Laplace add-1 smoothing formula for k classes: $P = (c + 1) / (N + k)$. For $k=2$, $P = (c + 1) / (N + 2)$.
The Shannon Entropy H(X) of a fair coin (P(Head) = 0.5) in bits is:
$H(X) = - (0.5 log2 0.5 + 0.5 log2 0.5) = - (-0.5 - 0.5) = 1.0 bit.$
In a polytree Bayesian Network with bounded variable cardinalities, exact inference using belief propagation can be computed in:
A polytree has at most one undirected path between any two nodes. With bounded variable cardinalities, belief propagation performs a linear number of local message computations, giving O(V) time.
The phenomenon where observing one cause reduces the probability of another cause for the same effect (in v-structure A -> C <- B) is called:
Explaining away occurs in v-structures when learning one cause explains the effect, decreasing belief in alternative cause.
In Gibbs Sampling, each step resamples variable $X_{i}$ according to the distribution conditional on:
Gibbs sampling updates $X_{i}$ by sampling from P($X_{i}$ | MB($X_{i}$)), as MB($X_{i}$) contains all relevant conditioning variables.