In LDA, the Within-Class Scatter Matrix S_W is computed as:
$S_W$ = $\sum_{$$i=1}^{C} sum_{x in c_i} (x - m_i)(x - m_i)^T measures variance within each individual class$.
In LDA, the Between-Class Scatter Matrix S_B is defined as:
$S_B$ = $\sum_{$$i=1}^{C} N_i (m_i - m)(m_i - m)^T measures scatter of class means m_i around overall mean m$.
The linear projection vector w in 2-class Linear Discriminant Analysis (LDA) is given by:
Maximizing Fisher's criterion J(w) = (w^{T} $S_B$ w) / (w^{T} $S_W$ w) for 2 classes yields optimal vector w proportional to S_W^{-1} (m_1 - m_2).
How is Singular Value Decomposition (SVD) of an n x d centered data matrix X related to PCA?
If $X = U \Sigma V^{T}$, then X^{T} $X = V \Sigma^2 V^{T}$. The right singular vectors V are the eigenvectors of X^{T} X (principal components).
If the singular values of centered data matrix X are sigma_1, sigma_2, ..., sigma_d, what are the eigenvalues lambda_i of the sample covariance matrix $C = (1 / (n - 1)) X^{T} X$?
Since C = (1 / (n - 1)) X^{T} $X = (1 / (n - 1)) V \Sigma^2 V^{T}$, the eigenvalues lambda_i = $sigma_i^2 / (n - 1)$.