Re: Linear regression vs. vectors from principal components analysis

They're answering different questions, solving different problems.

Mitch wrote:
Here's an understandably extremely loaded question:

Why isn't PCA used rather than linear regression?

Let me explain (so that this doesn't seem so contentious, or just plain
'not-even-wrong').

Both methods start with the covariance matrix.

- For regression, you solve a linear system based on that matrix.This
method is asymmetric in the variables, that is, you get different
regression lines depending on which variable you consider the
'independent' variable (this is the source of 'regression to the mean '
as an artifact, not of the data but of the process)
- For PCA, you compute the eigenvalues and eigenvectors of the
covariance matrix. all variables are treated symmetrically. The vector
with the largest eigenvalue could be interpreted as the line affording
the greatest predictive value (the shortest vector affording the least
variability).

Does that comparison make sense? Does PCA really come out of that
better? The asymmetry has started to bother me, but I've never heard it
addressed anywhere.

Mitch

.

Relevant Pages