Re: Linear regression vs. vectors from principal components analysis


Old Mac User wrote:
They are, indeed, intended for very different purposes.

Here's something to consider. If you run a PCA on a covariance
matrix... and then change one of the metrics (I'll explain) you'll get
a different "answer". Yes, the eigenvalues (and of course the
eigenvectors) will change.

Example: One of your variables is recorded in yards. Run the PCA. Now
convert the variable "in yard" to the metric "inches". Run the PCA
again. Oooops!!! The eigenvalues changed. The "answers" depend on the
metrics.

Example: Scale the variables by centering and divide by the respective
standard deviations.
Effectively... run the PCA on the correlation matrix. This produces a
dramatically different "answer".

yes, I see that now. but
1) isn't that a problem with PCA already (without this new use of
eigenvalues I'm proposing)?
2) doesn't that effect the regression line (and correlation) similarly?

But this makes me realize that I must have the hidden assumption that
the vars are 'isotropic', linked, from euclidean space.

Bottom line: Putting meaning/interpretation on the eigenvalues and
eigenvectors is a slippery thing.

But nonetheless, PCA is in no way a substitute for regression. OMU

you can't see how in any way that a largest eigenvector (sorry, with
origin the mean of both vars) might be construed as a prediction line?

Mitch

.

Relevant Pages