Re: Linear regression vs. vectors from principal components analysis


Ray Koopman wrote:
They're answering different questions, solving different problems.

The direction of my question was that I am proposing to use the methods
of PCA (the largest eigenvector of the covariance matrix) in order to
find a prediction line (which is what linear regression does), and that
this PCA constructed prediction line has some desirable features that
the regression line doesn't (symmetry of treatment of the variables).
So maybe my question can be refined to

- is that method 'not even wrong'? (with my explanation above, I
expect that it is not) They are not -totally- different problems,
right? And using the methods of one (PCA), one can attempt to solve the
same problem as the other.

- is that method bad? (the prediction is poor or poorer than linear
regression; I am supposing that the 'error' is a function of the
-shorter- eigenvector)

- is the symmetry of treatment of the variables a bad thing? In an
experiment the controlled variable is the independent one, and the
recorded data is the dependent one, a function of the independent
variable and therefore there is necessarily an asymmetry of treatment;
but outside of an experiment, when two vars are observed (one not
necessarily a function of the other), there is an inherent equal
importance of the two.


Mitch

.

Relevant Pages

  • Re: Logistic Regression
    ... I haven't looked into correlations between word usage carefully. ... > hindrance to standard multiple regression analysis. ... I have played around with using a Mahalanobis distance based K-Nearest ... PCA was popular, as an example the work of John Burrows of Newcastle ...
    (sci.stat.math)
  • Re: Principle Component Analysis
    ... I read a book about clearing multicollinearity of the independent variables by PCA. ... The result could be in the form of latent roots or latent vector but the problem is how do i use this PCA in regression? ... HOWEVER -- there is a serious problem here -- some of the principal components may not be predictive of the Y variables. ...
    (sci.stat.math)
  • Linear regression vs. vectors from principal components analysis
    ... Why isn't PCA used rather than linear regression? ... you compute the eigenvalues and eigenvectors of the ...
    (sci.stat.consult)
  • Re: Linear regression vs. vectors from principal components analysis
    ... Why isn't PCA used rather than linear regression? ... you compute the eigenvalues and eigenvectors of the ...
    (sci.stat.consult)
  • Re: ridge regression, good for prediction or representation?
    ... And isn't it a common sense that methods like PCA that ... >(LDA/logistic regression are good for discrimination)? ... >then implies that ridge regression serves as BOTH a prediction (since ... i-th element being d_i, so the i-th element of c from the ...
    (sci.stat.math)