Re: APS regression

In article <1128893439.644605.170530@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>,
Ray Koopman <koopman@xxxxxx> wrote:
>alphapoint05 wrote:
>> I'm a fan of all possible subsets regression; however, I have found
>> many postings directly opposed to this technique for variable
>> selection. I'm just wondering if there is anyone out there willing to
>> play devil's advocate. I'm not quite convinced that APS is grounds for
>> statistical excommunication.

>> Jeff Miller

>For years I've been telling people to do all-subsets regression and to
>look at results for all the subsets (or as many as their software will
>let them keep), ordered by the adjusted R-squares. The primary goal is
>not to identify a single "best" subset, but to see which predictors
>are always there in the better subsets at each subset size, and which
>predictors are never there in the better subsets. A secondary goal is
>to demonstrate how small the R-square differences among the better
>subsets are -- typically in the second or third decimal place -- and
>how foolish it would be to make any substantive distinctions based on
>those differences.

It depends on how many variables, and how they will be
used. One can compute the SSE for all subsets of k
variables in O(2^k) steps; another 10 variables multiplies
the time by a factor of 1000. Also, if there are more
than 10 variables, I suggest you have the computer keep
track of the ones you want to look at. You should use
a correction for the number of variables.

Possibly better would be a version of prior Bayes; this
would allow much more variety than can easily be described,
yet not as much as all regressions. If the object is
prediction, it might not make much difference, but if it
is understanding, or prediction in changed circumstances,
this is no longer the case.



--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hrubin@xxxxxxxxxxxxxxx Phone: (765)494-6054 FAX: (765)494-0558
.

Relevant Pages

  • Re: stepwise regression by GENSTAT
    ... My handbook considers only stepwise regression as a method to select ... leaving behind only "random variation" in the residuals (residuals = ... to which subset of these to use as predictors. ...
    (sci.stat.math)
  • Re: Questions about square errors
    ... Take a look at the 10X10 correlation coefficient matrix and the ... multicollinearities. ... least squares and/or multiple regression. ... Your model may have several unnecessary predictors. ...
    (sci.stat.math)
  • Re: Enter versus forward method for linear regression
    ... Regression, ... present the coefficents and p values of all predictors so that readers ... try Robert Abelson's book "Statistics as Principled Argument." ... and examine the effects on the coefficients. ...
    (sci.stat.edu)
  • Re: Using Ridge Regression to disentangle highly correlated explanatory variables
    ... the regression model, which did it's job at reducing the VIF greatly. ... the relative impact of each of the three explanatory variables. ... impacts of your three correlated predictors, ... and the VIFs should be 1 for each score variable. ...
    (sci.stat.math)
  • Re: Collinearity, confidence intervals and sampling
    ... read that if you have collinear variables, the best fitting plane of the ... predictors are highly correlated so the predictor values fall in a straight ... Does this mean that this is not a problem if you have population level data ... What about other assumptions of regression e.g. ...
    (sci.stat.consult)
Loading