Re: Approximate solution to linear regression

On 26 Jun., 14:30, Paige Miller <paige.mil...@xxxxxxxxx> wrote:
On Jun 22, 3:16 am, "S.W.Christensen" <s...@xxxxxxxxxxxxxxxx> wrote:

On 17 Jun., 21:46, "vincen...@xxxxxxxxx" <datashap...@xxxxxxxxx>
wrote:

Problem can have 40,000 variables, most of them highly correlated.
More variables than observations in some cases.

I haven't looked over your solution in great detail, but I would
suggest that:

1) Group your variables into clusters, based on their correlations
2) Construct an ensemble of regression models, each based on just one
exemplar from each cluster
3) Weight each model conservatively (because you have so many
variables); e.g. equal weighting.



Why create an ad hoc procedure, where the properties are not known,
and where you would have to defend the validity of the procedure, and
where you have to write the code yourself?


If you only choose to create procedures that are not ad hoc, then you
will never create a procedure. Talk about halting civilisation in its
tracks...

By the way: 1) the various parts of the method are thoroughly tested,
2) a method that consistently works has proven itself beyond the need
for defending, 3) writing code yourself is the best guarantee you can
ever have of its correctness (do you have faith in the correctness of
code written by software companies?)

Best regards,

Stefan W. Christensen

.

Relevant Pages

  • Re: Approximate solution to linear regression
    ... Group your variables into clusters, ... Construct an ensemble of regression models, ... variables); e.g. equal weighting. ... published examples of the validity of the procedure, ...
    (sci.stat.consult)
  • Re: Approximate solution to linear regression
    ... Group your variables into clusters, ... Construct an ensemble of regression models, ... variables); e.g. equal weighting. ... Stefan W. Christensen ...
    (sci.stat.consult)