Re: Approximate solution to linear regression

Further to the same point:

-- Systat also does PLS and it's $1300 (commerical, single user, according
to their website).

-- Excel is also an option: XLSTAT-Pro (a general-purpose stats add-in) plus
XLSTAT-PLS (its additional module for PLS) total $590 (quick check via their
website, academic would be less, with more additional modules a little
more).

-- There is also a computational chemistry website (http://www.vcclab.org/)
with a freely acessible Java applet for PLS regresison:
http://www.vcclab.org/lab/pls/

Regards,

Gaj Vidmar
Univ. of Ljubljana, Fac. of Medicine, Inst. of Biomedical Informatics

"Bruce Weaver" <bweaver@xxxxxxxxxxxx> wrote in message
news:f5b3b8024c3@xxxxxxxxxxxxxxxxxxxx
vincent64@xxxxxxxxx wrote:
On Jun 19, 12:44 pm, Paige Miller <paige.mil...@xxxxxxxxx> wrote:
On Jun 17, 3:46 pm, "vincen...@xxxxxxxxx" <datashap...@xxxxxxxxx>
wrote:

Problem can have 40,000 variables, most of them highly correlated.
More variables than observations in some cases. I came up with an
approach, and my question is
(1) is this an original approach?
(2) more importantly, does it always provide a fairly accurate
solution?
The problem and solution are described
athttp://datashaping.com/contest14004.shtml
. The newsgroup can not render the mathematical formatting.
I haven't tried to go through your solution in any detail.

In similar situations, I use Partial Least Squares (PLS) Regression,
which is also an "approximate" method (actually, its a biased
regression) that doesn't care if you have highly correlated X
variables and many more Xs than observations. If you use the maximum
possible number of dimensions in PLS, you will get an OLS solution
without having to invert a matrix.

So, with that in mind, it seems to me your approximate solution is
trying to fit into a niche where there already is a solution, and the
PLS solution has proven useful in zillions of published articles. So
unless you can show that your approximate solution has better
properties than PLS, I don't see much of a need for it.

--
Paige Miller
paige\dot\miller \at\ kodak\dot\com

Thanks for your reply. I've heard that Lasso regression does similar
things too. Anyway, being efficient is much more important than being
original in this context: I'm not trying to publish an article, this
is not academic research. If I need to spend $150K to get PLS
regression software (SAS Enterprise Miner) and spend many hours
getting it to work, I'm MUCH better off re-inventing the wheel. So I
could rephrase my question as follows: am I re-inventing the wheel
quite well, meaning my approach is not significantly inferior to PLS
regression?


You must be joking about the $150K. Doesn't R do PLS regression? And
it looks like Stata has a routine to run the SAS implementation.

http://ideas.repec.org/c/boc/bocode/s456810.html

A single-user corporate version of Stata (intercooled, v10) is $1150.00,
according to their website.


--
Bruce Weaver
bweaver@xxxxxxxxxxxx
www.angelfire.com/wv/bwhomedir


.

Relevant Pages

  • Re: HOR4s start order
    ... the ARA on their website. ... Whilst it is certainly true that competing in the olympics would put ... and should not and in fact *cannot* be declared on the regression form: ... 2001 11 pts (regressed) ...
    (rec.sport.rowing)
  • Re: sendmail 8.13.7 available
    ... Claus Aßmann wrote: ... It's a regression in 8.13.7, see the website. ... I've verified the behavior on two servers but haven't been able to figure out if I can trigger the behavior. ...
    (comp.mail.sendmail)
  • Re: Approximate solution to linear regression
    ... In similar situations, I use Partial Least Squares Regression, ... So, with that in mind, it seems to me your approximate solution is ... I'm MUCH better off re-inventing the wheel. ... A single-user corporate version of Stata is $1150.00, according to their website. ...
    (sci.stat.consult)
  • Re: Approximate solution to linear regression
    ... In similar situations, I use Partial Least Squares Regression, ... So, with that in mind, it seems to me your approximate solution is ... I'm MUCH better off re-inventing the wheel. ...
    (sci.stat.consult)
  • Re: Approximate solution to linear regression
    ... The newsgroup can not render the mathematical formatting. ... In similar situations, I use Partial Least Squares Regression, ... So, with that in mind, it seems to me your approximate solution is ...
    (sci.stat.consult)