Re: SVD for a large sparse matrix

In article <ef29fef.2@xxxxxxxxxxxxxxxx>, vivek <vivek.bannore@xxxxxxxxx> wrote:

John D'Errico wrote:

What is wrong with that?

So how will you expect to store a full version
of U and V? Remember they will be dense matrices
even if A was sparse.

HTH,
John D'Errico

Hi Guy_to forget, sorry i have no idea on how to interface vb with
matlab. But here's a doc which might or might not help you
<http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_external/>
<http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_external/f13397
5.html>
Thanks
vivek
-----------------------

Hi John,

There's nothing wrong with that but the problem that I'm facing is
that I do need the full U, S & V components from the SVD because the
other routines require that.

Unfortunately, I don't have a small matrix whose SVD i'm trying to
accomplish. Thisis a hige matrix we are talking about. Hence, the
search for help in achieving this if its at all possible. May be if
there's a way to keep clearing out unwanted variables or matrices
while the SVD is done, this would i guess deal with the 'out of
memory' situation. I'm not sure but this is just a vague idea....

Your help will be much appreciated.

No. Its not the incidental variables, and I realize
it is a huge matrix. Apparently you did not appreciate
what I was saying.

As I said, U and V will be full matrices. Since they
are orthogonal, you will build a full 254016x254016
matrix for V. It will require 520 gigabytes to store
the entire matrix.

You don't have that much memory, not unless you work
at some very profligate government agency. You can't
address that much memory anyway, even if you are running
64 bit Matlab. Windoze would probably take the first
510 gigabytes or so for itself. 8-)

Even were you to use the form svd(A,0), where V needs
only be an mxm matrix, V will still take up roughly 36
gigabytes of RAM.

You still don't have enough memory to compute the
full blown SVD. That is why svds is provided, because
many users only need a few singular values & vectors.

If you need the full thing, then you need to reformulate
your problem. SVD is not the answer. Period.

John


--
The best material model of a cat is another, or preferably the same, cat.
A. Rosenblueth, Philosophy of Science, 1945

Those who can't laugh at themselves leave the job to others.
Anonymous
.

Relevant Pages

  • Re: Need help using Sparse Matrix in MatLab. Please help
    ... Everytime I do this I get out of memory error: ... bytes to store the element. ... So your matrix still requires 7.5 gigabytes ... Do you have that much RAM period? ...
    (comp.soft-sys.matlab)
  • Re: Need help using Sparse Matrix in MatLab. Please help
    ... Everytime I do this I get out of memory error: ... bytes to store the element. ... So your matrix still requires 7.5 gigabytes ... Do you have that much RAM period? ...
    (comp.soft-sys.matlab)
  • Re: SVD, precision and size
    ... Peter Spellucci wrote: ... in order to compute the svd with U in explicit form (in the ... and this without getting "out of memory"? ...
    (sci.math.num-analysis)
  • Re: SVD, precision and size
    ... calculate U*P*U' where P is a diagonal matrix with ones and zeros on ... required for doing SVD on a very large matrix? ... if you can just store A in single precision, approximately 400 MB, ... and this without getting "out of memory"? ...
    (sci.math.num-analysis)
  • Re: SVD
    ... I tried with economy and 0 option, still out of memory. ... If you're calling SVD with the three-output syntax, ... about 760 MB of contiguous memory space. ... That will return a vector of the singular values. ...
    (comp.soft-sys.matlab)
Loading