Re: out of memory

spasmous wrote:



Bin Jiang wrote:

Thanks, run your suggested commend nnz for my matrix, the
answer
is
as follows:

ans =

36292

Any further comments?

Let me get this straight. You have a 20000x20000 matrix with 36000
elements - or approximately 2 per row. Hm that's like a 6000x6000
dense
matrix (+overhead). Quite big - if only MATLAB allowed single
precision
sparse matrices...

One solution might be to create your own sparse format (eg. see
numerical recipes) and use singles and ints for storage. Look at
CSPARSE -
<http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=11740&objectType=file>.
The source code cs.h defines the matrix like this:

typedef struct cs_sparse /* matrix in compressed-column or
triplet
form */
{
int nzmax ; /* maximum number of entries */
int m ; /* number of rows */
int n ; /* number of columns */
int *p ; /* column pointers (size n+1) or col indices (size
nzmax) */
int *i ; /* row indices, size nzmax */
double *x ; /* numerical values, size nzmax */
int nz ; /* # of entries in triplet matrix, -1 for
compressed-col */
} cs ;

I guess if you want it badly enough you can change the doubles to
floats (and elsewhere in the code).



This is actually a very small problem ... you just need to store it
as a sparse matrix, not as a dense one. You don't need much memory.
With only 36000 entries, you can trivially work in double precision.
You don't need sparse single precision.

A 20,000 by 20,0000 matrix with 36,000 nonzeros would take memory
equal to

4*20000 + 12*36000 bytes = 512K bytes.

This entire matrix will fit in the cache of most processors ... you
do work on it with hardly any memory at all.

Don't attempt to create the matrix as a dense one. Create a list of
nonzero entries: three arrays i,j,x of length 36000. Then the kth
entry in the matrix is in row i(k), column j(k), and has value x(k).
The order of entries in this list doesn't matter; they can appear in
any order. Then do

A = sparse (i,j,x) ;

then type "whos", and you'll see this problem is quite small.

You don't need to modify CSparse to solve this problem.

So what is it you want to do with this matrix?

Tim Davis (...author of CSparse, x=A\b when A is sparse, etc).
.

Relevant Pages

  • Strange breakage on amd64 RELENG_6
    ... *** Error code 1 ... L1 2MB instruction TLB: 8 entries, ... Physical memory chunk: ...
    (freebsd-questions)
  • Re: Kernel panic: Route cache, RCU, possibly FIB trie.
    ... My theory is that it is related to the route cache ... One of the crashed was caused by out of memory, but all the memory was allocated through slab. ... On this machine, rigth now, between 14000 to 60000 entries in the route cache. ... All the entries are freed via RCU which due to the deferred delete ...
    (Linux-Kernel)
  • Re: Auto correct
    ... problems in Word are often caused by low memory resources -- rebooting ... refreshes that memory. ... The number of AutoText entries is only limited by hard drive space. ... > state and zip for Dr. B Jones followed by a space and Dr. C ...
    (sci.med.transcription)
  • Re: Microcontroller Family Selection Guide
    ... DPTR to reference memory and the fact that most instructions do not ... AVR, where ALL Data memory goes via some pointer, or offset pointer. ... It is not the precision, it is the absence of Low Power on Chip OSC. ...
    (comp.arch.embedded)
  • Re: J3 request for interpretation, f.p. semantics of the REAL intrinsic function
    ... than for memory; e.g., 80 bits for registers and 64 bits for memory ... for the case of standard double precision floating point numbers. ...
    (comp.lang.fortran)
Loading