Re: memmapfile

"Dylan Walker" <blah@xxxxxxxxx> writes:

Once again, thank you both for your prompt responses. You are both
very helpful. I guess I didn't explain entirely what I am trying to
do. I am not trying to alter the data whatsoever [that is a ball of
wax I'd rather not deal with :)], I just want the ability to read the
data and input it into whatever functionality I implement in a mex
file. It is true that I could simply use C functionality to memory
map the file in parallel (i.e., a C memory mapped file in the mex
function and a matlab memory mapped file in matlab). I would in
principle need both as I need to access the data in an interpretive
setting, while loops in matlab are too slow to effiiciently perform
some operations over the entire data set. But memory mapping the file
twice is wasteful and I thought there might be a way to access the
data that is already mapped in matlab.

I have not been able to get an mxArray * to the data array that would
allow me to obtain a (e.g., double *) to the actual data. I have tried
using mxGetField(memmapobj,"data"), mxGetProperty(memmapobj,"data),
etc. and this just crashes. Perhaps I am not clear on what you
suggested before about using mexCallMatlab.

You mention that I can use the GET or SUBSTRUCT method with
mexCallMatlab to give me mxArray * of the class specified in format.
This seems like exactly what I am trying to do; so I would be very
much obliged if you could describe this method in a little more
detail. To access some portion of the data within matlab, I can call
memmapobj.data(i1:i2) , however "data" is not a proper array in the
usual sense and the traditional mxGet... functions seem to fail. I
think you are saying that the mexCallMatlab can be used as a way
around this, because the GET method in matlab is already "overloaded"
to deal with the memorymappedobj. But can I return the general
pointer to the data from mexCallMatlab (rather than just some slice of
the data that I call? Sorry if I am being unclear.

Memory mapping is an OS-level function. Once a file is memory-mapped, a
pointer to that data exists, and the user application (MATLAB) sees the
entire mapped region as valid memory that can be accessed normally.
Once the application tries to access a page of that memory region that
has not yet been read from disk, the OS transparently takes care of
faulting in that page. As far as I know, no object/subscripting
trickery is needed to access data. It's just a regular MATLAB variable
that has a Pr pointer that points to a memory-mapped region of virtual
memory.

So treat the Data element of the memmapfile object like any other MATLAB
array. That fact that the Pr pointer points to a memory mapped file
instead of a malloc-allocated region on the heap is completely
irrelevant. Just pass the Data element directly to the MEX file and be
happy.

Test code:

fid = fopen('testfile.dat', 'wb');
fwrite(fid, [17 18 19], 'double');
fclose(fid);

m = memmapfile('testfile.dat', 'Format', 'double');
s = memmaptest(m.Data);

clear m


----------------------------------------------------------------------------
memmaptest.c
----------------------------------------------------------------------------
#include "mex.h"

void mexFunction( int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[] )
{
double *x;

plhs[0] = mxCreateDoubleMatrix(1,1,mxREAL);
x = mxGetPr(plhs[0]);
*x = *mxGetPr(prhs[0]);
}
----------------------------------------------------------------------------



-Peter
.

Relevant Pages

  • Re: memmapfile
    ... It is true that I could simply use C functionality to memory ... function and a matlab memory mapped file in matlab). ...
    (comp.soft-sys.matlab)
  • Re: Memmap file without loading into memory?
    ... I can't store this in memory and so I've taken to memory ... matlab will keep this portion of the file in memory. ... since using a memory mapped file in the first place ...
    (comp.soft-sys.matlab)
  • Re: Memmap file without loading into memory?
    ... I can't store this in memory and so I've taken to memory ... matlab will keep this portion of the file in memory. ... since using a memory mapped file in the first place ...
    (comp.soft-sys.matlab)
  • Memmap file without loading into memory?
    ... I can't store this in memory and so I've taken to memory ... matlab will keep this portion of the file in memory. ... since using a memory mapped file in the first place ...
    (comp.soft-sys.matlab)
  • [PATCH] Documentation cleanup: trivial misspelling, punctuation, and grammar corrections.
    ... Different DMA engines may support different number of domains. ... +- API for paravirt ops for abstracting functionality for VMM folks. ... Read/Write attribute file that controls memory scrubbing. ... + Further information can be found in the interfaces section below for ...
    (Linux-Kernel)
Loading