Re: Very fast file system access

On 2007-05-24, Faeandar <mr_castalot@xxxxxxxxx> wrote:

I don't know about CXFS but suspect it's similar to QFS. GFS, SANFs,
and GPFS are fairly unknown to me other than the glossies.

You can probably rule out SANFS, as it seems to be replaced by GPFS:

ftp://ftp.software.ibm.com/common/ssi/rep_wh/n/CLW03003USEN/CLW03003USEN.PDF


Fast access meaning I'm not noticeably limited in access speeds by
metadata or cache or lock issues.

I believe GPFS has some per-directory locking when creating new files, which
has caused problem for me when flooding the same Maildir with new messages..
10's of thousands of new messages, but I wouldn't expect this to be a problem
for your large files.

With multi-node access to the data
via FC I expect there to be no noticeable difference than if it were
single-node access, all else being equal.

Agree.

Trust is not an issue. Cache will only get you so far as it simply
moves the bottleneck about 30 seconds into the future.

For streaming large files yes.. It's quite common to only give each
node 64-128MB local page cache with GPFS, which works quite well
for streaming. But my boxes are more random I/O, and then I normally
give it ~50% of available the memory.


Again my concern is with the architecture of multi-writer access
solutions.

AFAIK this is where GPFS is king. Google for "project fastball",
100GB/s of sustained read and write performance to a single file...


-jf
.

Relevant Pages