Re: Utility to write huge files instantly???

robertwessel2@xxxxxxxxx wrote:
On Jan 23, 1:41 am, Bill Todd <billt...@xxxxxxxxxxxxx> wrote:
robertwess...@xxxxxxxxx wrote:

...

You probably want to use SetFileValidData().
He'd only need that if he wanted to be able to *access* the garbage in
the file, which he said he doesn't need to do.


Well, the idea was that he'd be able to allocate space to a file
without having the OS zero all that space.

Yes - and there's no intrinsic reason why SetFileValidData should have any effect on that: the normal byte-granularity end-of-data marker should suffice regardless of where the allocation ends, unless it's maintained only as a small integer offset within the last file cluster rather than as a 64-bit integer.


Anyway, I played with this a bit, and using the SetFilePointer/
SetEndOfFile technique,

But not SetFileValidData?

Windows, at least on XP and Vista, *will*
allocate space without zeroing it (but will read it as zeros), so long
as the volume is NTFS. That makes a certain sense, since there's no
place in FAT to store such information.

A quick search of mike's postings to other newsgroups seems to indicate that his problem (zeroing the space allocated) occurred at Close time, not at allocation time. He says that SetFileValidData (to a small value) before closing the file did alleviate that, but not whether it also released the unused space at Close (which would be consistent with, say, maintaining ValidDataLength only in RAM rather than on disk).


He does appear to zero pages as needed if you write actual data in the
file. So while the allocation is very quick, seeking to the end of
the file and writing a byte gets the entire file zeroed.

OTOH, you *can* read all the zeros without delay.

Hmmm. If you can do that *without* using SetFileValidData, then apparently SetEndOfFile is moving the end-of-data mark rather than just allocating space - unlike (I think, though I haven't tried it) the case with using the NtCreateFile approach with an AllocationSize.

And in that case using SetFileValidData to move the end-of-data mark back to the start of the file would avoid the zeroing on Close that mike saw (without necessarily deallocating the space).


To do this on a removable drive you need to force the format to NTFS
(FAT being the default), which requires changing a parameter for the
drive.

Yeah - he did originally say FAT32 *or* NTFS, but all the subsequent discussion (including mention of sparse files) assumed the NTFS facilities.

- bill
.

Relevant Pages