Re: Finding REALLY hidden files?

On Thu, 11 Oct 2007 16:43:25 -0700, in 'rec.video.desktop',
in article <Re: Finding REALLY hidden files?>,
Gene E. Bloch <spamfree@xxxxxxxxxxxxxx> wrote:

On 10/11/2007, Frank posted this:
Not to be picky, but slack space is slightly different than the space
lost due to space being allocated in terms of clusters.

Virtually all hard disk drives use, at the physical level, a sector
size of 512 bytes (newer standards with larger sector sizes have been
proposed, but aren't yet in common use). Because of this, the drive
itself always reads and writes data in terms of sectors. So, if you
have a file whose length isn't an exact multiple of 512 bytes, that
wasted space within that last, partially used, sector of the file is
called slack space.

Example: a file that is one byte in length will still consume 512
bytes (one sector) on disk, even if the cluster size happens to also
be 512 bytes. In that case, the slack space is 511 bytes.

That's the worse case scenario, of course. A file which is 511 bytes
in length would only have slack space of one byte.

Slack space has nothing to do with space wasted due to having a large
number of small files on a partition that's been defined as having a
large cluster size (allocation unit size). I have seen partitions with
many thousands, even tens of thousands, of small files, where the
cluster size was 32 kB, thus causing a *lot* of wasted space on the
partition. This is an example of where it would have been more
efficient, from a space utilization viewpoint, of using a small, say 4
kB, cluster size when creating the partition.

There are programs on the market which, for security reasons, can
erase (write over) slack space at the end of files so that that data
can't be recovered. Sometimes interesting things can be found in slack
space. :)

However, the programs I have run that report the space wasted due to
unfilled clusters always called the space in question "slack space".

Yes, there are programs that do this. I don't have a major problem
with programs doing that, especially when they're not taking into
account the wasted bytes within the last sector of most files.

Also, most regular users seem unaware of the 512 byte sector size and
what that potentially means in terms of what gets written to disk when
they save a file. It usually means that whatever happened to be in the
buffer prior to the current write operation is what will end up in
those last few bytes of the sector. Even if we assume an average of
256 bytes unused in that last sector, 256 bytes can hold multiple file
names, e-mail addresses, a Web or FTP URL, even a small GIF image file
.... things that might be difficult to explain under certain
circumstances.

Any decent sector editor program will allow you to view this data.

Also think of this: if the cluster size is 32 KB (and under FAT32 I
*have* had drives with clusters that big) and you create a file of
length one byte, it is not just the 511 bytes in the first sector that
are lost to further use, it is also the 512 bytes in each of the other
63 physical sectors, for a total wastage of 32767 bytes.

Correct.

For wastage,
I'm *far* more interested in that number than in the 511. I'm also
*far* more interested in erasing all 32767 bytes when I care about
security holes...

Of course, but we have nothing to hide. :)

That is true in the Windows and DOS file systems I have used. If there
is a file system that will assign the unused sectors of a cluster to
other files, I have never experienced it, although I suspect some of
the disk-compression schemes of the bad old days might have done that.

NTFS is a bit different from FAT file systems in that under NTFS small
files are stored directly within the MFT (Master File Table) for
improved access times. Keeping the metadata and the filedata (data
set) close together speeds up access.

(Hmm - I seem to have forgotten the name of the compression software
that I used for a while back then.)

DoubleSpace and DriveSpace were two commonly used Microsoft
compression schemes, as was Stacker from Stac Electronics. There were
others as well, but I never used any of them, not being willing to
suffer the performance penalty due to compression/decompression of the
data, not to mention the implications involved in moving compressed
drives from one system to another - where the target system for the
move might not be equipped with the compression software.

Even so, whatever happened
internally, the API always presented normal clusters to the software.

Yes, for it would have been confusing, at least to the average user,
if it didn't.

--
Frank, Independent Consultant, New York, NY
[Please remove 'nojunkmail.' from address to reply via e-mail.]
Read Frank's thoughts on HDV at http://www.humanvalues.net/hdv/
(also covers AVCHD and XDCAM EX).
.

Relevant Pages

  • Re: Finding REALLY hidden files?
    ... The disk storage for a file is allocated in fixed size units called ... This means that if the cluster is 32KB and the file length is 12 times ... but slack space is slightly different than the space ... Virtually all hard disk drives use, at the physical level, a sector ...
    (rec.video.desktop)
  • Re: CONVERT FAT 32 to NTFS
    ... drives, three partitions in each. ... CONVERT changed the FAT32 system to NTFS, 512 bytes per Cluster ... (512 bytes per Sector, ...
    (microsoft.public.windowsxp.basics)
  • Re: CONVERT FAT 32 to NTFS
    ... I have two physical drives, ... three partitions in each. ... CONVERT changed the FAT32 system to NTFS, 512 bytes per Cluster (512 bytes ... per Sector, ...
    (microsoft.public.windowsxp.basics)
  • Re: file size and space used
    ... Has to do with 'slack space'. ... between the end of the file and the end of the cluster. ... "smorg" wrote: ... | scsi drives configured for soft raid with a mirror. ...
    (microsoft.public.win2000.file_system)
  • Re: Finding REALLY hidden files?
    ... Any left-over space in the last cluster of a file is lost, in that it will never be used for any other data. ... but slack space is slightly different than the space ... Virtually all hard disk drives use, at the physical level, a sector ... if the cluster size is 32 KB (and under FAT32 I *have* had drives with clusters that big) and you create a file of length one byte, it is not just the 511 bytes in the first sector that are lost to further use, it is also the 512 bytes in each of the other ...
    (rec.video.desktop)