Re: Incremental backups and renamed files?

In article <siegman-D30C60.11423329052007@xxxxxxxxxxxxxxxxx>,
AES <siegman@xxxxxxxxxxxx> wrote:
In article <fmoore-7279DF.14043729052007@xxxxxxxxxxxxxxxxxxxxxxxxxxx>,
Fred Moore <fmoore@xxxxxxxx> wrote:
Similarly, if you move a file/folder to another folder/directory, how
does the backup/cloning program know that moved file/folder is the same
as any other file/folder without doing a bit-by-bit comparison?

OK -- sounds eminently reasonable. I would have been equally ready,
however, to believe a response saying that each file on the system had
some unique serial number associated with it, that never got changed
even if the file was renamed, ...

This indeed is my understanding of how HFS+ works. The OS maintains a
database which ties the file name and any other info (metadata) to the
serial number. (In the Classic versions of the OS, this info was kept in
the invisible files Desktop DB and Desktop DF. I don't know where the
equivalent info is kept in X.) If you 'move' the file, you aren't really
moving all of the file's bits. You're just moving the _references_ to
the file's location. If you duplicate a file, the OS rewrites the file
to disk and assigns it a new, unrelated serial number.

... and also that each file had associated
with some metadata bits that were flipped if the file was opened, or
written to in any way, but not if the file was merely moved, renamed, or
had some of its other metadata changed -- in which case, if one is
trusting enough, one might only need to make any necessary changes in
the metadata of the previously backed-up and since renamed or moved
file, but not copy over the "data fork" part of the file (if that term
is even still used?).

What you propose sounds eminently reasonable, which is why that's not
how things work. ;)

In practice, the document's originating app determines how information
such as creation and mod dates are assigned and updated. Unfortunately
developers have not been consistent. A common example is whether just
opening and printing a document in a word processing program counts as a
modification even though _none_ of the content has been changed. It does
in some, not in others, and is a pref in still others. Further,
different apps which do auto-saving handle negated changes differently.
If you insert a character in a document then delete that same character
and close the doc, some apps will say the doc has been changed, others
won't. So, even if you had reliable file metadata being written
somewhere, the OS or the clone/backup utility would still have to know
how each and every app handled mod dates if they were going to attempt
to not copy duplicate files without a bit-by-bit comparison. As you can
imagine, this approach is fraught with way too many possible
complications to be useful.

A brief note on the word 'incremental': In all cases incremental means
'only copy new stuff; don't start from scratch'. However, 'What do I do
with old stuff?' varies from app to app. Retrospect popularized the word
'snapshot' as a variation of 'incremental' to indicate 'Keep all old
files so I can go back and get a previous version if I need to.' This
feature has saved many folk's employment when they accidentally lost a
crucial file. However, it requires a great deal of programming effort to
work reliably and hard drive space to implement, which explains at least
part of Retrospect's cost and complexity. Clone/sync apps just replace
everything and leave it at that.

--Fred
.

Relevant Pages

  • SSIS Object Model Programatically Script Dataflow Task
    ... I'm trying to design a console app that uses the SSIS object model to ... move data between 2 tables that have the exact same metadata ... // add oledb connection ...
    (microsoft.public.sqlserver.dts)
  • Re: File extensions are fun
    ... set its mac-dependent metadata based on the metadata available ... metadata (like a HTTP-server saying it's HTML and the file extension ... application-binding, open in the default app for that file type; ... since Tiger the Finder doesn't: It treats that file as JPEG ...
    (uk.comp.sys.mac)
  • Import document
    ... I'm tring to make a small app that will import documents into ... sharepoint and assign metadata based user inpout, ...
    (microsoft.public.sharepoint.portalserver.development)
  • Import document
    ... I'm tring to make a small app that will import documents into ... sharepoint and assign metadata based user inpout, ...
    (microsoft.public.sharepoint.teamservices)
  • Import document
    ... I'm tring to make a small app that will import documents into ... sharepoint and assign metadata based user inpout, ...
    (microsoft.public.sharepoint.portalserver.development)