Re: Documentation for SAVE in D3?

save
Invokes backup process.

Syntax:
save {(options)}

Description:
invokes the backup procedure to save the entire file system, an
individual account, or to perform an incremental save. It is the
process invoked by the "account-save" and "file-save" processes.

The appropriate peripheral storage device must be attached to the
current process before invoking this command. See the list of "set"
commands.

The following types of saves may be performed:

A "full file-save" is a complete logical backup of the Pick file
system.

An "incremental file-save" saves only those items which have changed
since the last full file-save. Changed items since the last save are
referred to as "dirty" items. If multiple incremental file-saves are
done between full file-saves, the most recent "incremental" file-save
contains the accumulated changes since the last full file-save. See the
"u" option.

A "list driven file-save" saves all items for a given file that are
specified in the "dm,file-of-files," starting at attribute 25 of the
item representing that file. Attribute 25 begins the "select list". See
the "l" and "m" options.

An "account-save" saves all the files in a single account

An "incremental account-save" is like an "incremental file-save", but
only affects the files in a single account.

A "dummy" or "fake" save is a save without the "t" (tape) option. This
is useful in cases where the Pick file system needs to be verified for
integrity.

"save" may also be driven from an "active" list, like any other TCL2
verb, although it is truly not a TCL2 verb. If a list is active when
save is invoked, each item in the list is treated as an account name.
On a "full" file-save with a list active, a fully restorable tape is
created. This automatically includes the "dm" account, even if it's not
in the list. On an account-save with a list active, the prompt for
account name does not appear. This type of save is not "fully
restorable", unless the "dm" account is in the list. This type of tape
may be used with "restore-accounts". In both of these types of saves,
the "dirty" bits are not cleared unless the "v" option is used. There
is a potential conflict with this function and the "l" and "m" options.
If these options are used and a list is active, the active list
replaces the list in the "dm,file-of-files,".

In any type of save process, the "d-pointer" to the file determines the
course of action for saving or not saving files. The "d-pointer" type
is attribute 1 (one) of its corresponding file-defining item. Files are
saved according to the following rules:

"d", "dc", "dl", and "dp" files are always saved.

"dx" files are not saved. Any files "beneath" a "dx" file are also not
saved. This can be overridden using the "x" option.

"dy" file pointers are saved, but NOT the data within them. After a
restore, a "dy" file will be empty. This can be overridden using the
"y" option.

The system does not save frames which are in the overflow table (unused
frames), so the amount of backup media required is related to the
number of frames currently in use.

When saving to multiple reels, the system prompts with:

Load volume #n and type 'C'
label 08:00:00 16 Jan 1997 DATA "account.name" # -

The prompt accepts either the letter "c", which tells it to continue
after inserting the next sequential reel, or the letter "q", to quit
and stop the save.

"Dirty Bits" - The item deletion scheme:

In order for the "incremental save" to work properly, the system
automatically "marks" (or "dirties") any item that is changed in any
way. This mark is normally cleared at the end of a complete save, and
displays the message "clearing dirty bits" as it does so, unless the
"v" option is specified.

If a file has a "dx" or "dy" code and an item in it is updated, that
item is not marked as changed when it is filed. Therefore, even if the
"x" or "y" is removed before the save, that item is not saved by an
incremental file-save.

The save processor locks groups as it saves them. While the group is
locked, no process may access any item in that group. Locking groups
prevents spurious Group Format Error messages that would occur if
another process changed an item while it was being saved.

Up to four groups may be locked at one time. These groups are the ones
containing: the mds dictionary pointer, the md pointer, the file
dictionary pointer, and the group for the data currently being saved.

Any changes made to the file system during the save (either a change or
a deletion) are NOT written to the save.

If the save is being done to streaming cartridge tape (SCT), an
explicit end-of-data sequence must be written, since SCT cannot back
up. Both the "t-weof" TCL command and the FlashBASIC "weof" statement
write the second eod needed to indicate the end-of-file. Without the
eod, "account-restore" or "sel-restore" fail to see the end of the
tape.

At the end of the save, the system displays the message "clearing dirty
bits...", and a series of frame numbers display, showing the groups
whose "marks" are being cleared. During this time, several files are
reopened and updated, renamed and cleared. "Deleted" flags are cleared,
and "save#" fields are reset.

Deleting Items:

The scheme by which D3 removes items from a file has changed radically
from the way it was handled in R83 implementations. Here are the rules
that it follows:

1) If the file is a "dx" or "dy" type, deleting an item behaves the way
it did in R83, meaning that the group adjusts immediately to compensate
for the deleted item.

2) If the file is NOT a "dx" or "dy" type, the item is deleted and
replaced with a "deleted-item/item", which includes the item-id and the
"delete" flag. These items are later deleted during the "clearing dirty
bits" phase of a full save.

3) If the group has an update lock, the item delete bit is set and the
last user to release the group lock compresses the group according to
the previous two rules.

The reason that this was implemented has to do with incremental saves,
which have to know that the item no longer exists.

The following scenario illustrates why this is handled this way.

1) Suppose there is an item called "fred" in the file "flintstones",
and it is saved on a full file-save.

2) After the save, "fred" is deleted.

3) An incremental save is performed.

4) A meteorite strikes the system, necessitating a full restore (this,
of course, is STRICTLY hypothetical.)

5) A full restore is performed, which brings back "fred".

6) At the end of the full restore, the option to perform an incremental
restore is offered and accepted. It is during this process that "fred"
is deleted.

Options:
a Creates saves prior to release 7.0. On releases 7.0 and higher,
systems contain added information which cannot be understood by old
releases. Such older releases may get "tape format error"'s when
restoring from newer saves. The (A option supresses the added
information (including all update stamps and binary item sizes).

b Saves the b-tree indexes. After a full-restore, this prevents having
to regenerate indices.

c Creates R83-compatible file-save tapes, generating a file-save media
that will restore on R83 machines. D3 files have dictionary codes that
are not recognized by R83 and vice versa. Accounts should be "updated"
when restored. This option does not work with incremental saves, only
file and account saves. "dl", "dp", and "ds" pointers are saved as
"normal" "d-pointers". "dc" and "dy" pointers are saved as they are.
Binary items are properly dumped as R83 "CC" pointer items. Items
larger than 32K and not in "dc" files are skipped, with a message
displayed. Binary code items not in "dc" files are skipped.

d Prevents the clearing of dirty bits. Note that this option should
only be used when performing a save in preparation for a full restore.

e Saves external "qs" pointers. "qs" pointers are special q pointers
which point to an OSFI data source. Note that only the spooler and Unix
OSFI drivers support the save functionality at this point.

f Displays the file names as they are saved. If omitted, only the mds
file and individual master dictionary (account) names are listed.

g Displays gfe (group format error) when found, but skips the group and
continues the save. Errors are logged in the "dm,errors," file in
attributes 14, 15, and 16 of the error item. Attribute 14 is the save
date, 15 is the primary group fid (from gfe.sr), and 16 is the "account
dict > data" display information. At the end of the save, a message displays indicating the total number of gfe's encountered, if at least one was found. If used with the "b" option, index corruptions are also skipped.

i (account-save) Individual master dictionary save. The system prompts
for the account name, if it is not specified at TCL. Using the "i"
option with an active list generates multiple account-saves. If a list
is active and the "i" option is specified, the system generates a tape
that may be restored with the "restore-accounts" command.

j Suppresses the dirty bit counter display.

k Reorganizes all "file-of-files" numbers sequentially on the tape.
This is also useful when frame one is corrupted or some file numbers
are negative. The incremental restore will abort with negative file
numbers. (releases 6.0 and higher only).

l Each file may have a list of specific item-ids within the file to
save. This option saves only the list of item-ids provided in the
associated item for the given file, located in the "dm,file-of-files,",
beginning at attribute 25. Any number of item-ids may follow in
attributes 26 and beyond. An "*" (asterisk) in attribute 25 saves all
items in the file. If there is no list, only the file-defining item is
saved, as if the file had a "dy" d/code. In addition, the "l" option
does not "unmark" (clear "dirty" bits) items that have changed. These
items will be saved on subsequent incremental file-saves or
account-saves. This is similar to the "v" option for the full
file-save.

m This is the same as the "l" option, but it additionally "unmarks", or
"clears" the "dirty" bits.

p Directs console listing to the printer, via the Spooler.

q Changes the terminal display from hierarchical format to indented
format. This is the "old" (R83) save display format.

r This is the same as the "l" and "m" options, except the list is also
saved with the file as a separate item in "dm,file-of-files,".

s An item is stored in "dm,file-of-files," for each file saved. The "s"
option generates additional statistics which may be reviewed with
"list-file-stats".

t Outputs to the "tape" (magnetic media) which has been previously
attached to the process. If the "t" option is not specified, "save"
does all the processing, but nothing is written to the media. The
system prompts for the file-save tape label which is written on the
tape as part of the tape label. Omitting the "t" option is useful for
re-generating system statistics and/or finding gfe's, and is the basis
of "fake" or "dummy" saves. See "tape label, definition".

u Performs an "incremental" file-save or account-save. This saves all
file items and data items that have changed since the last full
file-save or account-save.

v Performs a "full" file-save, but does not "unmark" the items that
have been changed. These items will continue to be saved on subsequent
incremental file-saves. Unless the "i" or "u" options are specified,
the system does a "full" file-save.

x Saves files with a "d-pointer" type of "dx". See "file-defining
items" regarding "dx" pointers.

y Saves files with a "d-pointer" type of "dy". See "file-defining
items" regarding "dy" pointers.

R83 "save" options:

The following options were available on all R83-compatible releases.
Some of the option characters are used in D3, but their functionality
has changed.

d Saves data files.

f Outputs filenames. When omitted, only system-level names are
displayed.

g Handles gfe's encountered during save. Rather than stopping and
waiting for a response, it skips the rest of the current group and
proceeds to next group. Logs entries in STAT-FILE when used with the
"s" option.

p Directs the report output (not the saved data) to the printer, via
the Spooler. If "p" is not specified, output is displayed to the
terminal.

n No overflow space is required. This makes it possible to perform a
save on a system that has no overflow space available. Note: If there
are more than 1500 files on the system, one (1) frame of overflow space
will be needed for each group of 125 files beyond the first 1500.

s Creates file statistics in STAT-FILE.

Example:
to dm
set-sct
Tape attached block size: 16384
[1082] Tape device is attached to quarter inch Streaming Cartridge Tape

save (bft)
file-save tape label = daily-backup
1 35 > dm
1 36 > dm > newac
1 37 > dm > newac > newac
..
..
Load volume #2 and type 'C'
label 08:00:00 16 Jan 1997 DATA dm daily-backup #
..
..
2 142 > lastaccount > lastfile > lastitem
Clearing dirty bits ...

Warnings:
It may be of some interest to note that the order in which data is
saved is the opposite of the way it is displayed. To clarify this
point, consider the following display.

1 35 > dm
1 36 > dm > newac
1 37 > dm > newac > newac

This suggests that the "dm" file (a level "1", or "md") is saved,
followed immediately by "dm > newac" (a level "2", or "dictionary"),
followed by "dm > newac > newac" (a level "3", or "data section"). In
fact, the "d-pointers" to the "dm" and "dm > newac" are saved, but not
the data within them. The "data" section is saved FIRST. It is followed
by the dictionary. The data from the md is saved LAST. This also
provides a major clue about how these files are restored.

See also:
check-dx TCL: Verb set-floppy TCL: Verb
file-save (R83) TCL: Verb file-save TCL: Verb
sel-restore TCL: Verb t-read Tape: Verb
restore-accounts TCL: Verb admin.files TCL: Menu
create-file TCL: Verb incremental save Definitions: Definition
dummy save TCL: Definition check-account TCL: Verb
active list Definitions: Definition t-verify Tape: Verb
dirty bits Definitions: Definition d/code Attribute-Defining Item

tlog-restore TCL: Verb list-file-stats TCL: Verb
tape label Tape: Definition :absload (R83) TCL: Verb
full restore Definitions: Definition account-restore TCL: Verb
deleting items Definitions: Definition incremental restore
Definitions: Definition
account-save TCL: Verb file-of-files Files: Definition
file control block Definitions: Definition f-resize TCL: Verb
basic-prot TCL: Verb network save/restore Definitions: Definition

restore Definitions: Definition check-file TCL: Verb
pointer item Definitions: Definition y Processing Codes:
Processing Code
Header Files Definitions: Definition t-copy TCL: Verb


Compatibility:
D3 7.0, AP, R83

.

Relevant Pages

  • Re: alternative method/s to file save
    ... The MV backup process is a one-size-fits-all tool that really doesn't ... sized you could do a regular file-save of say the bob account on monday, ... reported a failure in the restore. ... It's great to have your non-dm accounts in alphabetical order on the tape! ...
    (comp.databases.pick)
  • Re: alternative method/s to file save
    ... to cease system activity and perform a backup. ... we have a tape drive that is rated at 650 gig/hour. ... file-save / restore-accounts provides. ... sized you could do a regular file-save of say the bob account on monday, ...
    (comp.databases.pick)
  • Re: alternative method/s to file save
    ... and prepends dm account even if you didn't arrange for it to be present. ... restore their systems and found they had bad tapes - but the restore ... I find sorted account names on tape to be a real help when I later scan the tape ... to pare down your backup to just the essentials. ...
    (comp.databases.pick)
  • Restoring data problem
    ... I'm trying to restore an account on a D3/NT 7.5.2. ... is an account among others and I've done two file-save on two different ... When I try to restore it in the FSI, D3 gives me a lot of errors and, if I ... If I do a sel-restore of the missing files these go ok. ...
    (comp.databases.pick)
  • Re: FILE-SAVE versus AIX mksysb tape
    ... Also backing up to Disk vs tape much faster, then you can backup the backup ... We used to run full FILE-SAVE ... not have item/file restore capability, ...
    (comp.databases.pick)
Loading