updates (Re: implementing read/write archive/image format)
- From: "cr88192" <cr88192@xxxxxxxxxxxxxxxxxx>
- Date: Mon, 1 May 2006 00:55:16 +1000
"cr88192" <cr88192@xxxxxxxxxxxxxxxxxx> wrote in message
news:824a0$44515754$ca83b3d3$9643@xxxxxxxxxxxxx
so, a little closer to the start of the month, the issue and idea came up
for implementing a fully read/write archive format.
dunno if the idea (or format/implementation) is actually interesting to
anyone. I guess people can post responses if they feel compelled, or maybe
I can put a copy online so people can mess with it if they want (or I
could email zip files)?...
it may be useful to hear any design criticisms or similar...
still, misc updates on the effort...
still using deflate (but deflate is fairly fast and works pretty well).
since I am implementing a good deal of basic filesystem style functionality
(readdir, file operations, directory operations, ...). the effort is a
little more involved than a typical archiver (which takes a more direct
"building an archive" or "extracting from an archive" approach), so progress
is a little slower (things are done via an external api, with an interface
and expected behavior similar to the os-level api's).
even as such, it seems with the original commandline version of the tool, a
lot of shortcuts were taken. some major operations, eg, adding or extracting
files, interacted fairly directly with the directory and cache managment
code (vs taking the more externalized approach, eg,
opendir/readdir/stat/mkdir, open/read/write, deletion, ...).
likewise, I had relied some on the shells' tendency to do basic strings
matching (dealing with '*', ...).
so, upon implementing the shell-based version of the tool, a lot of things
were altered. I have now split off the code partly into a half-assed library
(closer to a mass of source files that include each other, but oh well, I
have seen a few libs done this way).
implementing a shell-based tool pointed out a lot of bugs in the
implementation as well, which I have been fixing (along the lines of "oh
crap, if the first file in the root is deleted, all the archive contents
disappear", "mkdir is making an empty file instead of a directory", ... type
stuff). also bolted down more of the semantics of the file-format (empty
directories are in fact empty, root entry 0 is special and serves mostly to
point to the real first root file vs having contents, ...).
other major things:
originally the avl trees were built by using malloc to allocate the nodes. I
have changed this to a big array, and now integers are used to refer to
nodes vs pointers (this was partly by realizing that with pointers, calling
realloc risked potentially somewhat messing things up, but this transition
was still a hassle).
more implemented the caching features (so, now, stuff is actually forced out
to disk when the cache fills...). there is also a limit set to the max
number of files that can be in cache (file caches are managed by another
array, albeit the buffers and similar are still allocated with malloc).
fragmented files are still not yet implemented.
the shell version of the tool presently uses readline (fairly convinient and
gives nice input handling, but not exactly all that portable).
ended up doing some strings matching stuff similar to the host shell.
'*' matches a variable number of characters, but only within a single
directory;
'**' matches a variable number of characters potentially between directories
(this idea came partly from VI, dunno if used elsewhere);
likewise, '?' matches a single character, and '\' works as an escape.
eg:
*.txt will match text files within the current directory;
*/*.txt will match text files within a direct subdirectory;
**.txt will match all text files in this and sub-directories;
....
note that this matching is done by the commands and not the shell, meaning
that not all commands will sanely accept patterns (eg: the ls variants
accept paths, but not presently patterns, ...).
the shell also works by maintaining 2 paths, one is the external working
directory, and the other is the virtual working directory. a lot of commands
(import, export, rm, ...) are done in relation to these paths.
still no big deal though, after the lib is cleaned up a little more, I may
go and integrate this with my main codebase.
if made into a standalone lib though, maybe some people might actually care
for all I know...
or such...
.
- References:
- implementing read/write archive/image format
- From: cr88192
- implementing read/write archive/image format
- Prev by Date: Re: Am I wise to convert all my AVIs?
- Next by Date: Re: Recursive Compression Method -- Eureka!!
- Previous by thread: Re: implementing read/write archive/image format
- Next by thread: Recursive Compression Method -- Eureka!!
- Index(es):
Relevant Pages
|
|
