Re: USB memory keys -- the plot thickens

Lon Hooker writes:

I'm a bit late to this thread (just now joined the group) so if I go
over any ground already covered, forgive me.

First, in order for removables to be used by an LVM system, there are a
few simple rules:

o ALL USB magnetic media is "partitionable"... no exceptions. This
specifically includes the so-called "Large Floppies".

That's very confusing. From the README.TXT file in the IDEDASD
package:

] Removable media devices are supported in one of two ways.
]
] LARGE FLOPPY The LS-120 / LS-240 drive.
]
] PARTITIONED All other magnetic removable media devices. This media
] appears as a removable hard drive.

Why make the distinction if both are partitionable?

History, mostly. The large floppy was originally invented to support
the LS-??? and similar devices. FAT16 was used because it needs no file
system drivers, and they were UN-partitioned for compatibility and
standardization.

Hence the distinction drawn above between "partition-ED" devices and
"large floppy" devices, since the latter are, by definition,
"UN-partition-ED" even though their media is "partition-ABLE".

When USB devices came along, however, the "large floppy" configuration
was a natural because it is OS neutral (anything descended from DOS can
use it) and requires no drivers except for USB support. That's why so
many USB removables work "out-of-the-box" with eCS-OS/2.

HOWEVER... it was also realized that there are file systems other than
FAT16, and that those all require a hard-disk type partition of some
sort. Therefore, all USB devices are made using "partition-ABLE" media.
All bases covered. It's up to you whether you want to partition and
re-format, or just leave it as a large floppy.

Partitioning a USB memory key works only if LVM sees the correct
capacity,
and it did not in the case of the 128 MB CompUSA memory key, listing it
as
having 31 MB.

You may find the following sequence of events interesting.

Today I looked at the directory on that memory key, and was puzzled
because
a subdirectory was empty. I didn't recall deleting any of the files in
that
subdirectory, but I still had them on a floppy, so I copied them to the
subdirectory on the memory key, and a new directory listing showed them
as one would expect.

Then I copied a large (74 MB) file to the memory key. After doing
another
directory listing, not only did the large file not appear in the
directory
listing, the files in the subdirectory were now gone!

So, I felt that it would be a good time to experiment with LVM on the
memory
key. I removed the existing volume and then created a new volume. LVM
would allow only a volume of 31 MB to be created (maximum size of the
memory key, or so it thought). I was beginning to wonder whether I had
just lost 90+ MB of capacity.

Decided to see if I could recover the lost capacity using a Windows XP
machine. But the Windows DISKPART tool doesn't even see the memory
key, and the GUI Disk Management tool left the "Delete partition"
option
grayed out. It did report the capacity of the memory key as 124 MB and
the size of the partition as 31 MB, but saw no free space on the
device.

So, back to OS/2 and LVM, which I used to remove the volume. Then back
to the Windows machine, which couldn't access the memory key anymore
because it wasn't formatted. Allowed Windows to format it, and now it
reports 129,945,600 bytes of total disk space.

Back to OS/2 and LVM, which now declares the partition table to be
corrupt
but sees the capacity of the memory key as 117 MB (previously it saw
only
31 MB, so the Windows reformat did something to it). Copied a large
file to
the memory key using COPY's /V option to verify the copy. After the
copy
operation was completed, a DIR command showed nothing in the directory!
Tried it again, but this time without the /V option, and now the file
appeared
in the directory listing. Copied the file a third time, this time with
the /V
option, but with the file already existing on the memory key (that is,
overwriting
an existing file), and this time the file persisted in the directory
listing. Then I
deleted the file and tried the copy operation a fourth time, with the
/V option,
but then OS/2 reported the disk as being inaccessible!

Reformatted the memory key using OS/2's FORMAT command. CHKDSK
reports 129,947,648 bytes of total space (2048 bytes more than Windows
reported, or one more allocation unit). Now the copy operation no
longer
generates the inaccessible disk error message, but because I used the
/V
option, nothing appears in the directory listing. Copying it a second
time
without the /V option worked. Sure looks to me that the use of the /V
option
on COPY causes serious problems. Not only does it prevent a file from
appearing in the directory listing (assuming it wasn't already there to
begin
with), it apparently also causes files to disappear from a
subdirectory,
because I'm pretty sure that the 74 MB file copy that I mentioned above
was
done with the /V option.

If somebody else out there is willing to experiment on a memory key
with
files they can afford to lose, I'd like to know if others find the use
of the /V
option on COPY to cause problems with the file not appearing in the
directory listing, or might it being something unique to the CompUSA
memory key?

Would this be considered a bug?

And now the WPS is misbehaving, with the machine beeping in response to
mouse clicks. I've got three windows with the active title bar color,
and
Ctrl-Esc finding things like the Volume Control not responding. Time
to
reboot. Coincidence?

o All USB removables must contain a valid partition table and be
"blessed" by LVM, such "blessing" to take the form of a "drive letter
allocation table", or DLAT. There is one exception...

o That exception is the "Large Floppy". It is defined as UNpartitioned
media having a FAT16 format. No partition table, no DLAT. Like the
familiar standard floppy, it is accessed directly by the OS and requires
no drivers beyond those needed by its hardware (the USB stack in this case)

I'm still confused. Above you said that all USB magnetic media is
partitionable with no exceptions,

"Partition-ABLE", but NOT necessarily "partition-ED"

and that it specifically includes large floppies.

Why is that confusing? I don't see how it could be any clearer... I
stated that a large floppy is "UNpartition-ED" with no partition table.
"Partition-ABLE" was not even mentioned.

o ALL partitionable media is supported and controlled by the
"USBMSD.ADD /REMOVABLES:n" statement in config.sys. Again, this
specifically includes "Large Floppies".

I thought that large floppies were not partitionable media.

No, they're not partition-ED... there's a difference.

Previously I had reported no joy getting an IBM 512 MB USB memory key
to work on my MCP2 FixPak5 system, even with the latest USB drivers.
A USB floppy drive works. A USB CD-ROM drive works. A device with a
2 GB Compact Flash card connected via USB works. But the memory key
did not work, even though Windows XP had no problem with it, and
identifies it as being formatted FAT16, which should be compatible
with OS/2.

Good. This says your USB sub-system is correctly installed... hopefully
with the latest USB stack, 10.162 being the latest, although 10.157 is
acceptable (those are the ones available to eCS... I have no idea what
is available for your MCP2 system). Older versions do have problems.

[E:\]bldlevel w:\os2\boot\usbmsd.add
Build Level Display Facility Version 6.12.675 Sep 25 2001
(C) Copyright IBM Corporation 1993-2001
Signature: @#IBM:10.162#@ OS/2 USB MSD Class Adapter Driver
Vendor: IBM
Revision: 10.162
File Version: 10.162
Description: OS/2 USB MSD Class Adapter Driver

Good!

So yesterday I spent a few bucks on a CompUSA 128 MB USB memory key to
use as a test. Inserted it in my system, and it worked right off the
bat!

No surprise... it's a "large floppy".

Isn't the 512 MB IBM memory key also a "large floppy"?

It doesn't have to be. If it was run through PQMagic or a W$ format, it
will have some sort of partition table, so cannot qualify as a large
floppy. Also, not all devices are created equal... some manufacturers
have really strange ideas about what they need to put on their devices,
not all of which are compatible with LVM. I have had devices that had
to be completely "zeroed" before they could finally be blessed by LVM
and formatted.

And how did you do that? DFSee?

So, question #1, why would a CompUSA 128 MB memory key work but not an
IBM 512 MB memory key? The readme that accompanied the USB drivers did
mention that the drivers wouldn't work with a tiny IBM memory key (I
think it was 16 MB or less), but that doesn't apply to this situation.

But the oddities didn't end there. When I inserted the IBM memory key
into the system, the Removable Device Monitor would dutifully report
that drive F: had been attached to the system, but no icon would appear
in the Drives folder, and no drive F: would appear in LVM's logical view.
LVM would show the memory key in the physical view, reporting 499 MB of
free space, which is approximately the right size for that memory key.

Apparently your IBM key is partitioned (i.e. has a partition table), but
has NOT been blessed by LVM. The USB system recognizes and attaches the
device, but LVM can't find a DLAT, so ignores it. Even worse, LVM can
do nothing to change things. If you do try to create a partition, LVM
tells you the freespace is unusable and does nothing. So you're in a
bind: To see the drive, you have to change it, but you can't change it
because LVM can't see it. :-(

Is there some way to tell whether a memory key has a partition table
or not? Presumably if it has two or more partitions, then it must have
a partition table, but if all the space is in a single "partition", how
can one tell if there is a partition table or not, other than assuming
that there is one because LVM doesn't recognize it?

To oversimplify: Every partition has a partition table. It's in the
first track of the partition... the MBR track. LVM's DLAT is located
there also. If there is no partition, there is no partition table.

Get DFSee and run that against the disk. See what the partition view
tells you. <http://www.dfsee.com/> or HOBBES. The download is not
crippled in any way, but if you need support, you will need to register
(MENSYS). The author, Jan van Wyjk, is very helpful to registered
users. AFAIC, it's one of a very few "indispensable" tools for eCS-OS/2.

After much trial-and-error, I found that the only consistently reliable
way to configure USB removables is with DFSee. It understands LVM, can
easily create a usable partition and even "bless" it with a new DLAT.
The only task remaining is to install a file system, and the current
version (8.00) can even do that (for FAT16/32 only).

Speaking of partitioning tools... NEVER, EVER, let either PQMagic or W$
touch a disk that will be used by LVM... they do really bad things to
the geometry. :-(

Not sure what you mean by "touch". You make it sound like installing
Windows and OS/2 on the same machine is asking for trouble, given that
Windows would have to "touch" the drives whenever the disk management
tool is run.

Could have been more clear. "Touch" means to allow it to write anything
to the partition table, MBR, whatever. The problem is that PQMagic and
W$ FDISK (among others) don't do things in "standard" ways.
Consequently, the disk geometry as seen by W$ and as seen by LVM can be
quite different. It doesn't seem to bother W$, but LVM is liable to
choke if things aren't "just right".

Well, for the CompUSA 128 MB memory key out of the box, LVM saw only
31 MB. After having both LVM and Windows muck around with it and
reformatting it using Windows, LVM now sees 117 MB.

As for W$' disk management tool, it's similar to LVM in that it assigns
drive letters at the user's whim instead of depending on a fixed
algorithm at boot. Unlike LVM, however, W$ uses an internal table to
store drive letter assignments instead of putting them 'on-disk' like
LVM's DLAT. Like LVM, however, those drive letter assignments are
invisible to any other OS.

Co-existence is no problem, provided that once OS/2 is installed and the
disk has been blessed by LVM, no partitioning tool except LVM or DFSee
is allowed to muck about with it again. IOW, go ahead and take a new
disk, partition it however you will using FDISK, PQMagic or whatever and
install W$. Then lock up all other partitioning tools and install
eCS-OS/2. Formatting the drives when OS/2 is installed takes care of
the W$ garbage.

With the CompUSA memory key, however, the Removable Device Monitor would
dutifully report that drive F: had been attached to the system, and an
icon would appear in the Drives folder. Yet LVM did not show any F:
drive in the logical view, and the physical view reported a drive with
only 31 MB of total and free space. Meanwhile, a directory listing in a
command window showed the correct amount of space (128 MB) on the drive.

So, question #2? Why can't LVM see what the command line sees? I'm
using the latest LVM code as well.

Again, no surprises. Per the rules, LVM's role is limited here.
Because the system accesses "Large Floppies" directly, LVM has no
function beyond passing them to os2dasd.dmd for generic drive letter
assignment. As for the command line, it uses system calls for drive
space so never looks to LVM at all.

In LVM's view, since there is no DLAT, the disk simply "isn't there".
What you see when you open it are a number of "dummy" containers, or
placeholders, equal to what you specified in "/REMOVABLES:n", but
they're useful only when a properly configured device is attached. IOW,
ignore LVM when dealing with large floppies... it's not only blind, it lies!

So, it seems we have three scenarios, but only one in which things
work properly:

1. No partition table: LVM lies.

With a large floppy, it surely does. Since large floppies are invisible
to LVM, it basically reports "dummy" information (there are nuances to
that, but it's fairly involved and not important, anyhow). Your 128
unit is usable? Read/write etc.? So why care what LVM thinks? Just
remember that it can be partitioned at any time, blessed by LVM and then
formatted any way you want... HPFS, FAT, FAT32 etc. (moving it into
scenario 3)

It's no longer just LVM that concerns me. COPY seems to be a problem
when the /V option is used.

2. Partition table not blessed by LVM: unusable.

Unusable but fixable. You may be able to bless it using LVM, but most
likely you'll need DFSee, since LVM most times won't touch it. From
your earlier post, it looks like your 512 unit needs DFSee.

3. Partition table blessed by LVM: works.

Right! And this is the best place for ALL USB removables... large
floppies are just a kludge.

Hope this clears things up a bit

Some, but now my concern has moved on to the COPY /V option. I'm
looking for a reliable storage method for changeable files other than
the
hard disks themselves, and floppies develop CRC errors all too
frequently. A memory key seems like a good solution, but not if files
can disappear anytime you forget and use /V on a copy operation.

.

Relevant Pages

  • Re: USB memory keys -- the plot thickens
    ... it is NOT LVM aware, so can't work with LVM systems. ... I use USB thumb drives routinely between Windows XP and OS/2 and never have any problems using either FAT16 or FAT32 file systems. ... Once the drive has a good partition table (one written either by DFSee or LVM) and has been blessed by LVM, there's no problem formatting and using it on a W$ machine, as you have noted. ...
    (comp.os.os2.bugs)
  • Re: USB memory keys -- the plot thickens
    ... it is NOT LVM aware, so can't work with LVM systems. ... I use USB thumb drives routinely between Windows XP and OS/2 and never have any problems using either FAT16 or FAT32 file systems. ... Once the drive has a good partition table (one written either by DFSee or LVM) and has been blessed by LVM, there's no problem formatting and using it on a W$ machine, as you have noted. ...
    (comp.os.os2.bugs)
  • Re: Cannot mount old drive with LVM partition - help!
    ... I have an FC3 install on a 200Gb IDE hard disk. ... ext3 partition. ... partition layout and LVM for root file system. ... rearrange them at will to make the drives appear as I wish. ...
    (alt.os.linux)
  • Re: Safe to use PQ Partition Magic with LVM?
    ... an OS that doesn't understand LVM drive letters. ... moving the drives? ... I use an old production partition, ...
    (comp.os.os2.misc)
  • Re: USB memory keys -- the plot thickens
    ... All USB removables must contain a valid partition table and be "blessed" by LVM, such "blessing" to take the form of a "drive letter allocation table", or DLAT. ... in the Drives folder, and no drive F: would appear in LVM's logical view. ...
    (comp.os.os2.bugs)