Re: Multiple IMAP connections to same folder

On Fri, 30 Dec 2005, Aaron Carlow wrote:
Now my question as a non-programmer is what would happen if you simply kept all descriptors open until the lock was released. Is this even feasible?

I don't think that you fully understand the problem. Suppose the file opening, operations, and closing are done by a subroutine library that your application calls. From the point of view of the application, the subroutine library is a black box; it doesn't necessarily know what the subroutine library does. This characteristic of fcntl() requires that the application know what the subroutine library does.

It gets worse. Thanks to links, files can be accessed through multiple names. So, an application may not know that fileA and fileB are really the same file. So the application may have to keep a table of all open files with their device and inode numbers, and prior to opening any file, check to see if it's open by that name.

How many descriptors are needed for a typical IMAP operation? I suppose these questions betray my ignorance on the matter but I'm an insatiably curious sort of fellow.

Here's what typically happens in an IMAP server. The client has a mailbox open. Since it's open, it has a shared lock open on the mailbox on the server. Now, the client uploads a message; the upload operation supplies the name of the destination mailbox and there is no requirement that it be the same mailbox that is open. So it opens another descriptor append the message to the destination, keeping the first descriptor for the open mailbox.

But suppose it is the same mailbox. It now has two descriptors open on the same file, and then when the upload finishes the second descriptor (which never locked, because it was just appending) is closed. That kills the lock on the first descriptor. That in turn allows some other session to open and lock the mailbox, and maybe do things like expunge it even though the first session still has it open...oh dear, oh my.

This is just one of many paths by which an IMAP server can innocently fall afoul of this fcntl() "feature" even though it thinks that what it's doing is completely unrelated to the currently open mailbox.

It gets even worse for a user program that may have multiple mailboxes open at a time.

The only reason that I can think of for fcntl() to have this feature is to protect novice programmers. fcntl() locking has deadlock (session A waiting for a lock that is held by session B, which is waiting for a lock held by session A) protection. Apparently someone thought that it was a better idea to break locks rather than have buggy programs deadlock (and thus cause the bug to be identified and fixed).

What it does is make fcntl() locking very painful for anything but the most trivial use of locking.

The way I put it, having combination locks and electronic safeties on guns sounds like a real good idea...until you're in a gunfight and your gun won't go bang when you want it to...

What other types of applications use locks? Is this just a mail oriented issue? 

Any sort of database would use locks.

Most of the traditional mail applications were very primitive and never attempted to provide multiple simultaneous access, much less multiple simultaneous read/write access. Instead, they just prevented mail delivery (e.g., from sendmail) and a single mail reader from stepping on each other; and if a user ran two mail readers at a time, he "deserved" having his mail corrupted.

[In fact, I very much remember being told in the late 1980s that multiple simultaneous access to mail was "impossible" on UNIX and that I shouldn't even think of such a bizarre concept.]

IMAP has multiple simultaneous access, and UW imapd offers it (albeit not with the traditional UNIX mailbox format).

And if this is as serious as you seem to indicate, I wonder why there isn't more of an uproar from the Linux application developers? 

Some programmers have squawked about it!

Other programmers have given up on having good lock support in UNIX and UNIX like systems, and consequently resort to other means to accomplish the synchronization that they need.

For example, the oft-touted Maildir format is often claimed not to use locks at all; and its proponents claim this to be a benefit. Actually it does; but these locks are kernel-internal locks inside the filesystem. Maildir application code doesn't do any locking itself.

Windows does have good lock support. I wouldn't be surprised if the evil empire uses this as a selling point for Windows. Traditional operating systems, such as OS/360, TOPS-10/20, VAX/VMS, Unisys, etc. (pretty much anything that ever had COBOL applications!) also had good lock support. It's pretty likely that your paycheck is still printed by some ancient operating system.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
.

Relevant Pages

  • Re: Multiple IMAP connections to same folder
    ... it has a shared lock open on the mailbox on the server. ... So it opens another descriptor append the message to the destination, keeping the first descriptor for the open mailbox. ... It's pretty likely that your paycheck is still printed by some ancient operating system. ...
    (comp.mail.imap)
  • Re: Tech says the problem is XP, is it?
    ... It is possible for software to "lock" the drive, disabling the button, ... there's no way for it to become locked again until windows is up. ... >>tray will not open. ... it opens some times others not unless I do the fix. ...
    (microsoft.public.windowsxp.general)
  • Re: Rename file in VB 6 if not in use
    ... If by 'in use by others' you mean it's out to an application that opens ... files without maintaining a lock on them (e.g. notepad) then you have no way ... but not write to it or rename or move it. ...
    (microsoft.public.vb.general.discussion)
  • Re: Need 5VDC 100mA to 12VDC 1A converter
    ... > solenoid that opens a lock. ... > charging up an electrolytic cap. ... swapping out the solenoid isn't an option. ... Is it possible to use a R/C servo that pulls the lock? ...
    (comp.arch.embedded)
  • Re: LOR with filedesc structure and Giant
    ... We hold the file descriptor lock for the duration ... > because I need to grab a vnode lock to authorize polling the vnode using ... The easiest fix is to take a reference on the descriptors in the ...
    (freebsd-current)