Re: Mulberry gone, now what?

Mark Crispin wrote:
> On Sun, 9 Oct 2005, Dave Cridland wrote:
> > But for a 6+ digit message count, the number of octets adds up very
> > quickly indeed even for a THREAD response (It's 683k or so for 100,000
> > message mailboxes), and there's no quick-fix like ESEARCH to reduce it,
> > nor any method for chopping it into manageable chunks.
>
> True, but do you really want to thread 100K messages from a mobile client?
>

Why not? Polymer already handles 100K message mailboxes perfectly well
over low-bandwidth connections, even in a cold-cache situation. Why on
earth wouldn't I want to thread them too? (I don't see any problem with
handling much higher message counts, either, I just can't be bothered
to build such a mailbox.)

> I suppose that you know that you can thread a range of messages; there is
> no requirement that you thread all messages. You do run the risk of your
> subset message range thread being different that a subset of the total
> thread.
>

Well, yes. I *could* thread only a small portion of the messages. But
what, exactly, does that mean? In general, it means I'll have lots of
partial threads, with no easy way of reconnecting the subthreads.

If I could select the threads matching a criteria, this would be
easier, but I can't - I have to select the messages, then thread them,
instead of threading the mailboxes, and then select the threads I want.

If I could do the latter, it'd be easy - I'd want to display threads
which had a message with a relatively recent internal date, sorted by
highest internal date. Or even better, the last N threads.

> > True. But it can only change the thread tree for one "thread". (It may
> > merge two threads, of course).
>
> It can merge more than two threads. The level of change can be
> spectacular...
>

Yes, true. It's possible that it can merge all threads in the mailbox,
providing a missing, and previously unknown, common ancestor for all
threads.

> > As it stands, I think THREAD is too expensive for mobile clients,
> > useless for disconnected mode clients, and only useful for client which
> > aim to provide a snapshot view of a mailbox - basically meaning webmail
> > clients.
>
> Huh? THREAD works quite well for online clients!
>

Oh, across a LAN, yes.

Perhaps it'd be fairer if I simply said that both SORT and THREAD lack
the level of scalability found elsewhere in the protocol.

> > This may sound a little harsh, but the fact is that those users likely
> > to want a threaded view are also likely to have large mailboxes.
>
> Yes, but few MUAs today work well with 4-digit message counts, even fewer
> work well with 5-digit message counts, and even fewer work at all with 6+
> digit message counts.
>

Polymer gets regularly tested against a mailbox with nearly 130,000
messages in. This was one of the primary motivating factors behind
ESEARCH, although the set-syntax helps almost equally well with 130
mailboxes each with 1000 messages in. I'm confident that Polymer would
perform approximately the same over much higher message counts.

FWIW, I regularly use Polymer on mailboxes with 70k messages in, as do
some of the other users. I do this reasonably often over GPRS, so I'm
putting my money where my code is, as it were.

> My observations suggest that people split up large message count mailboxes
> into multiple smaller mailboxes long before they approach the message
> count limits of their MUA.
>

That's largely because most MUAs rapidly degrade as the message count
increases, and users have been invariably trained to "sweep" messages
across.

> Mailboxes have grown enormously, but almost entirely in the vector of
> average message size as opposed to number of messages.
>

Oh, that's grown too, certainly.

> >> You can accomplish the same user experience with a separate mailbox in
> >> which you stash the data that you would have placed in an annotation.
> >> The same thing holds true for address books and configuration; it may seem
> >> to be a "poor man's" way of doing things, but it actually works remarkably
> >> well and doesn't require any additional infrastructure.
> > And it's awful over low bandwidth.
>
> What constitutes "low"?
>

Well, my personal target is 2400 baud. If I can get Polymer usable over
that I'll be reasonably happy. The "reading your mail" bit now seems to
work fine at that bandwidth (or at least, Polymer's emulation of it),
the "configuration" bit is mostly okay. SORT and THREAD aren't
implemented, and folder listing is currently synchronous, which
effectively means "slow". Connections also take far too long for my
liking - I'm thinking of seeing how practical deploying something like
HIP would be on a mobile phone. On a laptop it's a no brainer, of
course.

> I have yet to find a mobile client that I can tolerate, but I've used Pine
> over 1xRTT, and before than over CDPD. It wasn't all that long ago that I
> finally said goodbye to 14.4k modems. I won't claim that it was a speed
> demon, but it was faster than some other programs using broadband.
>

Well, yes, PINE's handling of IMAP is very good - as I'd expect. But
configuration and addressbook handling is weak, in my opinion, both in
terms of functionality and expressiveness. That's not meant as an
attack on PINE, just a demonstration that a "poor man's" method, while
sometimes "good enough", isn't "as good as it can be".

As an aside, the problem with 14k4 modems wasn't the amount of data
transferred, per-se, but the time spent online, at least in the UK.
Polymer is asbolutely not designed to minimize the time spent online.
It's designed for circumstances where you're charged by transfer.

> > I use ACAP for that
>
> I'm glad to see that there's still some life in ACAP; but I fear the jury
> is still out. Sad to say, LDAP seems to have won the address book war.
>

Everyone liked, and still appears to like, the notion of ACAP. It's
just nobody actually wanted to take the plunge and actually use it -
just one conformant server and four MUAs, two of which are commercial
including Mulberry.

LDAP I'm not so certain has won the addressbook war - I think it's
managed a successful land grab, but failed to consolidate. It's used
heavily for canonical lists of address data, such as global
addressbooks and actual directories, but there seems little motion
toward using it as a personal addressbook. Moreover, I don't think
anyone's seriously considering accessing LDAP over a mobile link.

> > If a client wishes to create a mailbox foo, with the potential
> > intention of creating inferior mailboxes *and* having foo contain
> > messages, is there any way of knowing a priori if this is possible?
>
> No.
>
> > If I understand your answer correctly, the answer's no, which is what I
> > thought - it was largely a rhetorical question, counter to your
> > assertion that if a client uses the tools available to it, it can know
> > the differences in hierarchy behaviour.
>
> That doesn't really make much of a point. You have no way of knowing
> whether you can create any name at all
>

True, but you have a reasonable idea of whether it's possible, and a NO
response from CREATE should be exceptional rather than normal.

> > So a UI for "Create subfolder" has to ask two questions: "What name
> > would you like?" and "Do you want subfolders or messages?". Even though
> > for many servers, there's no distinction, and it's somewhat confusing
> > for users.
>
> I eschew the ambiguous term "folder" for this reason.
>

Yes, I suppose it'd be cleaner to have "create submailbox" and "create
subcontainer" or something.

> Your mileage may vary, but I find it remarkable that people expect that a
> mailbox should contain other mailboxes. I haven't yet heard a groundswell
> of support for C language source files to serve a second purpose as a
> container of other source files.
>

I like the analogy, forced as it is, but remarkable or not, and even
expectation or not, some servers allow this, some don't, and there's no
easy way for a client to know which short of "suck it and see", which
doesn't strike me as a clean methodology.

> > Really there ought to be a CAPABILITY string or a NAMESPACE extension
> > string (or both) explaining what the hierarchy is actually capable of.
>
> That is the wrong granularity, since the state is per-mailbox and it is
> perfectly possible for a mail store hierarchy to have multiple types of
> mailbox objects.

*nods* True. But in some cases, the behaviour is known on a
per-NAMESPACE or per-server basis.

Dave.

.

Relevant Pages

  • Re: An argument AGAINST hosting your own email domain.
    ... the ISP for a client is currently hosting their email and we are bringing it ... the client should be receiving ... > system to 'auth attacks', NDR attacks, attacks which have yet to be ... get rid of your global mailboxes and set up ...
    (microsoft.public.windows.server.sbs)
  • Re: Setting up Exchange
    ... We're only concerned with configuring Exchange to send mail to ... > the ISPs SMTP server, or directly to the Local Mailboxes for local users ... > your ISP and route it to the local mailboxes. ... > Now, when you say you cannot receive email on client PCs, is that Local ...
    (microsoft.public.windows.server.sbs)
  • Re: Best MacOsX email client?
    ... I've seen Eudora gag and barf on half that much mail and destroy users' mailboxes in the process, and I've seen happen more than once. ... in my opinion is the QuickMail Pro - or with it's new name 'QuickMail Client' and 'QuickMail Office'. ...
    (comp.sys.mac.apps)
  • Re: Mail.app confusion
    ... Luke Siemaszko wrote: ... I also see all the shared mailboxes, as children of a folder called "user". ... This makes sense because that is what the top level folder of the mail store i called. ... Have you checked the Mail.app client versions? ...
    (uk.comp.sys.mac)