Re: Mulberry gone, now what?

On Mon, 10 Oct 2005, Dave Cridland wrote:
True, but do you really want to thread 100K messages from a mobile client? 
Why not? Polymer already handles 100K message mailboxes perfectly well
over low-bandwidth connections, even in a cold-cache situation. Why on
earth wouldn't I want to thread them too?

As does Pine; but the issue isn't a matter of software capability. It's a matter of user managability. Most users who have that many messages (and these are typically archived messages, not new messages) break them up into separate mailboxes, usually named in some way to categorize the archive (date range is popular, as is topic).

Yes, you can do searches instead. But do you really want to scroll through 100K messages, and enter in searches through 100K messages, on a mobile client?

(I don't see any problem with
handling much  higher message counts, either, I just can't be bothered
to build such a mailbox.)

OK, so your "can't be bothered" limit is at 100K. Very few people are there. Most drop out after 10K, with the peak being in the 4 digit range.

As it stands, I think THREAD is too expensive for mobile clients,
useless for disconnected mode clients, and only useful for client which
aim to provide a snapshot view of a mailbox - basically meaning webmail
clients.
Huh? THREAD works quite well for online clients!
Oh, across a LAN, yes. 

Or wi-fi.

I think that the GPRS/1xRTT guys are rapidly losing their window of opportunity. At about the same time that I finally dropped CDPD in favor of 1xRTT, I observed that my need for WAN wireless had been plummeting like a paralyzed falcon. So far, I've used WAN wireless *once* this entire year.

The culprit is wi-fi. It's springing up everywhere. I've found wi-fi to be ubiquitous and excellent in remote villages of Alaska and Canada where there is no GPRS or 1xRTT (or even analog cell phone service). Wi-fi hotspots are everywhere in cities.

You can tell that the mobile phone providers feel threatened, by their enraged protest against municipal wi-fi networks.

Blackberry faces a different threat in the US. It'll be interested to see how that turns out.

Perhaps it'd be fairer if I simply said that both SORT and THREAD lack
the level of scalability found elsewhere in the protocol.

That claim has been made many times. I disagree. It depends upon how you look at it. I consider SORT and THREAD to be a form of SEARCH. If you need the sequence/UID map to resynchronize, UID SEARCH ALL is the fastest way to do that.

A more credible claim is that it's too costly to reTHREAD when new messages arrive. That's true; but nobody has yet come up with a good way to represent a delta thread.

It also needs to be pointed out that SORT and THREAD scale much better than the previous methods. Note that the NNTP community, which threads more than anyone else, still depends upon downloading overviews for all articles!

Polymer gets regularly tested against a mailbox with nearly 130,000
messages in.


Tested, yes; but what about usage?

There is a reason that users limit the number of messages in a single mailbox at a point much lower than their MUA is capable of handling. I don't think that this reason has received adequate study.

Hierarchies of categorization work better for most humans than a flat space (even with excellent searching tools).

FWIW, I regularly use Polymer on mailboxes with 70k messages in, as do
some of the other users. I do this reasonably often over GPRS, so I'm
putting my money where my code is, as it were.

Well, yes. A reasonable IMAP client should work well with a 70K message mailbox over GPRS, even if you start with a zero-state cache (pure online client). I designed IMAP to work well over 2400 bps, which is what much of my early work was over.

My observations suggest that people split up large message count mailboxes
into multiple smaller mailboxes long before they approach the message
count limits of their MUA.

That's largely because most MUAs rapidly degrade as the message count
increases, and users have been invariably trained to "sweep" messages
across.

That's true, but I don't think that is the only reason. I think that there is a degradation of human ability to deal with an excessively large flat space; and that humans impose hierarchical categorization before their tools for managing the flat space degrades.

OK, yes, there are still some MUAs that degrade in the 3-digit message count range. But I don't think that proves your point. If anything, the fact that such MUAs are still considered worthwhile indicates to me that there is something else going on.

Well, my personal target is 2400 baud. If I can get Polymer usable over
that I'll be reasonably happy.

Yup, that's what I did. The problem is, it's getting very difficult to justify forcing people to develop for that low a bandwidth, as it's getting harder and harder to find places which are that low.

Of course, a poorly-performing wireless WAN can have an effective speed that is that low... :-)

Well, yes, PINE's handling of IMAP is very good - as I'd expect. But
configuration and addressbook handling is weak, in my opinion, both in
terms of functionality and expressiveness. That's not meant as an
attack on PINE, just a demonstration that a "poor man's" method, while
sometimes "good enough", isn't "as good as it can be".

Understood. It is impossible in human endeavor to produce anything that some other person can't produce something better. I personally consider Pine to be an overall "best" among today's offerings, but that's because I give almost zero value to GUI. If I'm using a small screen, GUI objects take away valuable real estate that can present text. What's more, none of the GUIs that I've seen have been visually compelling.

As an aside, the problem with 14k4 modems wasn't the amount of data
transferred, per-se, but the time spent online, at least in the UK.
Polymer is asbolutely not designed to minimize the time spent online.
It's designed for circumstances where you're charged by transfer.

I suspect that both pricing models (time online or by transfer) are doomed in the face of inexpensive flat-rate.

Everyone liked, and still appears to like, the notion of ACAP. It's
just nobody actually wanted to take the plunge and actually use it -
just one conformant server and four MUAs, two of which are commercial
including Mulberry.


Agreed.  Sadly, I feel that ACAP attempted to bite off too much.  That
last 2% is invariably 98% of the work.

LDAP I'm not so certain has won the addressbook war - I think it's
managed a successful land grab, but failed to consolidate. It's used
heavily for canonical lists of address data, such as global
addressbooks and actual directories, but there seems little motion
toward using it as a personal addressbook. Moreover, I don't think
anyone's seriously considering accessing LDAP over a mobile link.

The consolidation (and especialy use for personal addressbooks) is going slower than I predicted, but the lumbering dinosaur is in motion. I'm going to attend a dinosaur-pushing meeting shortly (and hope that I don't get buried under dino-dung...) And yes, people are talking about using it over mobile.

Yes, I suppose it'd be cleaner to have "create submailbox" and "create
subcontainer" or something.

Or just follow IMAP's lead, in which you create mailboxes and directories are created implicitly.

Your mileage may vary, but I find it remarkable that people expect that a
mailbox should contain other mailboxes.  I haven't yet heard a groundswell
of support for C language source files to serve a second purpose as a
container of other source files.

I like the analogy, forced as it is, but remarkable or not, and even
expectation or not, some servers allow this, some don't, and there's no
easy way for a client to know which short of "suck it and see", which
doesn't strike me as a clean methodology.

I actually agree; well, not about the analogy being "forced", but everything else you said. IMAP failed to take a stand and instead tried to forge a wretched compromise. The two sides of the time have come to accept that it would have been better to have lost in favor of the other side.

The reason why I don't think the analogy is forced is that "dual-use" mailboxes require the assumption that a message (rather than the mailbox) is the terminal node of the hierarchy. However, a message is not in the hierarchical namespace; you must open the mailbox first, then use non-name operations to address the data. There's also no particular reason to call a message a terminal node rather than a particular body part; the "dual-use" model is based entirely on the mh/netnews/maildir form of store.

Those of us who dislike the "dual-use" model point to the fact that there is no single "open" method for a "dual-use" mailbox. You need to have separate "open as mailbox" and "open as directory" methods. Whether or not you accept this argument depends upon whether or not you consider a single open method to be important.

A better analogy of a dual-use mailbox is the two-fork files of the old Macintosh operating system, where a name could have both a data and a resource fork. Even then, a name could not be a folder (which is the correct term in the Mac world) and have data/resource forks; there was always a single open method.

Really there ought to be a CAPABILITY string or a NAMESPACE extension
string (or both) explaining what the hierarchy is actually capable of.

That is the wrong granularity, since the state is per-mailbox and it is
perfectly possible for a mail store hierarchy to have multiple types of
mailbox objects.

*nods* True. But in some cases, the behaviour is known on a
per-NAMESPACE or per-server basis.

Alas, not on the namespaces and servers where that knowledge matters most to the client.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
.

Relevant Pages

  • Re: One user having messages stuck in Local Delivery
    ... so you won't have any effect to the client PCs. ... migration of the Exchange data, ... brand spanky new intel rackmount server which has only been sitting here ... tomorrow morning another mailbox or two will start failing. ...
    (microsoft.public.windows.server.sbs)
  • Re: deleted inbox messages return on next login
    ... You must be using the 1970s vintage traditional UNIX mailbox format. ... between client and server into a transcript file. ... The transcript file should reveal what the client is not telling you. ... Liberty is a well-armed sheep contesting the vote. ...
    (comp.mail.imap)
  • Re: Considering returning to Eudora
    ... Don't you have to tell your other favorite client which messages to delete? ... Isn't it equally easy to tell Eudora which messages to delete? ... at the moment you delete them; periodic "compact mailbox" operations ... Does any OS automatically sort into file folders ...
    (comp.mail.eudora.mac)
  • Re: Mulberry gone, now what?
    ... but do you really want to thread 100K messages from a mobile client? ... >> to build such a mailbox.) ... > don't think that this reason has received adequate study. ...
    (comp.mail.imap)
  • Re: One user having messages stuck in Local Delivery
    ... user accounts and new client PC accounts, even if you keep all the naming ... is a retail install of SBS that I am troubleshooting not one from MSDN ... tomorrow morning another mailbox or two will start failing. ... and then I added them in my exchange connection in outlook and manually ...
    (microsoft.public.windows.server.sbs)