Re: Mulberry gone, now what?


Mark Crispin wrote:
> On Mon, 10 Oct 2005, Dave Cridland wrote:
> >> True, but do you really want to thread 100K messages from a mobile client?
> > Why not? Polymer already handles 100K message mailboxes perfectly well
> > over low-bandwidth connections, even in a cold-cache situation. Why on
> > earth wouldn't I want to thread them too?
>
> As does Pine; but the issue isn't a matter of software capability. It's a
> matter of user managability. Most users who have that many messages (and
> these are typically archived messages, not new messages) break them up
> into separate mailboxes, usually named in some way to categorize the
> archive (date range is popular, as is topic).
>

I maintain that's learned behaviour, based on the inability of many
IMAP clients (and servers) to scale reasonably well. I'm also not
convinced that "most" is even accurate.

> Yes, you can do searches instead. But do you really want to scroll
> through 100K messages, and enter in searches through 100K messages, on a
> mobile client?
>

Well, I'm certainly not suggesting that a user of a mobile client is
likely to want to scroll through all 100k messages, but equally, you're
implying that organization of email into "chunks" by mailbox is a
workaround for the capabilities and user interfaces of clients.

> > (I don't see any problem with
> > handling much higher message counts, either, I just can't be bothered
> > to build such a mailbox.)
>
> OK, so your "can't be bothered" limit is at 100K. Very few people are
> there. Most drop out after 10K, with the peak being in the 4 digit range.
>

No, 100k is my "can't be bothered" limit for a test case. I suppose I
could just COPY 1:* to the same mailbox a few times, but that gives me
a very dull test mailbox.

> >>> As it stands, I think THREAD is too expensive for mobile clients,
> >>> useless for disconnected mode clients, and only useful for client which
> >>> aim to provide a snapshot view of a mailbox - basically meaning webmail
> >>> clients.
> >> Huh? THREAD works quite well for online clients!
> > Oh, across a LAN, yes.
>
> Or wi-fi.
>

Or indeed high bandwidth links in general.

> I think that the GPRS/1xRTT guys are rapidly losing their window of
> opportunity. At about the same time that I finally dropped CDPD in favor
> of 1xRTT, I observed that my need for WAN wireless had been plummeting
> like a paralyzed falcon. So far, I've used WAN wireless *once* this
> entire year.
>

I use it quite a bit, strangely - I find that the cost of WiFi access
points is quite high, and moreover they're not all that common here, in
the wilds of Wales.

Where GPRS wins out isn't so much in the laptop market, although that's
still important, but in the "email enabled mobile phone" market.

> > Perhaps it'd be fairer if I simply said that both SORT and THREAD lack
> > the level of scalability found elsewhere in the protocol.
>
> That claim has been made many times. I disagree. It depends upon how you
> look at it. I consider SORT and THREAD to be a form of SEARCH. If you
> need the sequence/UID map to resynchronize, UID SEARCH ALL is the fastest
> way to do that.
>

Actually, no. Polymer does use UID SEARCH for sequence/UID mappings,
but does so in small chunks, using a successive approximation technique
to find out which chunk most likely holds the UID it wants.

> A more credible claim is that it's too costly to reTHREAD when new
> messages arrive. That's true; but nobody has yet come up with a good way
> to represent a delta thread.
>

Yes, it's a tricky problem.

> It also needs to be pointed out that SORT and THREAD scale much better
> than the previous methods. Note that the NNTP community, which threads
> more than anyone else, still depends upon downloading overviews for all
> articles!
>

Oh, there's no argument there. It just doesn't scale well enough, yet.

> > Polymer gets regularly tested against a mailbox with nearly 130,000
> > messages in.
>
> Tested, yes; but what about usage?
>
> There is a reason that users limit the number of messages in a single
> mailbox at a point much lower than their MUA is capable of handling. I
> don't think that this reason has received adequate study.
>
> Hierarchies of categorization work better for most humans than a flat
> space (even with excellent searching tools).
>

The OSAF have done some work in that area. I'm not sure how much
academic rigour has gone into it, but certainly the rise of multiple
hierarchy models, tagging, and attribute matching, does appear to
suggest that while a single taxonomy - a folder tree - works well for
technical types, this is a learned skill much harder to acquire than
simple tagging, which has, after all, been present in IMAP for over a
decade.

> > FWIW, I regularly use Polymer on mailboxes with 70k messages in, as do
> > some of the other users. I do this reasonably often over GPRS, so I'm
> > putting my money where my code is, as it were.
>
> Well, yes. A reasonable IMAP client should work well with a 70K message
> mailbox over GPRS, even if you start with a zero-state cache (pure online
> client). I designed IMAP to work well over 2400 bps, which is what much
> of my early work was over.
>

I know - and indeed you know - at least one person who regularly reads
email over a 9600bps connection, in fact. He's using the free minutes
on his mobile to avoid bothering with a DSL line.

> > Well, my personal target is 2400 baud. If I can get Polymer usable over
> > that I'll be reasonably happy.
>
> Yup, that's what I did. The problem is, it's getting very difficult to
> justify forcing people to develop for that low a bandwidth, as it's
> getting harder and harder to find places which are that low.
>
> Of course, a poorly-performing wireless WAN can have an effective speed
> that is that low... :-)
>

I'm not thinking that anyone's ever likely to use Polymer, or its
underlying library, over 2400bps.

However, I figure that if it's usable over 2400bps, then using it over
9600bps - a GSM data call, about the most reliably obtainable bandwidth
- is going to be painless, and using it on GPRS is going to be
excellent.

> > Well, yes, PINE's handling of IMAP is very good - as I'd expect. But
> > configuration and addressbook handling is weak, in my opinion, both in
> > terms of functionality and expressiveness. That's not meant as an
> > attack on PINE, just a demonstration that a "poor man's" method, while
> > sometimes "good enough", isn't "as good as it can be".
>
> Understood. It is impossible in human endeavor to produce anything that
> some other person can't produce something better. I personally consider
> Pine to be an overall "best" among today's offerings, but that's because I
> give almost zero value to GUI. If I'm using a small screen, GUI objects
> take away valuable real estate that can present text. What's more, none
> of the GUIs that I've seen have been visually compelling.
>

I have been toying with the idea of a TTY variant. Oddly enough, my
biggiest sticking point is a name. :-)

> > As an aside, the problem with 14k4 modems wasn't the amount of data
> > transferred, per-se, but the time spent online, at least in the UK.
> > Polymer is asbolutely not designed to minimize the time spent online.
> > It's designed for circumstances where you're charged by transfer.
>
> I suspect that both pricing models (time online or by transfer) are doomed
> in the face of inexpensive flat-rate.
>

No, by-transfer is showing a marked increase in the UK. GPRS is charged
entirely by transfer, and some DSL models charge by transfer over a
certain limit.

> > Everyone liked, and still appears to like, the notion of ACAP. It's
> > just nobody actually wanted to take the plunge and actually use it -
> > just one conformant server and four MUAs, two of which are commercial
> > including Mulberry.
>
> Agreed. Sadly, I feel that ACAP attempted to bite off too much. That
> last 2% is invariably 98% of the work.
>

MAKECONTEXT with DEPTH is the tricky one. That and by-class/byowner
namespace duality. The rest is actually quite easy. Even making it
scalable is, essentially, easy, I just haven't had the time. (Oh, I
tell a lie - it's quite tricky figure out which datasets you need to
hold a lock on for a STORE.)

> > LDAP I'm not so certain has won the addressbook war - I think it's
> > managed a successful land grab, but failed to consolidate. It's used
> > heavily for canonical lists of address data, such as global
> > addressbooks and actual directories, but there seems little motion
> > toward using it as a personal addressbook. Moreover, I don't think
> > anyone's seriously considering accessing LDAP over a mobile link.
>
> The consolidation (and especialy use for personal addressbooks) is going
> slower than I predicted, but the lumbering dinosaur is in motion. I'm
> going to attend a dinosaur-pushing meeting shortly (and hope that I don't
> get buried under dino-dung...) And yes, people are talking about using it
> over mobile.
>

Really? Randall Gellens has some interesting data about that, relating
to OTAPA.

> > Yes, I suppose it'd be cleaner to have "create submailbox" and "create
> > subcontainer" or something.
>
> Or just follow IMAP's lead, in which you create mailboxes and directories
> are created implicitly.
>

Yes, true, but that's a pain to do in a GUI as well.

> >> Your mileage may vary, but I find it remarkable that people expect that a
> >> mailbox should contain other mailboxes. I haven't yet heard a groundswell
> >> of support for C language source files to serve a second purpose as a
> >> container of other source files.
> > I like the analogy, forced as it is, but remarkable or not, and even
> > expectation or not, some servers allow this, some don't, and there's no
> > easy way for a client to know which short of "suck it and see", which
> > doesn't strike me as a clean methodology.
>
> I actually agree; well, not about the analogy being "forced", but
> everything else you said. IMAP failed to take a stand and instead tried
> to forge a wretched compromise. The two sides of the time have come to
> accept that it would have been better to have lost in favor of the other
> side.
>
> The reason why I don't think the analogy is forced is that "dual-use"
> mailboxes require the assumption that a message (rather than the mailbox)
> is the terminal node of the hierarchy. However, a message is not in the
> hierarchical namespace; you must open the mailbox first, then use non-name
> operations to address the data. There's also no particular reason to call
> a message a terminal node rather than a particular body part; the
> "dual-use" model is based entirely on the mh/netnews/maildir form of
> store.

Yes, sure. But IMAP allows dual-use to happen, and yet cannot signal
the client as to what is going to happen.

C: A01 CREATE foo/
S: A01 OK I did something, now guess what.

> >>> Really there ought to be a CAPABILITY string or a NAMESPACE extension
> >>> string (or both) explaining what the hierarchy is actually capable of.
> >> That is the wrong granularity, since the state is per-mailbox and it is
> >> perfectly possible for a mail store hierarchy to have multiple types of
> >> mailbox objects.
> > *nods* True. But in some cases, the behaviour is known on a
> > per-NAMESPACE or per-server basis.
>
> Alas, not on the namespaces and servers where that knowledge matters most
> to the client.

Hmmm.

C: A01 CREATE foo ((TYPE DUAL))
S: A01 NO Cannot create dual-use mailboxes.

But in any case, for servers that only support either all-dual or
no-dual, a CAPABILITY string or NAMESPACE string would avoid the wasted
command.

Dave.

.

Relevant Pages

  • Re: Mulberry gone, now what?
    ... But do you really want to scroll through 100K messages, and enter in searches through 100K messages, on a mobile client? ... and only useful for client which aim to provide a snapshot view of a mailbox - basically meaning webmail clients. ... There is a reason that users limit the number of messages in a single mailbox at a point much lower than their MUA is capable of handling. ... The reason why I don't think the analogy is forced is that "dual-use" mailboxes require the assumption that a message is the terminal node of the hierarchy. ...
    (comp.mail.imap)
  • RE: SBS 2003 and Service Pack 2 - Issues with Exchange OMA
    ... I forgot to add to my previous post that all mobile clients are sync'ing ... Did the problem happen just after you applied Exchange Server 2003 SP2? ... ActiveSync settings on both Windows XP client computer and the Windows ... user mailbox, I suspect that the mailbox is corrupted. ...
    (microsoft.public.windows.server.sbs)
  • Re: Send autoconfirmation
    ... but since multiple people open this mailbox it seems to stop sending the ... relpy for some reason. ... You can either do this in the client or you can do it ...
    (microsoft.public.exchange2000.admin)
  • Re: Mobile Admin findet Accounts nicht
    ... XMS2K3, Windows Mobile 6, HTC. ... Danach das Gerät einem anderen User (User2) zugeordnet. ... "Failed to access user's Mailbox, verify that the mailbox and user account are enabled and functional. ...
    (microsoft.public.de.exchange)
  • Re: ActiveSync problem with digitally signed messages
    ... Before moving mailboxes to Exchange 2007 it was possible to read signed mails on the mobile devices... ... Before the migration we had Exchange 2003 Mailbox and Front End Servers. ...
    (microsoft.public.exchange.admin)