Re: [Gravity] Reconstruct thread

Alan LeHun wrote:

catwheezel says...

Slrn does it using the XPAT command on the server (so can't reconstruct a
thread if the server doesn't support XPAT). In essence, search articles
(in the current group) for those which have the current Message-ID in
their References header.

As has been said, that only recovers all the previous articles from one
branch of the thread. It doesn't reconstruct the /whole/ thread.

Once you follow the one subthread all the way up to its first or
originating post then you've found the originating post for ALL
subthreads. Then you search forward on the References. So how far you
can trace backward to get at the parent post(s) (search from References
to MID) will decide from where you can trace forward (search for MID in
References) to find the child posts. So why do you think that the
References header can only be traced backward? Why can't the MID in a
parent post be searched forward to find if it is in the References
header of the child posts?

Obviously you can only trace forward from the the furthest point back
that you can obtain a parent's MID in one of the subthreads; however,
from that point, you can search forward looking for the MID to show up
in the References header of another posts.

Odd how Dialog's reconstruct button works so well despite your claim.
You really want to base your expertise on a statement from chuckcar?

If you were to reconstruct this thread from this post, then the reply
that Jim gave to Frag, and the resulting posts in that branch of the
thread, would not be recovered.

Oh really?

Your References header:
References:
<MPG.26d4b9ecfb0514e5989684@xxxxxxxxxxxxxxxxxxxx>
<MPG.26d4d9693f797a419898da@xxxxxxxxxxxxxx>
<i4evpk$602$1@xxxxxxxxxxxxxxxxx>
<MPG.26d7a9b8821f9a0c9896d3@xxxxxxxxxxxxxx>
<slrni6rftc.lea.catwheezel@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx>

Search for the MID value of each post in your References header to see
if another post has that parent post's MID value in its References
header. If so, you have another subthread to track *forward* (searching
for MID in References). If not, there's no subthread started from that
parent post. For each of those posts listed in your References header,
check if it has a References header and continue tracing backward. Once
you hit a post with no References header, you've hit the root node of
the thread (as far as you can trace backward).

You think reconstruction only moves forward (to later posts) through a
subthread? You thought reconstruction only works on your local cache of
headers and posts? For reconstruction to work means tracing back to
rebuild the thread but searching for other posts that include the MIDs
of the parent posts in other [child] posts. Well, if you expired or
deleted the parent posts then you have to go retrieve their headers
again (or just the MID and References headers), and if you expired or
deleted the child posts of those posts then you have to wait until the
headers retrieved again get searched for the MID values from their
parents showing up in their Refs headers. Reconstruction isn't
instantaneous. It'll take some header re-retrieval and searching which
takes some time. A reconstruct in Dialog might take about a minute
during which there is a traffic between Dialog and NNTP server to
retrieve headers again.

Unlike what chuckcar claims and others seem to spew as their expertise,
reconstruction does not require storing all the headers for the parent
posts. It does mean, however, that the NNTP server still have them all
to re-retrieve. If the old parent posts aren't on the NNTP server then
obviously you cannot retrieve them to walk back to find all the parents
(and then walk forward to search for their MIDs to show up in References
headers of other posts). Of course reconstruction is going to require
re-retrieving headers from the server. If you already had them then you
wouldn't need to reconstruct the thread.

Since your newsreader doesn't have the parent posts anywhere, just where
did you think they would come from? If your newsreader doesn't support
the XPAT command to search through posts (to look for MID values listed
in References headers) or if your NNTP server doesn't support XPAT then
obviously you can't do the searching forward from a node found when
tracing backward using the References header alone. Tracing back
through the References header only gives you a single string going
backward and none of the strings going forward from any node. You need
to be able to search for each node in the traced back string to see if
their MID value is used in other posts' References header.

If you had those old posts then why would you even have to reconstruct
anything? It's when you do NOT have those old posts in your newsreader
that you have to trace back and search forward to rebuild the thread by
retrieving the headers again from the server. In Dialog, doing a thread
reconstruct generates network traffic to retrieve headers. For those
using Xnews, there isn't network traffic to the server to reconstruct a
thread? Just how could the XPAT command work without generating network
traffic? XPAT doesn't re-retrieve all headers. It doesn't re-retrieve
all headers other than those you still have in your newsreader. It only
returns the headers for the matching articles based on the criteria you
specified in the XPAT command. Doing a reconstruct in Dialog might take
a minute to search forward to find the MID values of parent posts found,
so far, in a tree walking algorithm for them used in References headers
of other posts. It does not take the long time to re-retrieve all
headers from the server for a newsgroup from scratch.

Some example articles mentioning XPAT:
http://www.newsgroupdownloaders.com/newsgroup-xpat-search/
http://www.thalunatik.com/NewsPro/XPAT.htm
http://www.mibsoftware.com/userkt/nntpext/0026.htm

You need a newsreader that can use XPAT while rebuilding a tree to
reconstruct a discussion. You need a server that supports XPAT
commands. If either is missing, yeah, then you have no means to search
forward on a MID value showing up in References header and you're stuck
with only tracing back along a single string using the References
header. I would think that any newsreader that offers the "reconstruct
thread" function would have to support the XPAT command. So if your
newsreader has a reconstruct, you're dependent on whether the NNTP
server will accept the XPAT commands. I suppose it would also depend on
how large a tree (max branches, max nodes per branch) that the
tree-walking algorigthm can handle that is used by the newsreader's
reconstruct function.
.

Relevant Pages

  • Re: [Gravity] Reconstruct thread
    ... The MID reference's are contained within the individual posts looking ... you must have the headers for every post and therefore you have ... present in the References header, you'd get back all of 18 articles, not ...
    (news.software.readers)
  • Re: slrn setup guide for Dapper
    ... How do you get slrn to do that? ... position of posts' References header (and having those threads appear at ... the top of your headers pane. ... posts, however far downthread from one of yours, that contain your MID ...
    (news.software.readers)
  • Re: Killfiling direct reponses to Jeff Relf
    ... killing Relf's posts. ... You asked for a filter that would kill replies to JR. ... It's not "anywhere in the headers", ... Is your understanding of the References header on par with JR's ...
    (news.software.readers)
  • Re: Killfiling direct reponses to Jeff Relf
    ... killing Relf's posts. ... Relf anywhere in the headers just yet, since that might be an overbroad ... You asked for a filter that would kill replies to JR. ... it's specifically on the References header. ...
    (news.software.readers)
  • Re: [Dialog] Option to retrieve bodies of new posts does not work reliably
    ... I enabled the option "Retrieve bodies for all new posts". ... after retrieving their headers, there are still many posts that are ... But those filters will fail (and ... download the bodies of all new posts. ...
    (news.software.readers)