Re: [NGET] bad_alloc with large newsgroup
- From: Sano <me@xxxxxxxxxxxxxxxxx>
- Date: Wed, 24 Oct 2007 11:44:58 -0000
- lee <empty@xxxxxxxx> - wrote in
news:slrnfhtggc.7ud.empty@xxxxxxxx:
Thanks for the links Mark.
They were useful pointers (memory) but it is a bit out of my
league to fiddle with kernel parameters.
Instead I have shuffled around my partitions and allocated a much
larger swap partition (8gb) instead of the previous 750mb.
After another long download to create the cache, nget simply
"died" without bringing any error and of course without creating
any cache.
It is right at the end when it is sorting the cache that it has
problems. TOP showed that the old swap partition was full - so now
it has plenty of room there.
Unfortunately, nget, like slrn does not use any temp files - they
rely on memory.
From a recent post:
XNews on a winbox-
1.2 millions headers is roughly 314MB, so if one d/led _all_ the 22
million in ohmster's example .divx ng, that might touch 6GB of headers.
No?
With the huge binary ng's that now exists nget has its problems -
though it is only a problem when one first starts using a ng. Once
the cache has been created it is from then on just a matter of
updating it - and now with the large swap it no longer crashes.
I've had similar problems with slrn with these groups - it just
calmly announces that it is going to "kill" itself when it has
nearly finished scoring, scoring and linking the headers.
With slrn one has to hence limit the number of heads one loads at
one time - I keep it to 500,000.
With nget I shall just have to cancel it with control-c after it
has been running a certain amount of time - it seems to
accept that (it calls it signal 2) and then goes on to
successfully terminate with the headers that it has already
downloaded.
Thus nget should ask, like slrn does, how many headers one wants
to download to the cache - rather than attempting to download the
whole lot.
Again - thanks for the links.
Lee
On 2007-10-22, Mark Blain <mblain2@xxxxxxxxx> wrote:
In the dim dark past I used the Win32 versions of both nget and
newsplex, but never for groups anywhere near that large. There
are nget forums tied to its sourceforge page, but activity there
looks pretty low. https://sourceforge.net/forum/?group_id=3121
The closest thing to an answer I found in a direct search was
this: http://archive.netbsd.se/?ml=port-amd64&a=2005-12&t=1605306
.
- References:
- [NGET] bad_alloc with large newsgroup
- From: lee
- Re: [NGET] bad_alloc with large newsgroup
- From: Mark Blain
- Re: [NGET] bad_alloc with large newsgroup
- From: lee
- [NGET] bad_alloc with large newsgroup
- Prev by Date: Re: New to Xnews
- Next by Date: Re: Question?
- Previous by thread: Re: [NGET] bad_alloc with large newsgroup
- Next by thread: [xnews] Access Violation error and cannot save setup changes.
- Index(es):
Relevant Pages
|
