Re: Perl hashing speedup ?



On Mon, Sep 04, 2006 at 06:20:21AM -0700, sujit wrote:
This is regarding perl hashing. I am creating a 3 level hash (actually
2 such hashes at a time) with approx. 2.6 million entries.

In case you need any additional information about memory usage,
Memory used by process - 26 MB (Approx)
Virtual memory usage - 720 MB (Approx)
RAM available on machine - 512 MB

Under the suspicion that the cause for slowdown is that you have reached
your memory limit, I'd suggest trying a smaller hash:

http://search.cpan.org/dist/Tie-GHash/

Note that according to the documentation, the actual access is *slower*
than a Perl native hash, but if it prevents you from hitting the disk,
it may be worth it many times over.


--
Gaal Yahas <gaal@xxxxxxxxxx>
http://gaal.livejournal.com/
.



Relevant Pages

  • OutOfMemoryException: Webservice, DataSet and file transport
    ... For transportation of a file, approx 850kB, I use a webservice which return ... The memory usage ...
    (microsoft.public.dotnet.framework.compactframework)
  • Re: FTS query performance on SQL 2005
    ... This is a new SQL install. ... 12GB and a full-text index with files of approx. ... When I look at task manager for memory usage, ... number of queries, these numbers steady out to the ones listed above. ...
    (microsoft.public.sqlserver.fulltext)
  • Re: Issues With Exchange 5.5
    ... Grows in size, or in memory usage? ... Exchange will use as much memory as it ... > The backup went through fine. ... I estimate that it has grown approx ...
    (microsoft.public.win2000.general)
  • Re: 32bit vs. 64bit memory usage
    ... Thanks Simon. ... 32bit vs. 64bit memory usage ... We encountered some problems with Oracle 9i. ... If we switch back to 32bit, and bring the app up, We have approx. ...
    (AIX-L)
  • Re: How memory stacks up.
    ... > However, by default, ArrayList instantiation, it allocates space for 16 ... > entries. ... So I'd have 16*4*1000 bytes memory usage right out of the box, ...
    (microsoft.public.dotnet.languages.csharp)