Re: Many threads in Linux

Scott Gifford <sgifford@xxxxxxxxxxxxxxxx> writes:

Eric Sosman <Eric.Sosman@xxxxxxx> writes:

[...]

[ ... extremely helpful information elided ... ]

The rest of your post was extremely helpful, and I very much
appreciate all the time you've put into responding to my posts! I
will try some of your suggestions and report back what I find.

OK, here is what I found.

The limitation I was hitting was running out of PIDs. I increased the
maximum PIDs in the system and recompiled, and was able to get up to
about 60,000 threads before the Linux out-of-memory killer killed my
process. It happened when my process reached about 1GB of VM, and I
have my kernel configured with a 3G/1G memory split, so I will try
with a 2G/2G split and see if that makes a difference.

The test I'm doing is a very simple echo server. The server is in C
and uses pthreads, one thread per client. The client is in perl and
uses epoll. I've posted the code I used here (I modified it somewhat
to improve clarity, including increasing the stack size by 4KB to
allow printing error messages, so results will not be identical),
along with a description of the changes I made to my system and the
benchmark results:

http://www.suspectclass.com/~sgifford/manythreads/

Here are the results I saw:

THREADS VIRT MEM RES MEM REQ/S
------- -------- ------- -----
1000 18m 9m 35218
10000 159m 79m 35216
20000 317m 159m 33659
30000 474m 238m 33733
40000 632m 318m 35183
50000 789m 398m 28864
60000 947m 477m 28018

It does start to slow down a bit, but overall the performance is
not too bad.

The client and server ran on the same machine, which kept network
latency to a minimum, but meant they shared the system's 2 cores. I
didn't try to determine whether the performance was limited by the
client or server, but top reported that the client was using about 10%
more of the CPU than the server on all runs.

I will report back if I have luck with the 2G/2G split. In the
meantime, any thoughts are welcome, particularly if you see flaws in
my methodology or code.

----Scott.
.

Relevant Pages

  • Re: How long should statspack.snap take to run?
    ... While the default report has limited ... database seemed to be performing slower than expected. ... 2.2GHz Oracle server with a dual 3.66GHz Oracle server. ... a client computer problem and the finger is pointed at the computer ...
    (comp.databases.oracle.server)
  • Re: What doesnt lend itself to OO?
    ... >> proxy and instructs the server to constuct the real object. ... rather than client code. ... If 'clock' is instantiated in the server, ... > for the server interface at the OOA level. ...
    (comp.object)
  • This is going straight to the pool room
    ... or not the client has privilege to do what they're trying to do, ... The server environment is this: ... 3GL User action Routines that Tier3 will execute on your behalf during the ... Routine Name: USER_INIT ...
    (comp.os.vms)
  • [Full-Disclosure] R: Full-Disclosure Digest, Vol 3, Issue 42
    ... Full-Disclosure Digest, Vol 3, Issue 42 ... SD Server 4.0.70 Directory Traversal Bug ... Arkeia Network Backup Client Remote Access ...
    (Full-Disclosure)
  • Re: What doesnt lend itself to OO?
    ... > rather than client code. ... no way to do that without also touching the object with clock semantics ... will not encapsulate both clock semantics and network semantics. ... The server can do whatever it wants ...
    (comp.object)