Re: Pthread fd and memory leak
- From: CJ <spambox1@xxxxxxxxxxxxxx>
- Date: Fri, 26 Sep 2008 08:29:02 -0700 (PDT)
On Sep 12, 4:10 pm, Steve Watt <steve.removet...@xxxxxxxx> wrote:
In article <5f13d341-b633-4a6e-a34f-435f298f9...@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx>,
CJ <spamb...@xxxxxxxxxxxxxx> wrote:
I have created a pipe-line of threads to manage processing TCP/IP
connections. It seems an fd (file descriptor of a socket) created in
one thread and closed in another is not returned to the process, and
not reused.
The first thread manages a listen socket passing the connected
(accept) sockets to another thread via an unnamed pipe, a worker
thread. The worker thread "processes" the connection, shuts it down
(shutdown), and closes the socket file descriptor (close). When the
server thread accepts the next connection the new fd is one greater
than the previous connection, a classic fd leak.
Wow, passing file descriptors over pipes between threads. What a
strange concept. Are you sure you're using threads? If so, why
go to all the trouble?
When you send an fd over a pipe, the fd is still open in the sending
process. The receiver will get a new fd in their own descriptor
space that also points to the same open file description.
To "correct" the leak, I created another unnamed pipe and the worker
thread sends the "processed" socket fd back for closing. The server
reads from the pipe and closes it. This seems to "fix" the problem,
but raises the following question.
Which still shouldn't have fixed the problem.
Think of an fd pass through a pipe to another thread in the same
process as exactly equivalent to calling dup(fd). Now you have
two fds to close.
Are fds shared by all threads of a process or unique to the thread
that "created" them?
Shared.
I am also finding that allocated heap memory created in one thread and
passed as a pointer value via an unnamed pipe to another thread has a
similar problem. The receiving thread can access and manipulate the
memory at the pointer value and free it without errors, but the memory
is never reclaimed causing a memory leak. Same question: is heap
memory shared by all threads within a process or unique to the thread
that allocates the memory?
Almost the same answer: Shared. However, there are some heap managers
that optimize for the case of free()s from the same thread that did
the allocation. Or you could just be seeing a leak in your own code.
Any thoughts? Any sites that provide specifics to what is and what is
not unique to a pthread would also be helpful.
Is there some reason for using pipes between threads? It seems like
a strange design.
--
Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.5" / 37N 20' 15.3"
Internet: steve @ Watt.COM Whois: SW32-ARIN
Free time? There's no such thing. It just comes in varying prices...
Yes there is a method to my madness and a reason for passing messages
"downstream" to processing threads. The reasons are similar to the
design of STREAMS, which I'll assume all the helpful posters grasp
already and will not explain.
Thanks for all the helpful posts. My (bone head) problem has been
found and turned out to be a memory error. Why the incorrectly
unreleased memory caused the problem (without any out-of-memory or
segv signals) is unknown.
cj
.
- References:
- Pthread fd and memory leak
- From: CJ
- Re: Pthread fd and memory leak
- From: Steve Watt
- Pthread fd and memory leak
- Prev by Date: Linux threading: question from a real beginner...
- Next by Date: Re: pthread_create C++ and variable problems
- Previous by thread: Re: Pthread fd and memory leak
- Next by thread: how to ensure other threads load new value from memory
- Index(es):
Relevant Pages
|
Loading
