Re: Design Questions on Termination

On Dec 13, 3:01 pm, Eric Sosman <esos...@xxxxxxxxxxxxxxxxxxxx> wrote:
There are two principal difficulties: (1) Most Pthreads
operations are off-limits to a signal handler, and (2) it can
be hard to control which of the program's many threads handles
a given signal.

Thanks for this clarification. Actually, it proved that I understood
something awfully wrong: I thought that when a signal occurs, all
select() operations would be unblocked in all threads. Obviously, this
is not the case.

The prime use case, I thought, of the signal was to break out of the
blocking operations. This (now I see) obviously is nonsene and so I do
not need signals at all for my implementation. Except, unfortunately,
SIGHUP, which I need to preserve for historical syslogd compatibility
- it is used to request a config reload request. I can block signals
for all threads but a housekeeping thread, so this is fine. Thanks a
lot for this advise (and those of the others), it gets me closer to
understanding all of my options.

But now let me ask another question - as I said, I'd like to know all
about my options (from an experience point of view that you guys
obviously have) before I settle down for a solution.

The self-pipe trick seems to play very nicely with select() and I
assume it does with poll(), epoll() etc. For sleeps, I can also use
select(), so it plays nicely with that too. What I am now looking for
is a generic approach on how to terminate blocking operations. Let's
for example say I'd implement tls secured connections via openssl. A
simple way to do that is by utilizing blocking openssl operations. But
this means my thread will block in the openssl read/write. So if I now
get a termination request, how do I gracefully shutdown that thread? I
know that I can simply cancel it, but that is not my understanding of
graceful termination... (e.g. I'd like to do connection cleanups,
whatever - the actual thing is not of relevance here).

What I can envision is to do the following: I have one thread (let's
call it Twork) that does all of the modules work, except for the
blocking operation. That thread has associated a condition variable
(CTwork). This condition variable is also known to a high-level thread
that does app-internal housekeeping - the one that receives the
shutdown request. Let's call that thread Thk. When thread Twork
initializes, it creates another thread (called Tio) that does nothing
but the blocking IO. Tio also has an associated condition variable
(CTio). Its pseudo-code is as follows:

while(1) {
select(params placed by Twork in shared memory)

It is more or less a wrapper for the select. The reason for it is to
make sure my module worker thread isn't blocked. Of course you can
replace the select() with any other blocking operation (e.g. openssl

The pseudo code for Twork is as follows:

While(1) {
Do some work, eg. fill select params
If(termination flag set) {
kill thread Tio
Do graceful shutdown
Exit thread
Do work

The point here is that Twork waits on CTio, which is signaled when the
blocking operation is finished. Now comes the termination criteria:
When housekeeping thread Thk receives the termination request, Thk
first sets the termination flag and then signals CTwork, too. Thus,
upon termination Twork awakes, detects it is requested to terminate
and then "cancels" the blocking request and finishes graceful.

I think that approach will work. However, it has the drawback of
additional complexity and I am also concerned about the overhead
involved. After all, I need to at least do two additional context
switches to activate Tio and come back after blocking call completion.
Also, the minimum number of threads per module doubles. I always need
at least two, because one is exclusively reserved for the blocking
operation. This all just to handle a quite exceptional situation...

How do you feel about this approach? Any feedback is highly
appreciated. And if I didn't express myself clearly enough, please let
me know.

Thanks again,