Re: Multiplatform threadsafe reference counter

Markus Elfring wrote:
However, I was specifically looking for something that doesn't require
POSIX/PTHREADS
(I need to add them on an higher layer).

Are semaphores an appropriate alternative for you? ;-)
http://wefts.sourceforge.net/wefts-apidoc-0.99c/classWefts_1_1Semaphore.html

Heheh, yes they would indeed be OK. Just, I don't want a threading
library in the core engine of my language because I want embedding apps
not to be require to link against it. Even if that library is mine ;-).

Also, under Linux and solaris Wefts uses PTHREADS... so it would be ok
for the higher level module, but not for the engine for the above reason
(yes, even if pthreads are POSIX standard; you may have now and in
future many different implementations on the same system, and I don't
want an embedding application to end up linking (just in example) NTPL
while I am trying to stuff in linuxthreads... ).




The core of my application should be free from hypothesis on threading.

How much do you follow discussions about memory models for programming
languages?

Not very much. Except for C++, it doesn't seem to have been very
productive up to date (and someone may argue that also C++ evolution is
still requiring some finetuning).

Let's say I observe it.

I much appreciate C# approach, which seems to mimic posix semantics
correctly wrapped in language structures.

Any hint is helpful; I never looked down on any approach. Just, I have
still not found anything that was fitting my concerns and needs.



Otherwise, I am planning to give my engine an "atomic provider" that can
be configured live; it may be a function pointer or a class with virtual
inheritance.

I am also curious on thread-safety issues for this detail aspect.

Well... that is pretty thread safe. Everything (i.e. the atomic
provider) is guardanteed to be setup before the first thread can be
started. Also, refcounts in my machines are safe; reffing is extremely
limited (a couple of objects in the whole system), and there is no
possible destroy-while-increffing race condition.

My only concern is about having a contemporary read-inc-write access
from two separate CPUs on some those exact couple items. Namely, one is
the reference count for the read-only module description structure
(which can be shared across CPUs), and the other is the refcount for the
error-on-exit instance, in case a threaded VM terminates with error.

Objects whose lifespan can be granted holds references to the items I am
concerned about, so the only problem may be a concurrent write on a
spurious read, which may mess up the reference count. In other words,
the only problem is that of being sure that reference counts are
correclty perfomred and that they are atomically visible across CPUs.

As I don't want to bring library-specific functions in the engine, I'd
prefer to avoid using pthreads directly there.

So; alternatives are 2: either having a system-but-not-library specific
"visibility propagator", as the lockfree libraries suggested by Chris,
and that can be used directly in the engine without breaking its
"library neutrality", or having the extension dealing with threading
libraries to setup its own idea of "threadsafe reference counting".

System-specific is OK, as I can have different implementation modules on
each system, but library-specific is not OK, as I may force choices on
embedding applications, and that must be avoided.



int var = 0;
// no external libs, only syscalls/asm allowed.
some_nice_atomic_inc( &var );

How far do you get with the data type "sig_atomic_t" for this use case?


Not very much; that type ensures you are correctly aligned and that
read-writes are atomically perfomed in one single CPU step. If the
process receives a signal while writing a sig_atomic_t, you are sure
that the data is either written or not-written; you can't possibly have
just part of it (i.e. just LSB) written.

The problem I want to prevent is different. When two threads perform a
read-increment-write operation from two different CPUs, they may see the
same number (say zero), increment (and obtain, say, 1) and then write
their local idea of the incremented number. This results in an
underextimation of the real holders of a reference, with possible early
deletion that will cause crash in a far, very far region and moment in
the program.

I think also that, without proper memory visibility semantics, different
CPUS, having chached the memory in slightly different moments in time,
may also see non-updated data.

I.e; considering unfenced data access:

Data in Memory: 0

Thread1:
read (0)
inc (1)
write(1) >> in cache, but still not flushed

....very small interval...

Thread2:
read (0) << from cache, still not from memory
inc(1)
write(1)

while holders of the resource are actually 2.

But wether this "visibility" problem is a real problem or not on my
target architectures, the overall read-inc-write problem stays under any
architecture, and requires proper lock-free algorithms or lock semantics:

Thread1: Thread 1:
read (0) -
inc (1) read(0)
write(1) inc(1)
- write(1)


Said this; practically nothing is reffed in my programming models, and
this is expecially true for the language I am developing, and I may also
remove the reffing for those things I still reffed. Either cloning of
the termination-error item, which is flat and quite small (possibly in
the order of 100 bytes) and enforcing of outer referencing of the
read-only module definition structures (i.e. forcing them to be held
safely in the embedding apps) are viable solutions. Just, refcounting
them is simpler and more efficient; so, if there is a protable way to do
threadsafe incref-decref, I'd prefer it.

Sorry if I bombed you with lot of extra infos, partially irrelevant; and
maybe I confused a bit "generic" and "specific" scopes, but I know
you're in the field, so you have very probably an idea of what I am
doing and why.

Bests,
Giancarlo.

.

Relevant Pages

  • Re: Good Dungeon Mapping e-Tool?
    ... Relatively recent Linux kernels and GNU libc have gotten most ... pipelined across multiple thread stages. ... Given two libraries that both want to ... I think you're mixing up dynamic loading and threading here. ...
    (rec.games.frp.dnd)
  • Re: Good Dungeon Mapping e-Tool?
    ... and signals are now threadsafe. ... Given two libraries that both want to ... Under Linux, with its lightweight process model, this is virtually the ... I think you're mixing up dynamic loading and threading here. ...
    (rec.games.frp.dnd)
  • Re: Walkthrough Instruction Error
    ... What I said was a VB6 app can do threading using activex.exe but that is out of process. ... What Olaf implied was that his external library written in some other language isn't out of process, yet what he did say was his threading was not light weight and that his library uses pipes. ... What's Olaf's libraries do *Exactly* is hard to tell because he won't put the source code out there, but we do know there is also marshalling occurring. ...
    (microsoft.public.vb.general.discussion)
  • Re: Walkthrough Instruction Error
    ... The msdn library documents much about services: ... BTW I knew that there were libraries available to enable doing this stuff in VB 6, ... seldom-used VC++ books on Windows programming and actually read up on the dirt on programming these things in VC++ and then see if I can translate soem of that into VB. ... Those who needed threading with VB6 always found ...
    (microsoft.public.vb.general.discussion)
  • Re: Joel hammers the final nail into Lisps coffin
    ... and in FreeBSD it's CMUCL. ... threading code we had to write, reddit would not run on my Mac, and I ... was the lack of widely used and tested libraries. ... Inexperience is also appearance in the ...
    (comp.lang.lisp)