Re: RFC - New Object-Oriented Method of Parallel Programming (Shorter Version)
- From: "Ross A. Finlayson" <raf@xxxxxxxxxxxxxxx>
- Date: 15 Apr 2006 10:40:33 -0700
Slawomir J. wrote:
For those who can spare the time to look at it - I would appreciate all
comments and suggestions.
Now it includes a very short - programmer model only - specification:
http://www.stressflow.com/cpt_introduction.htm
Thanks in advance
info@xxxxxxxxxxxxxx
--
Hi Slawomir,
It seems you've put lot of thought into that. In balance, I can tell
you I haven't put much thought into this.
I am recently afforded the opportunity to work with some big iron, and
I think it's interesting. It's basically a commodity cluster using the
Sun Grid Engine to batch processes using the Message Passing Interface,
MPI.
Something like that really opens a lot of ideas to the casual
programmer, and even to the lazy one like myself.
It's interesting to write some toy code implementing some perceived
patterns. Basically I define a variety of roles for the nodes on the
MPI BSP, Bulk SCheduling Processor. I just learned that term from
Radu Calinescu's "Architecture-Independent Loop Parallelisation", what
with the s for z appearing British as it as well appears to come from
the BSPLib people, who don't appear to have updated their web page
since 1996 and are quite proud of that.
So, I define roles along the lines of "Control", "Data", "Result", and
"Worker" nodes. Then, each process has a bitmask of the roles it is to
portray or fulfill. "Fill my role, loop de loop" - David Lee Roth.
Anyways, then I started categorifying messages, basically to around
fifteen or twenty messages,
typedef enum sr_message_t{
INSTRUCTIONS,
INPUT_FILE_NAME,
WORKER_TOKEN,
READY_TO_WORK,
NO_MORE_WORK,
ALL_WORK_ALLOCATED,
ALL_WORK_COMPLETED,
INPUT,
RESULT,
DATA_COLLECTION_COMPLETE,
DATA_DELETE,
EXIT,
IMPLEMENTATION_LIMIT,
ERROR,
SENDER_ROLE_MASK,
USER_DATA_MASK
} sr_message_t;
Then, those enumerated values are to have literal values that reflect
the roles of sender and receiver, as a message in the message passing
interface appears to be a block of data and then one allowable integer
of indicator, the tag of the message. I'd like to see lists of MPI
tags of working apps.
Then, I got to looking at MPI, as prototyped in mpi.h. It seems that
the async message sends require the buffer to be unmolested. This is
where I basically want to send quite a few lightweight messages, so I
figure something like the control node which helps coordinate the
workers and data nodes and result collecting nodes and assigning the
nodes to shift roles and so forth will have a pool of buffers, and that
way it can send messages using any of a variety of them, and then on
testing their receipt flagging them for reuse, all asynchronously to
_avoid_ the barrier synhronization as much as possible, with the
receivers having as well a pool of receipt buffers.
The data dependency issues in the parallelism, it seems I have read
about them in the compiler design books, with the register coloring and
so forth. With these modern processors with some 30 or more general
purpose registers already and hooge caches, learning how to implement
algorithms for the processor first might be wiser than learning how to
implement the parallel algorithms on the shifting sands of the
heterogenous networked environment, ie PVM, one of the many MPI
implementations, MPI, etcetera, with various cluster type batching
programs or PBSs. Having so many general purpose registers kind of
defeats the purpose of small function bodies, but that is what
compilers are for.
I got to thinking about the INSTRUCTIONS message above, and what the
message might contain. This is where I want to dynamically effect the
program instructions so that as apps are instrumented, profiled, and
checkpointed, that the algorithm selection and work assignment would
vary based upon the processing requirements towards the
feedback-directed optimization while not having to reinstall apps on
the server. I got to thinking that embedding a Java virtual machine
might be a good way to go, what with the profiling Java virtual
machines and their ability to load byte code over the network. Aglets,
ObjectSpace Voyager, Mole, ..., are/were Java distributed mobile code
platforms, but not necessarily about the parallel processing.
I hope you can refer me to some well-implemented MPI apps in C. I hear
from well-informed people things along the lines of "masterless nodes
are better" and "all portable software parallel app frameworks haven't
left the ground." I'd like to get your opinion on how i should go
about implementing parallel processing applications, for 1 to n nodes.
I've been looking at the the quite terse MPI 2.0 spec, the MPICH web
pages and their nice tutorials, mpi.h, I again hope you can refer to me
some well-designed and functional MPI/BSP apps to examine.
Ross F.
--
.
- References:
- Prev by Date: Re: Channels?
- Next by Date: Re: Channels?
- Previous by thread: RFC - New Object-Oriented Method of Parallel Programming (Shorter Version)
- Next by thread: Re: Virtual Shared Memory Cluster
- Index(es):
Relevant Pages
|
