Re: Cluster computing drawbacks


In article <87br4pwswm.fsf@xxxxxxxxxxxxxxxxx>,
Ketil Malde <ketil+news@xxxxxxxxx> writes:
|> nmm1@xxxxxxxxxxxxx (Nick Maclaren) writes:
|>
|> >> Sure, but the claim wasn't that it was impossible, just that it
|> >> involved more work for the user.
|>
|> > And that is the claim that I am stating you have not justified.
|>
|> Okay, here's one example: TGICL, freely downloadable from NCBI. Run
|> on a single computer: ./tgicl file Run on an SMP: ./tgicl -c N file
|> (where N is the number of parallel processes to spawn, not unlike -j
|> for GNU Make). This is trivial to do. How would you go about running
|> it on a batch-queued system?

However, you will very often find that the trivial command doesn't
do what you think that it does, because managing SMP systems isn't
a trivial task - let alone tuning for them! Many people who have
done that have found other people beating down their office door
and threatening mayhem.

To answer your question, on our system, do the following once:

cat > fred
#!/bin/sh
../tgicl -c $NSLOTS file
<control-D>
chmod +x fred

And then, for each run:

qsub -Q s96 fred

If you want to run N copies, each using one process, then change
the ./tgicl line to:

mpirun mpi_spawn ./tgicl file

The use of $NSLOTS is automatic with my mpirun script. More
generally:

mpirun -np $NSLOTS mpi_spawn ./tgicl file

|> >>>> certain type of CPU interconnect. SSI clusters may qualify.)
|>
|> >>> Server Side Includes?
|>
|> >> Single System Image.
|>
|> > Boggle. Does anyone still sell them?
|>
|> Uh, somebody mentioned VMS, and I thought that was their term. Sorry
|> if I was mistaken.

It may be. I have managed a single system image, distributed memory
Unix system (the Hitachi SR2201), but damn few people can say the
same - they are more common than hen's teeth but ....


Regards,
Nick Maclaren.
.