Re: Cluster computing drawbacks

In article <1kmFe.28938$0f.12173@xxxxxxxxxxxxxxxxxxxx>,
Randy <joe@xxxxxxxxxxxxxxx> wrote:

>Interesting. I'd like to know more about the HW used to generate the
> Pathscale 1.31 us number. The fact that it appears to beat the Cray
>XD1's MicroArray implies a connection that exceeds the performance of
>any available or next-gen Opteron bus.

No, that's not the implication. The Opteron bus is not a bottleneck in
this situation. The XD1 uses an FPGA and we use an ASIC, and we build
our interconnect software with the best Opteron compiler (our own).
>From the XD1 dual-port bandwidth numbers it's clear that they're using
8-bit HT and we're using 16-bit HT, but that doesn't affect the Random
Ring latency much.

>BTW, 1.31 us is 1310 ns. According to SGI, the Altix's cache line
>latency (128 bytes) within a C brick is 145 ns, or 9 times faster than
>1310 ns (0 bytes).

That's an awfully small system -- that 1.31 number is good up to 12
nodes/24 cpus. BTW, SGI's numbers are all optimistic and depend on the
fact that only one CPU is moving a line, not all the cpus in the
system. That's why Random Ring Latency is a good benchmark, it uses
all the cpus in all the nodes in a system.

> Worst case latency on a 512 PE system is 800 ns on
> a NUMA Link 3, or about 640 ns on a NUMA Link 4, which still beats
> 1310 ns handily.

You're comparing apples and oranges. Try testing it with all the cpus
getting a line, and you'll find radically different numbers.

>That introduces an interesting point... that commodity cluster nodes
>and interconnects like Cray MicroArray and Pathscale's Infinipath may
>soon converge the term SMP with cluster.

How could it, given that we don't provide shared memory semantics?

-- greg

.

Relevant Pages

  • Re: why mainframes are still used?
    ... We don't know how many CPUs the 3090 was using. ... which claimed this 25 MFLOPS machine was a supercomputer. ... was usually Cray. ... warrant the cost. ...
    (comp.os.vms)
  • Re: why mainframes are still used?
    ... "Supercomputer" is a term used very loosely by some people. ... and when a 1 nanosecond decrease in the Cray ... parallel array of fairly fast CPUs. ... I think we figured out once that for a 96% parallel problem ...
    (comp.os.vms)