Re: The Path to Parallelism

On Dec 12, 6:59 pm, "Michael J. Mahon" <mjma...@xxxxxxx> wrote:
mdj wrote:
On Dec 8, 6:31 am, "Michael J. Mahon" <mjma...@xxxxxxx> wrote:

Right on!

Have you taken a look at the Parallax "Propeller"--it has eight 32-bit
processors on one little chip, just to play with! Kind of like a wide
AppleCrate on a chip. ;-) Not massively parallel, but a start. ;-)

Production systems have just hit this point - The UltraSPARC T2's are
8x8 (being 8 core, 8 thread). I am very much looking forward to seeing
the next iteration; 16 cores is a lot to play with on one 'little'
system.

True--and servers are the one environment where we almost know what
to do with a lot of processors. Most servers are doing hundreds of
essentially independent transactions--almost as "embarrassingly
parallel" as you can get.

In fact, the biggest problem in exploiting high degrees of parallelism
on servers has been locating and fixing all the places in the system
software where we have inserted serial dependencies that are of our
own creation, and not part of the application at all!

Absolutely... The more recent operating systems have fully reentrant
network stacks - something that simply wasn't needed a few years ago.
The T2 mentioned above also has on-die a few purpose built encryption
engines and the ethernet controllers - which enables a massive amount
of parallelism, and the ability of each execution thread to consume <
2 watts of power. Quite impressive.

I agree that massive parallelism is the future--"Ya' canna' change
the laws of physics!" The problem is that less than 1/10th of 1
percent of living programmers have a clue about what that means, so
it's going to be the wild west for a generation or more.

The interesting thing will be whether any additional laws of physics
are discovered that change the currently defined limits ;-)

That would be nice--but I wouldn't buy the IPO. ;-)

:-) Wise move...

The most likely scenario, in my opinion, is that the raw numbers of
processors will grow much more rapidly than our ability to use them
efficiently, so crude "compiler-like" tools will ham-handedly spread
computations over them like peanut butter. For years, this will be
the best we can (usually) do, until new understanding and adaptive
tools begin to exist, delivering at least an order of magnitude of
"free" computation as a result of increasing efficiency.

I agree - although I imagine the tools sophisticated enough to pull
that off will also dramatically improve linear computation performance
as well

That would surprise me, since I expect they will execute 2x-3x times
as much code per thread, but effectively utilize 100-200 processors
while doing it. ;-)

I think there's a lot of potential optimisation available as we move
to higher levels of abstraction. The ability for a runtime system to
choose between alternate algorithms for collections (for instance)
could provide some big improvements down the track.

Matt
.