Re: RAC or Large SMP...?

Robert Klemme <shortcutter@xxxxxxxxxxxxxx> writes:

On 10.10.2008 08:38, Tim X wrote:

There is a growing argument that one of the main limitations of
multi-core systems isn't at the hardware level, but rather, at the
software level. In particular, the lack of good support in many
languages for writing software that is able to exploit parallel
processing efficiently. The few languages that provide such support
often rely heavily on the ability of the programmer to understand the
complexities and have the skills to adopt algorithms suitable for such
processing. such programmers are rare, particularly in the current era
of out sourcing and commodity development where the emphasis tends to be
towards programmers that are API driven code monkeys that have little
depth of knowledge regarding algorithms and data structures generally
and probably little, if any, knowledge of issues associated with
parallel processing.

I believe this analysis is spot on, but...

There are some very interesting approaches being
developed and I'm confident that in time, many of these considerations
will be handled by clever compilers and sophisticated language support
and APIs that are multi-core aware, but thats probably a way off yet.

... I am not too optimistic here. I do believe that we will see some
improvements over time (for example, since Java 5 there are more
abstractions of concurrency constructs which help building concurrent
applications that do scale better). But I do not see that we will come to
a situation where the programmer provides a sequential problem description
and gets parallel execution. For a very long time the people implementing
parallelized applications will have to do the analysis and design work that
determines which parts can be done concurrently and how synchronization
must take place. And from my experience this is the crucial part in
getting scalability through concurrency into an application.


Yes, I agree. I probably wsn't clear enough. I don't think we are ever
likely to reach the point where programmers can stay in a sequencial
mindset and the tools will take care of the rest. Programmers will need
to change the way they think about problems and learn to apply ADTs and
develop algorithms that are able to exploit the parallelism. My amin
point was that even those programmers that are able to think that way
are somewhat constrained by current language and compiler technology
that make it quite complex and difficult to express parallelism in a way
that is concise and clear. Furthermore, due to the lack of adequately
expressive constructs, being able to reacj a satisfactory level of
confidence re: correctness etc is very difficult. to some degree, what
is already available seems very much like a poorly fitting kludge bolted
onto languages that were primarily designed for sequencial non-parallel
environments.

I listened to a talk by 'Matz' (Ruby) who said that what he wanted to
achieve with Ruby was a language that had a close fit with how he
thought about problems. His arguement was that this allowed him to
express his problems in a way that was clearer and faster because he
didn't have to translate his mental model into a program model. to some
extent, I think this applies with parallel programming. currently, those
programmers that have the knowledge/skills to think about problems in a
parallel fashion are constrained by the fact they have to then translate
their mental model into a language that was designed primarily for
sequencial problem solutions and which has cumbersome 'enhancements'
designed to provide a safety net when operating in a parallel
environment, but which often don't fit 'naturally' with the language.
I'm talking about mainstream languages rather than specialised languages
like Occum or other langauges that have been designed specifically for
developing applications that can exploit parallelism.

I've actually been surprised in the past by the degree to which most
programmers have a 'sequencial' mindset. for example, I was surprised at
the number of programmers that got 'stung' in the early days of the web
because they had an assumption that visitors would arrive in a
sequencial manner. I still come across 'bugs' that are primarily due to
such thinking. for example, the number of web page counters that are
incorrect because they assume that no other process will access their
datafile in between reading the number and writing back a new number
that is incremented by 1. I often have to explain to novice programmers
that in many situations you cannot assume sequencial access to shared
resources. You even see similar issues with many database
applications. How often do we see confused questions regarding locks,
commits, rollbacks etc.

I think I must have been very very lucky. The second course I did in
computing science was on parallel programming. It was back in the late
80s and I was lucky enough to have access to a simulated hypercube. It
was a special PC board that cost a rediculous amount of money (even for
back then when a PC with a 20Mb hard drive and EGA graphics that ran at
12Mhz cost over $6k. I think the bord cost something like $30k or
more. (this was when I purchased my first house, which cost only $60k and
which I sold a couple of years back for $300k) You had a buggy DOS (I
think it was only DOS 1 or 2, most PCs were DOS 3.3 from memory) and a
hacked C compiler that had some additional features that made it easy to
address specific nodes in the hypercube and some basic suppoort for
semaphores and mutex type concepts. As primitive as it was, it was a
hell of a lot of fun. You could develop interesting sorting algorithms
that addressed specific nodes in the hypercube using a mitmask
addressing scheme. However, once you began to get even slightly complex,
it became extremely difficult to map your mental model to the software
level and debugging problems was a complete nightmare! Still, I think
having done that course put me in a very good position for much of the
stuff I worked on later in life.

There is some interesting work I've seen passing references to that
looks quite promising, especially in the area of providing safe access
to resources in a predictable way. For example, I think Microsoft has
been doing some work on a sophisticated optimistic memory model in which
access is controlled by the udnerlying framework and which aims to
eliminate the programmer from having to worry about such issues. The
model is very similar to how oracle handles such issues and enables
consistent views of the data. in fact, I think from memory that they
said the model they were developing was based on how databases have
addressed the need to provide consistent views of data in an environment
with many clients. It is technologies like this that will enable
programmers to devleop parallel programs that don't become so complex
that either they never get out of the bug squashing stage or when they
do, they die due to maintenance problems.

Of course, all of this may become completely mute when quantum computing
comes into play and all our software is in every possible state at every
possible moment. Now that will be a paradigm that is likely to melt a
few programmer minds that have grown up in the von neumann world!
Luckily, I expect to be retired by then.

Tim

--
tcross (at) rapttech dot com dot au
.

Relevant Pages

  • Re: Case sensitivity in programming languages.
    ... Programmers who deliberately create different ... languages and seen different case used on different projects, ... standards which are simply the conventions and personal choices of one group ... readfile(), readFileand ReadFile. ...
    (comp.lang.php)
  • Re: Is Lisp more ideal than Smalltalk?
    ... Meaning that they can design, design, design, and ... A recognised problem with any form of abstraction, ... Smalltalk than other OO-ish languages I use. ... so C++ programmers get a distorted idea of the costs of allocation ...
    (comp.lang.smalltalk.dolphin)
  • Re: Linux in Forth?
    ... tendency to mix applications together so it takes work ... The advantage that Forth has over other languages ... >things the way programmers in other languages do them. ... But I think for that to happen the C/Forth ...
    (comp.lang.forth)
  • Re: Thinking Clearly
    ... Managed and unmanaged code can be mixed together in the same ... with all other .NET languages and native unmanaged C++ code. ... C++ is thus very convenient for interoperability between programmers ... is to support this though. ...
    (borland.public.delphi.non-technical)
  • Re: Programming Language Productivity: The Stupidity of Programmers
    ... There is a theory that programmers tend to prefer smaller languages ... because abstractions are designed in a partic- ... average programmers are not well-equipped to design ...
    (comp.object)