Re: RFC : SOME IDEAS FOR THE APPLE II FPGA'ers

mdj wrote:
Michael J. Mahon wrote:


I wondered how long it would be before we "hit the wall" and
necessity took over...


The interesting thing now will be to see how long it takes for the next
wall to be hit...

I think we're already hitting it. It's the explicit parallelism
wall.

For decades, we refused to learn (or teach) paradigms for dealing
reliably with concurrency--it was always considered a "black art"
that only a few system programmers really needed. As a result,
we have a raft of programmers who don't know how to decompose a
problem into concurrent sub-parts and how to manage (correctly)
the data flow between them to maximize parallelism.

When the "thread" model showed up, I was amazed that everyone
seemed to think that it was a good idea to allow threads--by
default!--to see each others' data space! Wow! Kind of like
building cars as open frames, with the engine sitting a foot
in front of the driver! ;-)

I proposed a "cookbook" approach, with a half-dozen to a dozen
"standard" parallel decompositions, each with a set of recommended
synchronization and data passing operations, that could be applied
recursively to any problem until an appropriate level of parallelism
was obtained with a tolerable communication cost. Still not easy,
but it prevents the continual buggy reinvention of wheels...

My argument for why this would be necessary was that Moore's "Law"
was reaching practical limits in the first decade of this century,
and it would be at least another decade before a new crop of
parallel-savvy programmers could be trained. That leaves about
a decade of "performance famine" unless we can find a relatively
quick way to get the current crop of programmers up to a reasonable
level of productivity (and especially reliability) with concurrent
programming.

Well, here we are, with most performance improvements coming from
MP, not uniprocessor performance, and most application programmers
still in the dark about how to use MP to speed up their apps. ;-(

The Digitek company, in the 1960's, wrote FORTRAN compilers for,
initially, minicomputers. Inspired by the SDS 940's POPs, they
designed a compiler using a multi-stack pseudo-machine that could
be either interpreted or macro-expanded into "fast calls" or in-line
code.

After porting a compiler to a new machine, they would run it fully
interpretive, in no more than half the memory of the machine, and
use the other half of the machine's memory for profile counters
for the pseudo instructions.

Then they would mark the *big* peaks for in-line expansion, the
middling peaks for fast call expansion, and the rest for interpretation.

This optimization tool allowed them to make the smallest, fastest
compilers in the world for several years--and it was all based on
being able to fit the compiler into less than half the available
memory!


That's a brilliant piece of engineering, in any time, but especially
for its day... I'll have to learn more about it.

I agree--it almost made me want to go to work for them!

As is well known, the vast majority of *all* code is executed very
seldom (if at all), and could be compressed by an order of magnitude
if expressed in space-efficient interpretive form. (Or even literally
compressed, but that's another story.) And executing "cold" code a
little slower would have essentially no negative effect on system speed.

I've always been a fan of that approach myself. It's interesting to see
similar techniques applied in current generation languages. Both Java
and .NET utilise "Hot Spot" compilation, which recompiles the bytecode
the machine code at runtime for heavily frequented codepaths. The net
result is that you achieve virtually identical "perceived" runtime to
machine coding the whole app, at a fraction of the cost.

Exactly. Dynamic optimization is where this eventually leads.

In essence, it takes Knuths famous quote "Premature optimisation is the
root of all evil" and embodies it in a runtime system.

Which is why it's a good idea to delay optimization until runtime
reveals that it's worth it and exactly what can be done to improve it.

For example, a virtual method may be invoked inside a tight loop, but
with almost all the iterations being on the same class. Dynamic
optimization would dynamically rewrite the inner loop with a static
method call, prefixed with a "test out" for a changed class. In most
cases, this would allow substantial inlining, opening up still more
optimization opportunities.

This is why I loved Apple Pascal so much; I could write applications
using a high level language that had shared libraries, dynamic loading,
etc. and hand craft the critical sections in assembly language. Of
course, most people did/do this with Applesoft, but to me BASIC is
hideous

There is no doubt--Pascal *is* beautiful. I loved the data type
constructors at first sight. ;-) (Then I learned how often I needed
to get around its strong typing, but that's another story.)

But, Applesoft is pervasive, much more widely known, and "good enough",
for writing an "event loop" and file access code, so _real politik_
often makes it the simplest solution.

I've always found syntax + compiler semantics to be quite weak in
catching the interesting errors. It's fine for spelling errors and
the occasional type mismatch, but there are just too many utterances
that are perfectly syntactic, but meaningless:


Absolutely true. If we look at the common errors such type checking is
supposed to solve, for instance, array bounds issues that are so common
in C programs, it becomes obvious in retrospect that the problem was
caused not by lack of typing, but by lack of language features to allow
a succinct idiom. There are plenty of languages out there that are
typeless, but programs written in them don't suffer that failure mode,
since they offer a 'foreach' construct to iterate collections in an
error-free way.

Yes, I put just such features into a system programming language based
on Pascal. Since Pascal allowed subrange types to be named (like
10..63), I allowed FORs to iterate over subranges (FOR i IN <subrange>).
(Counted loops do pose interesting issues, though, with termination
conditions that typically exceed the range.)


It's interesting to watch at the moment, things moving slowly away from
Java/.NET style VM's towards interpreted languages like Ruby. Why?
Programmer productivity.
>
I'm actually quite enjoying the change, and about the only thing I miss
about the stronger typing models of Java or C++ is that I now have to
commit a lot more API calls to memory, as the language is typeless and
IDE's/Editors can't autocomplete and hint at method invocations
anywhere near as well.

But then, it's also meant shedding complex tools in favour of the good
old text editor and command line combo, as the heavy tools can no
longer justify there own weight.

The productivity and "liveness" of an interactive interpretive
system is unmatched. It's what makes even Applesoft fun!

In the end, it's a fun journey watching a discipline mature, as long as
you can withstand the bumps in the road along the way :-)

Yes, and a microcosm of life itself. ;-)


Absolutely! And as long as one remembers this the bumps are a source of
amusement rather than frustration ;-)

And a primary source of learning! "Learning is never what one expects."

"...we now return you to your regularly scheduled Apple II discussion."
;-)

-michael

Music synthesis for 8-bit Apple II's!
Home page: http://members.aol.com/MJMahon/

"The wastebasket is our most important design
tool--and it is seriously underused."
.

Relevant Pages

  • Re: "STL from the Ground Up"
    ... high-level intermediate language than can interoperate with many other ... If your language lacks expressive features then you cannot write code ... memory management in comparison. ... Mostly because type errors mean that the programmer and compiler disagree ...
    (comp.programming)
  • Re: casts
    ... This is why most shit programmers refuse to learn languages including ... C Sharp and Java. ... compiler in a later edition of Visual Basic, ... language for processing data. ...
    (comp.lang.c)
  • Re: Executable Memory in a Driver
    ... >> criminal to expose users to the added bluescreen and security risk. ... In a language that can't access outside an array, ... that doesn't need to move memory. ... > desired in the compiler. ...
    (microsoft.public.development.device.drivers)
  • Re: Theodore Adorno, a prophet of data systems design
    ... C may not be a forgiving language, ... >> attention get burned with the memory leaks, and buffer overruns, ... by a small, compact team of brilliant programmers, or it may be ... so you agree that it is not a limit on the length of the string, ...
    (comp.programming)
  • Re: Theodore Adorno, a prophet of data systems design
    ... C may not be a forgiving language, ... >> attention get burned with the memory leaks, and buffer overruns, ... by a small, compact team of brilliant programmers, or it may be ... so you agree that it is not a limit on the length of the string, ...
    (comp.programming)