Re: OT: compilers, interpreters, and VMs


"Robbert Haarman" <comp.lang.misc@xxxxxxxxxxxxx> wrote in message
news:20070611102203.GA3385@xxxxxxxxxxxxxxxxxxxxxxxxxxx
On Mon, Jun 11, 2007 at 02:09:36AM +0000, Marco van de Voort wrote:
On 2007-06-02, Robbert Haarman <comp.lang.misc@xxxxxxxxxxxxx> wrote:
On Sat, Jun 02, 2007 at 09:38:26PM +0200, Marcin 'Qrczak' Kowalczyk
wrote:
I disagree. Very few programs would benefit from the ability to change
code on the fly.

I agree with all your other points, but not this one. It sounds like
the
same argument that gets made against anonymous functions, macros, etc.
by people used to languages that don't have these features.

So following your reasoning, a language should absorb each and every
feature ?

Not exactly. I was trying to point out that run-time code generation,
like certain other features, is one of those things that people don't
realize the utility of, unless they've worked with it pretty
extensively. Indeed, many existing programs have been written without
run-time code generation, anonymous functions, exceptions and so on.
Clearly, we don't need these features. Does that mean they aren't useful
or desirable? You'll find people both sides. However, I claim that these
features are useful and desirable, and many people arguing otherwise
just haven't realized their power yet.

Does this mean we should add each and every feature to a language? Not
at all. What we should strive for, in my opinion, is a language with a
minimal set of features than can be used to _build_ as much
functionality as possible. Macros are my favorite examples of this. With
this one feature, we can build new syntactic forms. This eliminates the
need to bake a lot of constructs into the implementation (e.g. looping
constructs), but also to build syntactic forms that nobody had thought
of when the language implementation was made.


yes.

as note, I am not by any means integrating the dynamic compilation as a core
part of "the language", rather, as a particularly elaborate (and bulky) set
of library functionality.

however, I will go on to claim that, to my knowlege, something exactly like
what all I am imagining, at this point, does not exist (even if none of the
concepts or pieces, themselves, are all that original).


Like other features, being able to change code in a running program has
its supporters.

(I'm only supporter for certain specific goals. Like mathlab and similar
programs, and e.g. MUDs and other online games. Not for general purpose
programming)

Ok. So if we define "general-purpose" as "every kind of program, except
the ones Marco wants to use dynamic code generation for", then a
general-purpose language should not include dynamic code generation. I
see your point, though. Indeed, if there are only one or a few
application domains in which a feature would be useful, then it makes
sense to not include that feature in a general-purpose language. I just
don't agree that this is the case for dynamic code generation.


I agree here, I feel there are many kinds of apps which could benefit, and
actually, likely more than one might easily suspect. for example, I wouldn't
be surprised if, for example, this is not what is hoped for in, say, the
masses of programs that use VMs such as Python or Lua.

it may not be so much that they want to use a language such as these, but
rather, they want what these languages can offer, and that being the ability
to run custom code at runtime (and with a generally less displeasing syntax
and design than 'some other' options...).


in my case, I want an option that does not fracture my codebase. since most
of my codebase is C, and much of the rest of the world's codebase is C, C is
a good choice for a language...

and if I can go further, having both the dynamic abilities and flexability
of a scripting language, but the raw power and capabilities (and
performance) of a language like C, I can't really say I would complain...


For example, I seem to recall Paul Graham bragging about
how they would fix bugs in the running system while the customer was
still on the phone. The ability to make changes to a program without
having to take it offline can be very valuable.

It can yes. However messing with a program runtime can also be a great
way
to break a program or installation.

Absolutely.

On the other hand, the situation we're talking about is one where the
running program is already known to be broken. Whoever is doing the
debugging will just have to make the right decision about whether to
make the changes in the running system or not. Note that nothing
prevents you from testing the proposed changes off-line, and only
merging them into the running system once you're satisfied they are
indeed an improvement.


yes, good point.

of course, I am throwing in the onions...


Imagine that you could
upgrade the software on your mission-critical system, without having to
restart it!

Imagine messing with mission-critical systems runtime. Brrr.

It could kill you or it could save you. What if the satellite has
already been launched? Your choices are "Too bad, we'll get it right in
the next release." and "We can at least try to fix it on the fly."
Unless, of course, your implementation does not support run-time
changes. In that case, you have rejected the latter option up front.


some high profile examples of this particular ability exist.

AFAIK, this has been done in the cases of both the mars rovers and the
hubble.


And if we really have the ability to modify software on the
fly, why not also use it to update, say, your web browser, without
having to restart it?

Well, for starters, your software needs to be modifyable (think bigger,
slower here),

Bigger, probably. Slower, not necessarily. But you're right, there are
costs.


yes.

it would be slower if I were indirecting my calls and memory/variable access
somehow, but I am not.

what does a call sequence look like:
push foo_str
call printf
pop ecx

hardly a speed drain here (actually, in many of my test cases, I am getting
IMO comprable code output to gcc, with default settings, which is not too
terrible of a result).


but, how then can I dynamically reassign printf?...
well, the secret is that I keep track of every known relocation to printf,
among other things, and if later on printf is modified, all the relocations
are modified to reflect the new address.

now, there is a problem, in that on windows, exe's typically have
relocations stripped. this leads to a sad, but easily managable situation.
in effect, the core exe is kept minimal, potentially little more than the
linker, and a few other pieces (maybe excluding even the compiler
framework).

and so, when loaded, the app is largely assembled from some mass of static
libraries and object files, and then run (at this point, maybe the compiler
framework comes into play, and loads additional code or subsystems,
potentially from object files, source files, or by other means).

this is the idea at least, though all this has not been tested beyond some
trivial level.


there is an integrety issue there (who is allowed to?),

Yes. On the other hand, is this really different (for purposes of
authorization) from applying a patch off-line and restarting?


the main issue I guess is if you allow code from "untrusted" sources (such
as web sites, ...).
if so, there may be a higher risk if there is not some kind of linker or
memory-access validation going down.

actually, a help may come from some tricks employed in se linux.

some tricks could include things such as randomly reorganizing things in
memory during linking, using keys in controlling the linking process, and
assigning security access controls to pieces of code.

sadly, this would require special measures to be taken to indicate the
security policies of pieces of code (an example being, for example, special
strings giving security info).

of course, there is little to say that the code can't make the app crash,
and some holes could be difficult to close (such as the original host exe
and various dll's being at known addresses), but these are a different
issue...

better would be not to accept untrusted code in some form that can't be
validated, which would probably exclude C in general, but yeah...


but most importantly, it is pretty incontrollable.

Again, more so than applying a patch off-line? Depending on the
specifics, dynamic modification could even be more controlled:
presumably, dynamic modifications would have to stay within the
framework of the language (type safety etc.), whereas an off-line patch
could overwrite arbitrary parts of the binary with arbitrary data.

I also wanted to point out that the ability to modify a system at
run-time is not exactly revolutionary. In fact, it is completely taken
for granted in certain domains. And I don't just mean certain obscure
domains that hardly anybody ever touches. I'm thinking of web
development and the shell. In both cases, we can modify the system by
adding, removing, and editing files. In both cases, I think, the ability
to modify the system without having to restart it is clearly valuable.

I am a aware that the whole infrastructure underlying a web site is not
like how we would typically imagine a programming language. However, if
you imagine URLs as designating functions (which is actually what they
do in many web applications I've worked on), you should get the picture.
The shell, on the other hand, is very much like how we would imagine a
programming language. The external programs _are_ the functions of the
shell (although, of course, most shells also support functions that
aren't external programs -- but these can typically be modified
dynamically, too).


interesting...


Various programs allow run-time customization to greater or lesser
degrees. Emacs can be completely stood on its head (or maybe that should
be "many heads" in Emacs's case). Mutt, while clearly not allowing the
core program itself to be altered at run-time, still allows extensive
modifications to its behavior through the ability to run commands and
modify various hooks. The line between program code and configuration is
fuzzy; some configuration languages are so powerful that they can be
used to alter the purpose of a program. To stick with the two programs
already mentioned: Emacs (a text editor) can be used to play Tetris, and
mutt (a mail client) can be used to send and receive Usenet news...and
the necessary changes can be made without restarting the running
programs.


yeah...


Regards,

Bob

--
"Cross platform apps are like unisex underwear."


.

Relevant Pages

  • Re: OT: compilers, interpreters, and VMs
    ... by people used to languages that don't have these features. ... a language should absorb each and every feature? ... run-time code generation, anonymous functions, exceptions and so on. ... general-purpose language should not include dynamic code generation. ...
    (comp.lang.misc)
  • Re: OT: compilers, interpreters, and VMs
    ... by people used to languages that don't have these features. ... a language should absorb each and every feature? ... run-time code generation, anonymous functions, exceptions and so on. ... general-purpose language should not include dynamic code generation. ...
    (comp.lang.misc)
  • Multiple printers
    ... One of the features of the language is the ability ... to use up to eight simultaneous printer files, so that you can for example ...
    (microsoft.public.office.developer.vba)
  • Re: Where did the prophet really live?
    ... features were in the spoken language or not. ... Now the point that Macdonald makes is that Sabaic was the prestige ... medieval arab historians had reported inscriptions ...
    (soc.religion.islam)
  • Re: Is C99 the final C? (some suggestions)
    ... I can't think of a single feature in C99 that would come as ... There are a number of features in C99 that I will steer away from if ... > * a library function that allows the retrieval of the size of a memory ... With the restrictions of the C language, I think you are going to find ...
    (comp.lang.c)