Re: forth and virtual memory
- From: gavino <gavcomedy@xxxxxxxxx>
- Date: Mon, 28 Apr 2008 15:08:21 -0700 (PDT)
On Apr 28, 1:21 pm, Thomas Pornin <por...@xxxxxxxxx> wrote:
According to gavino <gavcom...@xxxxxxxxx>:
http://varnish.projects.linpro.no/wiki/ArchitectNotes
This is an impressive read........they guy seems to do some smart
things by letting the os kernel do a lot of the work its designed for
and he does some tricks to avoid systems calls and keepign things in
filesystems that are expensive.
Another way to see this text is the following: this guy read the man
page for the mmap() system call and thought he was Enlightened. He's not
the first. Using a file-backed memory area and relying on the kernel and
the MMU to move things around between the RAM and the disk is not a new
idea. I even did it myself ten years ago when writing a C preprocessor
(I had weird hobbies at that time), and that was no invention of mine
either.
That technique is no silver bullet. On a modern operating system, the
part which handles the MMU (often dubbed the "kernel") tries to do smart
things when it comes to deciding which chunk of data should reside in
RAM and which should not; but it does so "generically", without really
knowing what the application will do. Distinct applications have
distinct access patterns and distinct needs. Letting the kernel do the
job means that you believe in the "one size fits all" concept, which
often turns out to have only limited applicability to what is
colloquially known as the "real world".
Filling a file from a virtual memory view also often leads to high
filesystem fragmentation, unless you preallocate the complete file,
which is almost invariably a waste of space on a non-dedicated system.
Besides, relying entirely on the MMU means that you merge the adresse
space and the disk space, which implies that on a 32-bit system, you
will have trouble handling more than 2 gigabytes of data.
Some research projects have gone much farther along those lines. For
instance, the EROS operating system was built around the idea of
"checkpointing", in which not only the RAM is used as a cache over the
disk, but the system arranges to save its complete state at regular
intervals, transactionaly, so that power failures can be mostly ignored;
this can be achieved efficently with the MMU. The EROS project was
launched in 1991.
That modern systems have layers of cache to cope with a slow RAM and
very slow disk is not a novel idea either. It is true -- and the guy is
right to mind cache issues -- but as far as I can see, the world of
programming did not wait for him to get the point.
Can forth use this level of detail when it exists on linux? How does
forth interact with virtual memory?
The issue is mostly orthogonal. On a Linux system, the kernel handles
the MMU, and applications talk to the kernel through system calls
(namely, software-triggered interrupts). Low-level programming languages
allow the application programmer to issue direct system calls, and Forth
is not different from C in that respect.
Of course, most of Forth usage is done on much smaller systems which do
not have a MMU, and usually no disk either. Which makes the whole point
quite moot.
There _are_ some cache-related optimizations which are not directly
usable in Forth -- and not in C either. As was pointed out before, cache
issues dominate the question of performance in most cpu-intensive tasks.
Best performance is achieved when memory accesses are performed in a way
which fits well the behaviour of caches. When the programmer uses a
strongly-typed programming language, it becomes possible for the
operating system (or virtual machine implementation) to actually _move_
objects in RAM dynamically to better fit the access patterns. This is a
feature of modern Garbage Collectors and is applicable only if the GC
precisely knows which data element is a pointer and which is not
(because moving an object also means adjusting the references to that
object). Standard Forth is somewhat typeless; a GC for Forth has been
published, but it is a conservative GC which does not move objects
around. There are some Forth-like systems with types, which may allow
for advanced GC techniques (e.g. StrongForth).
If simply using mmap() is 2006-programming, then improved cache
coherency through an object-moving GC must be from year 2015, at least.
Strangely enough, Sun Microsystems has sold the idea at industrial
levels for the last decade (under the name "Java").
--Thomas Pornin
wait since this is a cache program it HAS to be good with 2G+ ......
.
- References:
- forth and virtual memory
- From: gavino
- Re: forth and virtual memory
- From: Thomas Pornin
- forth and virtual memory
- Prev by Date: Re: forth and virtual memory
- Next by Date: Re: Separated headers
- Previous by thread: Re: forth and virtual memory
- Next by thread: Re: forth and virtual memory
- Index(es):
Relevant Pages
|
