Re: How does this make you feel?

John Mashey wrote:
>
> Eric P. wrote:
> > Ok, but this does not necessarily imply the OS support _dynamic_
> > page size selected and changeable on the fly. For example
> > a boot time selected page size would suffice, with an API
> > call for apps to query the current system page size.
>
> Actually, that turns out not to be true...
> Google: irix page dplace OR irix pagesize_data OR altix dplace
>
> In practice, you don't have to do a lot of dynamic changes, but it
> really helps to be able to select pagesizes at various times and
> differently for different programs. http://www.ukaff.ac.uk/hints.shtml
> is a web page offering advice to its users, for example. There is a
> hierarchy of advice:
>
> 1) Sysadmin can set default page sizes for system.
> 2) Users can give compile-time options, so that page-size wishes are
> expressed in the binary object file.
> 3) Users can set environment variables PAGESIZE_DATA and
> PAGESIZE_STACK,
> which allows them to easily experiment, even using somebody else's
> binary.
> 4) The dplace(1) command can modify the pagesize settings before
> executing a command, although it's original reason for existence was to
> handle ccNUMA memory placement.
>
> Code text-size is the easy one, because it's fixed, regardless of who
> is sharing it, and in any case, for most programs that care about this
> stuff, data code-size << data size. SGI Origins were at 1 Terabyte of
> main memory by 2001, and I think individual Altix systems go up to
> 24TB. I know there are customer systems actually out there with 10TB
> or more. Sometimes an individual program (parallelized across
> processors) uses almost the entire memory for one problem.
> It's quite easy for big fluid dynamics codes to use arbitrarily-large
> amounts of data; it's perfectly reasonable that a 1-10TB data usage
> could be incurred by a program whose code was 1-10MB, i.e., the data is
> on the order of amillion times bigger than the code.
>
> Anyway, in practice, one doesn't need the full generality of having the
> OS figure out dynamic combination/splitting of pages, but it is clear
> that one size doesn't fit all, even on the same machine. Especially
> with the environment variables, it doesn't take much work to try a few
> combinations to see if it makes a difference.
>
> It's been a long time, so I don't recall the details, or how much
> experimentation (if any) there was with full-dynamic page-size
> handling. Note that the ccNUMA versions of IRIX already did a lot of
> dynamic page-migration, and that was already tricky enough.

Dropping the general split/merge features would seem to help.
It gives 95% of the functionality and eliminates nasty design issues.

But I'm not so sure it can be completely eliminated by decree.

- My intuition tells me that the free list would degenerate
into a set of mostly small 4 KB frames. Assuming a binary buddy
free frame manager, and apply a set of random allocate and free
requests, then after a relatively few iterations I suspect
there would be no large frames left in the pool. Because the
working sets are randomly ordered then trimming the WS list
would not likely produce significant frame coalescence.

This tells me that the OS will likely wind up dealing with
small pages anyway. Without page coalescence programs would
degenerate into small page sizes.

- If a large frame is desired but not available then the OS cannot
wait for one to show up because there is no guarantee that would ever
happen. So the OS must be prepared to assign a small page.
This means that one can associate a size hint with a virtual memory
region/section, but cannot expect it to be obeyed.

- The start and end of virtual memory regions are not necessarily
aligned and sized for convenient large page assignment. So the
OS must deal with fiddly bits at either end. This means a region
can consist of a variety of page sizes anyway.

- If a large page is outswapped, ideally to a single large swap block,
there is no guarantee that a large page will be available to
swap it back in. If there were only smaller pages available then
the OS has to split up the swap block.

- Outswap file space could also degenerate into a pool of small
free block and not necessary be contiguously available for outswap
of a single large page. So the OS must be prepared to scatter a large
page to a set of small swap blocks or a deadlock might result.

If only page splits can take place but no merges then over time
it the system would degenerate into just small page allocations.

So it appears to me that such an OS cannot avoid at least minimal
split-merge support. It might be easier if merge could only happen
on inswap because then it could be dealt with as just a variation
on page fault clustering. Split could only occur on outswap.

Eric

.

Relevant Pages

  • Re: Direct Copying To Share Memory In NDIS ProtocolReceive
    ... access the buffer. ... one driver using IRp that is pended, ... Another concern is locking memory. ... frame that is transmitted to the miniports via the protocol must ...
    (microsoft.public.development.device.drivers)
  • RFC/Doc/BUGs: CONFIG_PACKET_MMAP
    ... configurable circular buffer mapped in user space. ... circular buffer of unswapable memory mapped in the capture process. ... the name indicates, this function allocates pages of memory, it allocates a power ... There is another vector of pointers, wich hold references to each frame ...
    (Linux-Kernel)
  • Re: Website Upload- Easy Reading Promotion of Numberless Cryptography - Adacrypt
    ... He may regulate asleep monopolys, do you frame them? ... ruling drafts to large Abdul, ... Try not to shoot a memory! ...
    (sci.crypt)
  • Re: C3088 CMOS Imaging Sensor Questions
    ... Which require LOTS of computation time and memory to process. ... To maintain an reasonable speed on frame update or just for processing. ... having an input buffer and an output (or temporary ... Some people use algorithms that ASSUME that if they take several pictures ...
    (comp.arch.embedded)
Loading