Re: Code density and performance?


pg_nh@xxxxxxxxxxxxxxxxxxx (Peter Grandi) writes:
> Indeed, because the long-forgotten statistics I had mentioned show
> that programs that have not been specifically locality improved tend
> to have ''hotspots'' of about 100-200 bytes long... 512 bytes is
> sort of a decent compromise between too small and too large for that
> sort of program.

one of the studies at the cambridge science center
http://www.garlic.com/~lynn/subtopic.html#545tech

in the early 70s was doing full I & D traces and attempting to re-org
programs for better virtual memory characteristics. the application
was released as vs/repack product in the mid-70s. besides looking at
instruction hotspots and repacking programs ... it was also used to
study storage utilization patterns. it was used as part of analysing
and redesigning the apl\360 garbage collection scheme in the port to
cms\apl (basically from swapping whole real storage workspaces ... to
large virtual memory workspaces).

as an aside ... the science center had a number of different
performance analysis related activities; besides vs/repack ... there
was kernel instrumentation and 7x24 activity recoding, hot-spot
investigation, multiple regression analysis, analytical modeling, etc.
one of the analytical modeling tuols evolved into the performance
predictor on the world-wide sales and marketing system
http://www.garlic.com/~lynn/subtopic.html#hone
where sales & marketing people could input some customer details and
ask performance & configuration what-if questions. the extensive
performance monitoring laid the groundwork and evolved into capacity
planning
http://www.garlic.com/~lynn/subtopic.html#bench

360/67 had 4k pages, 1mbyte segments, 24bit & 32bit addressing.

370 was initially announced with only real storage.

when virtual memory was announced for 370 ... virtual memory support
2k & 4k pages, 64k & 1mbyte segments and 24bit addressing.

operating systems targeted for smaller real memory 370 configurations
implemented 2k pages ... while the operating system targeted for
larger real memory 370 configuration implemented 4k pages.

the morph from cp67 to vm370 had vm370 using native 4k pages for
virtual machine emulation .... although it had to support both 2k page
and 4k page operation for shadow pages to correspond to what ever the
virtual machine was doing.

vs/1 was one of the 2k page operating systems ... gaining better
packing in small real storage configurations. however, starting in the
mid-70s ... standard real storage configurations were getting much
larger.

part of ecps (virtual machine microcode performance assist) on 138/148
(and later 4341) ...
http://www.garlic.com/~lynn/subtopic.html#mcode

it was possible to tune vs/1 running in a virtual machine to run
faster than vs/1 directly on the real hardware. part of it was
configuring vs/1 such that rather than vs/1 doing its own paging in 2k
units ... it effectively relied on vm370 to do paging in 4k
units. effectively any better packing based on using the smaller page
size was more than offset by the overhead of the smaller disk transfer
sizes. as an aside, some amount of the hotspot analysis was used for
selecting 6kbytes of kernel code for migration into ecps microcode.

the smaller page sizes and improved packing is targeted at optimizing
real storage useage as a scarce resource. as machines were getting
faster and larger ... system bottlenecks were shifting from real
storage constrained to disk transfer constrained (file i/o, paging,
all kinds of disk transfers).

on the larger real storage configurations any loss in real storage
optimization going from 2k to 4k pages ... was more than offset by
improvement in disk optimization (especially since major system
bottlenecks were shifting from real storage limited to disk limited).

this was further highlighted in the early 80s with the implementation
of big pages. the cpu hardware was still 4k pages ... however the page
transfer management collected ten 4k pages (into a big page) at a time
for page-out. any subsequent (4k) page fault would fetch all ten 4k
pages (of the related big page) in one transfer. note that this was
slightly better than straight 40k pages ... since the aggregation of
ten 4k pages into a big page was based on reference locality and not
contiguous 40k virtual memory.

some past big page postings
http://www.garlic.com/~lynn/2001k.html#60 Defrag in linux? - Newbie question
http://www.garlic.com/~lynn/2002b.html#20 index searching
http://www.garlic.com/~lynn/2002c.html#29 Page size (was: VAX, M68K complex instructions)
http://www.garlic.com/~lynn/2002c.html#48 Swapper was Re: History of Login Names
http://www.garlic.com/~lynn/2002e.html#8 What are some impressive page rates?
http://www.garlic.com/~lynn/2002e.html#11 What are some impressive page rates?
http://www.garlic.com/~lynn/2002f.html#20 Blade architectures
http://www.garlic.com/~lynn/2002l.html#36 Do any architectures use instruction count instead of timer
http://www.garlic.com/~lynn/2002m.html#4 Handling variable page sizes?
http://www.garlic.com/~lynn/2003b.html#69 Disk drives as commodities. Was Re: Yamhill
http://www.garlic.com/~lynn/2003d.html#21 PDP10 and RISC
http://www.garlic.com/~lynn/2003f.html#5 Alpha performance, why?
http://www.garlic.com/~lynn/2003f.html#9 Alpha performance, why?
http://www.garlic.com/~lynn/2003f.html#48 Alpha performance, why?
http://www.garlic.com/~lynn/2003g.html#12 Page Table - per OS/Process
http://www.garlic.com/~lynn/2003o.html#61 1teraflops cell processor possible?
http://www.garlic.com/~lynn/2003o.html#62 1teraflops cell processor possible?
http://www.garlic.com/~lynn/2004e.html#16 Paging query - progress
http://www.garlic.com/~lynn/2004.html#13 Holee shit! 30 years ago!
http://www.garlic.com/~lynn/2004n.html#22 Shipwrecks
http://www.garlic.com/~lynn/2004p.html#39 100% CPU is not always bad
http://www.garlic.com/~lynn/2005h.html#15 Exceptions at basic block boundaries
http://www.garlic.com/~lynn/2005j.html#51 Q ALLOC PAGE vs. CP Q ALLOC vs ESAMAP
http://www.garlic.com/~lynn/2005l.html#41 25% Pageds utilization on 3390-09?

early in this period ... i had started asserting that the relative
system disk thruput had declined by a factor of ten times over a
period of years. the disk division assigned the disk performance group
to refute the claims. after a couple weeks they came back and said
that i had slightly understated the problem. some past posts on
the subject:
http://www.garlic.com/~lynn/93.html#31 Big I/O or Kicking the Mainframe out the Door
http://www.garlic.com/~lynn/94.html#43 Bloat, elegance, simplicity and other irrelevant concepts
http://www.garlic.com/~lynn/94.html#55 How Do the Old Mainframes Compare to Today's Micros?
http://www.garlic.com/~lynn/95.html#10 Virtual Memory (A return to the past?)
http://www.garlic.com/~lynn/98.html#46 The god old days(???)
http://www.garlic.com/~lynn/99.html#4 IBM S/360
http://www.garlic.com/~lynn/99.html#112 OS/360 names and error codes (was: Humorous and/or Interesting Opcodes)
http://www.garlic.com/~lynn/2001d.html#66 Pentium 4 Prefetch engine?
http://www.garlic.com/~lynn/2001f.html#62 any 70's era supercomputers that ran as slow as today's supercomputers?
http://www.garlic.com/~lynn/2001f.html#68 Q: Merced a flop or not?
http://www.garlic.com/~lynn/2001l.html#40 MVS History (all parts)
http://www.garlic.com/~lynn/2001l.html#61 MVS History (all parts)
http://www.garlic.com/~lynn/2001m.html#23 Smallest Storage Capacity Hard Disk?
http://www.garlic.com/~lynn/2002b.html#11 Microcode? (& index searching)
http://www.garlic.com/~lynn/2002b.html#20 index searching
http://www.garlic.com/~lynn/2002e.html#8 What are some impressive page rates?
http://www.garlic.com/~lynn/2002e.html#9 What are some impressive page rates?
http://www.garlic.com/~lynn/2002.html#5 index searching
http://www.garlic.com/~lynn/2002i.html#16 AS/400 and MVS - clarification please
http://www.garlic.com/~lynn/2003i.html#33 Fix the shuttle or fly it unmanned
http://www.garlic.com/~lynn/2004n.html#22 Shipwrecks
http://www.garlic.com/~lynn/2004p.html#39 100% CPU is not always bad
http://www.garlic.com/~lynn/2005h.html#13 Today's mainframe--anything to new?
http://www.garlic.com/~lynn/2005k.html#53 Performance and Capacity Planning


misc. past posts mentioning vs/repack
http://www.garlic.com/~lynn/94.html#7 IBM 7090 (360s, 370s, apl, etc)
http://www.garlic.com/~lynn/99.html#68 The Melissa Virus or War on Microsoft?
http://www.garlic.com/~lynn/2000g.html#30 Could CDR-coding be on the way back?
http://www.garlic.com/~lynn/2001b.html#83 Z/90, S/390, 370/ESA (slightly off topic)
http://www.garlic.com/~lynn/2001c.html#31 database (or b-tree) page sizes
http://www.garlic.com/~lynn/2001c.html#33 database (or b-tree) page sizes
http://www.garlic.com/~lynn/2001i.html#20 Very CISC Instuctions (Was: why the machine word size ...)
http://www.garlic.com/~lynn/2002c.html#28 OS Workloads : Interactive etc
http://www.garlic.com/~lynn/2002c.html#45 cp/67 addenda (cross-post warning)
http://www.garlic.com/~lynn/2002c.html#46 cp/67 addenda (cross-post warning)
http://www.garlic.com/~lynn/2002c.html#49 Swapper was Re: History of Login Names
http://www.garlic.com/~lynn/2002e.html#50 IBM going after Strobe?
http://www.garlic.com/~lynn/2002f.html#50 Blade architectures
http://www.garlic.com/~lynn/2003f.html#15 Alpha performance, why?
http://www.garlic.com/~lynn/2003f.html#21 "Super-Cheap" Supercomputing
http://www.garlic.com/~lynn/2003f.html#53 Alpha performance, why?
http://www.garlic.com/~lynn/2003g.html#15 Disk capacity and backup solutions
http://www.garlic.com/~lynn/2003h.html#8 IBM says AMD dead in 5yrs ... -- Microsoft Monopoly vs. IBM
http://www.garlic.com/~lynn/2003j.html#32 Language semantics wrt exploits
http://www.garlic.com/~lynn/2004c.html#21 PSW Sampling
http://www.garlic.com/~lynn/2004.html#14 Holee shit! 30 years ago!
http://www.garlic.com/~lynn/2004m.html#22 Lock-free algorithms
http://www.garlic.com/~lynn/2004n.html#55 Integer types for 128-bit addressing
http://www.garlic.com/~lynn/2004o.html#7 Integer types for 128-bit addressing
http://www.garlic.com/~lynn/2004q.html#73 Athlon cache question
http://www.garlic.com/~lynn/2004q.html#76 Athlon cache question
http://www.garlic.com/~lynn/2005d.html#41 Thou shalt have no other gods before the ANSI C standard
http://www.garlic.com/~lynn/2005d.html#48 Secure design
http://www.garlic.com/~lynn/2005h.html#15 Exceptions at basic block boundaries
http://www.garlic.com/~lynn/2005.html#4 Athlon cache question
http://www.garlic.com/~lynn/2005j.html#62 More on garbage collection
http://www.garlic.com/~lynn/2005k.html#17 More on garbage collection
http://www.garlic.com/~lynn/2005m.html#28 IBM's mini computers--lack thereof


--
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/
.

Relevant Pages

  • Re: how much swap size did you take?
    ... virtual memory is used to contain executing processes. ... processes than can fit into real storage. ... to allocate a corresponding page on disk. ... This "duplicate" allocation scenario requires that amount ...
    (alt.os.linux.redhat)
  • Re: Virtual memory
    ... A common cause of low virtual memory message is that you have insufficient ... How large is your hard disk and how much free ... The Windows XP files alone can take around 3 gb. ...
    (microsoft.public.windowsxp.perform_maintain)
  • Re: [Bulk] Re: [opensuse] defrag
    ... that defragmenting is completely useless with Linux, ... explain how fragmentation is a problem on Linux. ... Shortly after Virtual Memory came along ... space adb disk buffer space meaningless. ...
    (SuSE)
  • Re: Recd pop up notice re virtual memory
    ... At present the free disk space of 10 gb exceeds 50%, ... Performance Settings, Advanced, Virtual Memory, Change and place the ... "system managed size" will that let Windows allot the correct memory ... How large is your hard disk and how much free disk space. ...
    (microsoft.public.windowsxp.basics)
  • Re: Is Greenspun enough?
    ... > George Neuner wrote: ... >> Seriously, you need to read up on MMUs, virtual memory and demand ... > that they cache everything from disk, ... The caching sometimes happens at first access, and not at mmap time. ...
    (comp.lang.lisp)