New subject: VM Maker: ..VMMaker.oscog-eem.618.mcz

14 Feb 2014

      Is it intentional that this package and the previous one are prefixed with 
dots?
Levente
On Thu, 13 Feb 2014, commits@source.squeak.org wrote:
...
Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/..VMMaker.oscog-eem.618.mcz
==================== Summary ====================
Name: ..VMMaker.oscog-eem.618
Author: eem
Time: 13 February 2014, 11:51:10.104 am
UUID: 3756532a-fdbf-4e96-8cc9-be0f35157864
Ancestors: ..VMMaker.oscog-eem.617
Some musings on Spur compaction.
=============== Diff against ..VMMaker.oscog-eem.617 ===============
Item was added:

----- Method: SpurMemoryManager>>abstractCompaction (in category 'compaction - analysis') -----
abstractCompaction
"This method answers a rough estimate of compactibility."
<doNotGenerate>

| lowestFree freeChunks used movable |
lowestFree := SmallInteger maxVal.
freeChunks := Set new.
used := Set new.
movable := Set new.
self allObjectsInFreeTreeDo:
[:f|

(self addressAfter: f) < endOfMemory ifTrue:

	[freeChunks add: f.

	 f < lowestFree ifTrue: [lowestFree := f]]].

self allOldSpaceObjectsFrom: lowestFree do:
[:o| | size delta best |

size := self bytesInObject: o.

delta := SmallInteger maxVal.

freeChunks do: [:f| | fs |

	((fs := self bytesInObject: f) >= size) ifTrue:

		[delta > (fs - size) ifTrue:

			[delta := fs - size. best := f]]].

 best ifNotNil:

	[movable add: o.

	 used add: (freeChunks remove: best)]].

^{ totalFreeOldSpace. movable inject: 0 into: [:s :o| s + (self bytesInObject: o)]. used inject: 0 into: [:s :o| s + (self bytesInObject: o)] }!

Item was added:

----- Method: SpurMemoryManager>>abstractPigCompaction (in category 'compaction - analysis') -----
abstractPigCompaction
"This method answers a rough estimate of compactibility using a pig (a large free chunk)."
<doNotGenerate>

| pig pork moved unmoved nmoved nunmoved |
pig := self findAPig.
pork := self bytesInObject: pig.
moved := unmoved := nmoved := nunmoved := 0.
self allOldSpaceObjectsFrom: pig do:
[:o| | bytes |

bytes := self bytesInObject: o.

bytes <= pork

	ifTrue:

		[moved := moved + bytes.

		 nmoved := nmoved + 1.

		 pork := pork - bytes]

	ifFalse:

		[unmoved := unmoved + bytes.

		 nunmoved := nunmoved + 1]].

^{ self bytesInObject: pig. pork. moved. nmoved. unmoved. nunmoved }!

Item was added:

----- Method: SpurMemoryManager>>biggies (in category 'compaction - analysis') -----
biggies
"This method answers a sorted collection of the objects >= 1,000,000 bytes long,
above the lowest large free chunk, sandwiched between nilObj and the end of memory."

<doNotGenerate>

| lowestFree biggies |
lowestFree := SmallInteger maxVal.
self allObjectsInFreeTreeDo:
[:f| (self addressAfter: f) < endOfMemory ifTrue: [f < lowestFree ifTrue: [lowestFree := f]]].

biggies := SortedCollection new.
self allOldSpaceObjectsFrom: lowestFree do:
[:f|

(self bytesInObject: f) >= 1000000 ifTrue:

	[biggies add: f]].

^{{nilObj hex. #nil}}, (biggies collect: [:f| {f hex. self bytesInObject: f}]), {{endOfMemory hex. #endOfMemory}}!

Item was added:

----- Method: SpurMemoryManager>>compactionIssues (in category 'compaction - analysis') -----
compactionIssues
<doNotGenerate>

"Compaction isn't working well.  It rarely moves more than a few tens of kilobytes.  Why?
Load an image and before you run it, or just before a GC, run these anaylsis routines.

e.g.

self abstractCompaction #(63230272 75456 63210648)

shows we can move 75456 bytes of objects, but that would use 63210648 of free space.

i.e. there are lots of big free chunks in play, not many small ones that fit the bill.

self largeFreeChunkDistribution

	#(	#('16r31C788' #nil)

		#('16r1423AC0' 2061864)

		#('16r1B705E8' 1515200)

		#('16r1D31D20' 2011152)

		#('16r1F37818' 1491480)

		#('16r2225968' 1450512)

		#('16r24C92C8' 48575672)  (16r24C92C8 + 48575672) hex '16r531C780' a free chunk

		#('16r531C788' #endOfMemory))

shows there's plenty of large free chunks.  And the trailing 16-byte free chunk shows coallescing is not working properly.

self biggies #(#('16r31C788' #nil) #('16r531C788' #endOfMemory))

shows there are no large objects to be moved.

So... looks like compaction should hold onto the lowest large chunk and preferentially move objects into that.

Let's call it a pig.  Compaction needs to whittle away at the pig.

e.g.

self abstractPigCompaction #(2061864 0 2061864 18759 2018224 34757)

shows we can move 18759 objects that will occupy 2018224 bytes into that

low pig of 2061864 bytes."!

Item was added:

----- Method: SpurMemoryManager>>findAPig (in category 'compaction - analysis') -----
findAPig
"Answer a large low free chuink."
<doNotGenerate>

| pig |
self allObjectsInFreeTreeDo:
[:f|

(self bytesInObject: f) >= 1000000 ifTrue:

	[(pig isNil or: [pig > f]) ifTrue:

		[pig := f]]].

^pig!

Item was added:

----- Method: SpurMemoryManager>>largeFreeChunkDistribution (in category 'compaction - analysis') -----
largeFreeChunkDistribution
"This method answers a sorted collection of the free chunks >= 1,000,000 bytes long,
sandwiched between nilObj and the end of memory (ignoring the large chunk often found at the end of the heap)."

<doNotGenerate>

| freeChunks |
freeChunks := SortedCollection new.
self allObjectsInFreeTreeDo:
[:f|

((self addressAfter: f) < endOfMemory

 and: [(self bytesInObject: f) >= 1000000]) ifTrue:

	[freeChunks add: f]].

^{{nilObj hex. #nil}}, (freeChunks collect: [:f| {f hex. self bytesInObject: f}]), {{endOfMemory hex. #endOfMemory}}!

Re: [Vm-dev] VM Maker: ..VMMaker.oscog-eem.618.mcz