Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3323.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.3323
Author: eem
Time: 24 April 2023, 1:23:30.244395 pm
UUID: 6699e583-2551-423c-b85a-f2243165c8af
Ancestors: VMMaker.oscog-eem.3322
Ignore that last comm it. Fix Slang generation of SpurSegmentManager>>segmentContainingObj:
=============== Diff against VMMaker.oscog-eem.3322 ===============
Item was changed:
----- Method: SpurSegmentManager>>segmentContainingObj: (in category 'accessing') -----
segmentContainingObj: objOop
"Answer the segment containing an object. This is mostly for assert checking, but
variations on the incremental GC may use it in anger. Binary search is (of course)
marginally slower than linear search for a single segment (e.g. in a 720k object heap,
67.1ms vs 61.3ms, or 9.5% slower to derive the segment containing every old space
entity), but usefully faster for many segments (e.g. 92.7ms vs 116ms, or 20% faster
in the same heap extended with enough large arrays to require 11 segments; and this
is pessimal; there are fewer objects at high addresses since the large arrays are there)."
<export: true>
<returnTypeC: #'SpurSegmentInfo *'>
| high low mid seg |
- <var: 'seg' type: #'SpurSegmentInfo *'>
low := 0. mid := numSegments // 2. high := numSegments - 1.
+ [seg := self addressOf: (segments at: mid).
- [seg := segments at: mid.
(self oop: objOop isGreaterThanOrEqualTo: seg segStart)
ifTrue:
[mid = high
ifTrue:
[^(self oop: objOop isLessThan: seg segLimit) ifTrue:
[seg]]
ifFalse:
[low := mid.
mid := mid + high + 1 // 2]]
ifFalse:
[high := mid - 1.
mid := low + mid // 2].
low <= high] whileTrue.
^nil!
Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3322.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.3322
Author: eem
Time: 24 April 2023, 12:57:14.035907 pm
UUID: 14111a12-3992-426d-b299-775b91bccb65
Ancestors: VMMaker.oscog-eem.3321
Fix a slang issue with SpurSegmentManager>>segmentContainingObj:
=============== Diff against VMMaker.oscog-eem.3321 ===============
Item was changed:
----- Method: SpurSegmentManager>>segmentContainingObj: (in category 'accessing') -----
segmentContainingObj: objOop
"Answer the segment containing an object. This is mostly for assert checking, but
variations on the incremental GC may use it in anger. Binary search is (of course)
marginally slower than linear search for a single segment (e.g. in a 720k object heap,
67.1ms vs 61.3ms, or 9.5% slower to derive the segment containing every old space
entity), but usefully faster for many segments (e.g. 92.7ms vs 116ms, or 20% faster
in the same heap extended with enough large arrays to require 11 segments; and this
is pessimal; there are fewer objects at high addresses since the large arrays are there)."
<export: true>
<returnTypeC: #'SpurSegmentInfo *'>
| high low mid seg |
+ <var: 'seg' type: #'SpurSegmentInfo *'>
low := 0. mid := numSegments // 2. high := numSegments - 1.
[seg := segments at: mid.
(self oop: objOop isGreaterThanOrEqualTo: seg segStart)
ifTrue:
[mid = high
ifTrue:
[^(self oop: objOop isLessThan: seg segLimit) ifTrue:
[seg]]
ifFalse:
[low := mid.
mid := mid + high + 1 // 2]]
ifFalse:
[high := mid - 1.
mid := low + mid // 2].
low <= high] whileTrue.
^nil!
Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3321.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.3321
Author: eem
Time: 24 April 2023, 12:48:14.391365 pm
UUID: 6b9e6729-3881-41bd-928b-15323a50759c
Ancestors: VMMaker.oscog-eem.3320
fix a warning
=============== Diff against VMMaker.oscog-eem.3320 ===============
Item was changed:
----- Method: SpurMemoryManager>>objectsAccessibleFromRoots: (in category 'image segment in/out') -----
objectsAccessibleFromRoots: arrayOfRootsArg
"This primitive is called from Squeak as...
arrayOfRoots uniquelyAccessibleObjects
This primitive answers an array of the receiver and every object in its proper tree of subParts (ie, that is not refered to from anywhere else outside the tree).
The primitive can fail for the following reasons with the specified failure codes:
PrimErrNoMemory: additional allocations failed"
<inline: false>
| arrayOfRoots arrayOfObjects |
- <var: 'segAddr' type: #usqInt>
self runLeakCheckerFor: GCCheckImageSegment.
"First scavenge to collect any new space garbage that refers to the graph."
self scavengingGC.
arrayOfRoots := self updatePostScavenge: arrayOfRootsArg.
"Now compute the transitive closure, collecting the sequence of objects to be stored in the arrayOfObjects array.
Included in arrayOfObjects are the arrayOfRoots and all its contents. All objects have been unmarked."
arrayOfObjects := self objectsReachableFromRoots: arrayOfRoots.
arrayOfObjects ifNil:
[^self integerObjectOf: PrimErrNoMemory].
"If objectsReachableFromRoots: answers an integer there is not enough continuous free space in which to allocate the
reachable objects. If there is sufficient free space then answer an error code to prompt a compacting GC and a retry."
(self isIntegerObject: arrayOfObjects) ifTrue:
[totalFreeOldSpace - self allocationUnit >= (self integerValueOf: arrayOfObjects) ifTrue:
[^self integerObjectOf: PrimErrNeedCompaction].
^self integerObjectOf: PrimErrNoMemory].
self assert: self allObjectsUnmarked. "work to be done when the incremental GC is written"
self deny: (self forwardersIn: arrayOfObjects).
^arrayOfObjects!
Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3320.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.3320
Author: eem
Time: 24 April 2023, 12:42:06.749576 pm
UUID: 43db9142-e5a3-41f0-b78e-ba9f0165183d
Ancestors: VMMaker.oscog-eem.3319
SpurSegmentManager>>totalBytesInSegments do the counting. Use this in the accessors for total memory size, SpurSegmentManager>>totalOldSpaceCapacity/totalOldSpaceSize.
Fix a disagreement between the IGC and the non-IGC in SpurSegmentManager>>collapseSegmentsPostSwizzle (in the IGC the slot size of the last bridge is 1, whereas here it is 0) and make sure that the bridge at the end of memory prints correctly.
=============== Diff against VMMaker.oscog-eem.3319 ===============
Item was changed:
----- Method: SpurMemoryManager>>growOldSpaceByAtLeast: (in category 'growing/shrinking memory') -----
growOldSpaceByAtLeast: minAmmount
"Attempt to grow memory by at least minAmmount.
Answer the size of the new segment, or nil if the attempt failed."
| ammount headroom total start interval |
<var: #segInfo type: #'SpurSegmentInfo *'>
"statGrowMemory counts attempts, not successes."
+
statGrowMemory := statGrowMemory + 1."we need to include overhead for a new object header plus the segment bridge."
ammount := minAmmount + (self baseHeaderSize * 2 + self bridgeSize).
"round up to the nearest power of two."
ammount := 1 << (ammount - 1) highBit.
"and grow by at least growHeadroom."
ammount := ammount max: growHeadroom.
"Now apply the maxOldSpaceSize limit, if one is in effect."
maxOldSpaceSize > 0 ifTrue:
+ [total := segmentManager totalOldSpaceSize.
- [total := segmentManager totalBytesInSegments.
total >= maxOldSpaceSize ifTrue:
[needGCFlag := true.
^nil].
headroom := maxOldSpaceSize - total.
headroom < ammount ifTrue:
[headroom < (minAmmount + (self baseHeaderSize * 2 + self bridgeSize)) ifTrue:
[needGCFlag := true.
^nil].
ammount := headroom]].
start := coInterpreter ioUTCMicrosecondsNow.
^(segmentManager addSegmentOfSize: ammount)
ifNil: [needGCFlag := true. nil]
ifNotNil:
[:segInfo|
self assimilateNewSegment: segInfo.
"and add the new free chunk to the free list; done here
instead of in assimilateNewSegment: for the assert"
self addFreeChunkWithBytes: segInfo segSize - self bridgeSize at: segInfo segStart.
self assert: (self addressAfter: (self objectStartingAt: segInfo segStart))
= (segInfo segLimit - self bridgeSize).
self checkFreeSpace: GCCheckFreeSpace.
segmentManager checkSegments.
interval := coInterpreter ioUTCMicrosecondsNow - start.
interval > statMaxAllocSegmentTime ifTrue: [statMaxAllocSegmentTime := interval].
segInfo segSize]!
Item was changed:
----- Method: SpurMemoryManager>>oldSpaceSize (in category 'accessing') -----
oldSpaceSize
+ ^segmentManager totalOldSpaceSize!
- ^segmentManager totalBytesInSegments!
Item was added:
+ ----- Method: SpurMemoryManager>>printBridge:on: (in category 'debug printing interpreter support') -----
+ printBridge: oop on: aStream
+ <var: 'aStream' type: #'FILE *'>
+ <inline: false>
+ '%P is a bridge hdr%d slot size %ud\n'
+ f: aStream
+ printf: { oop asVoidPointer.
+ (self hasOverflowHeader: oop) ifTrue: [16] ifFalse: [8].
+ self numSlotsOfAny: oop}!
Item was changed:
----- Method: SpurMemoryManager>>printCantBeObject:on: (in category 'debug printing interpreter support') -----
printCantBeObject: oop on: aStream
<var: 'aStream' type: #'FILE *'>
+ (oop bitAnd: self allocationUnit - 1) = 0 ifTrue:
+ [((self isInNewSpace: oop)
+ and: [self isForwarded: oop]) ifTrue:
+ [^self printForwarder: oop on: aStream].
+ oop = (segmentManager bridgeAt: segmentManager numSegments - 1) ifTrue:
+ [^self printBridge: oop on: aStream]].
- ((oop bitAnd: self allocationUnit - 1) = 0
- and: [(self isInNewSpace: oop)
- and: [self isForwarded: oop]]) ifTrue:
- [^self printForwarder: oop on: aStream].
'%P%s\n'
f: aStream
printf: {oop asVoidPointer.
((oop bitAnd: self allocationUnit - 1) ~= 0
ifTrue: [' is misaligned']
ifFalse: [coInterpreter whereIs: oop])}!
Item was changed:
----- Method: SpurMemoryManager>>totalMemorySize (in category 'accessing') -----
totalMemorySize
+ ^scavenger newSpaceCapacity + segmentManager totalOldSpaceSize!
- ^scavenger newSpaceCapacity + segmentManager totalBytesInSegments!
Item was changed:
----- Method: SpurPlanningCompactor>>unmarkPinned: (in category 'private') -----
unmarkPinned: pinnedObj
<inline: true>
+ (manager isSegmentBridge: pinnedObj)
+ ifTrue:
+ [self assert: (manager isMarked: pinnedObj)]
+ ifFalse:
+ [manager setIsMarkedOf: pinnedObj to: false.
+ manager segmentManager notePinned: pinnedObj]!
- (manager isSegmentBridge: pinnedObj) ifFalse:
- [manager setIsMarkedOf: pinnedObj to: false.
- manager segmentManager notePinned: pinnedObj]!
Item was changed:
CogClass subclass: #SpurSegmentManager
+ instanceVariableNames: 'manager numSegments numSegInfos segments firstSegmentSize canSwizzle totalHeapSizeIncludingBridges'
- instanceVariableNames: 'manager numSegments numSegInfos segments firstSegmentSize canSwizzle sweepIndex preferredPinningSegment totalHeapSizeIncludingBridges'
classVariableNames: ''
poolDictionaries: ''
category: 'VMMaker-SpurMemoryManager'!
+ !SpurSegmentManager commentStamp: 'eem 4/23/2023 15:19' prior: 0!
- !SpurSegmentManager commentStamp: 'eem 6/7/2017 14:00' prior: 0!
Instances of SpurSegmentManager manage oldSpace, which is organized as a sequence of segments. Segments can be obtained from the operating system and returned to the operating system when empty and shrinkage is required. Segments are kept invisible from the SpurMemoryManager by using "bridge" objects, "fake" pinned objects to bridge the gaps between segments. A pinned object header occupies the last 16 bytes of each segment, and the pinned object's size is the distance to the start of the next segment. So when the memory manager enumerates objects it skips over these bridges and memory appears linear. The constraint is that segments obtained from the operating system must be at a higher address than the first segment. The maximum size of large objects, being an overflow slot size, should be big enough to bridge the gaps, because in 32-bits the maximum size is 2^32 slots. In 64-bits the maximum size of large objects is 2^56 slots, or 2^59 bits, which we hope will suffice.
When an image is written to a snapshot file the second word of the header of the bridge at the end of each segment is replaced by the size of the following segment, the segments are written to the file, and the second word of each bridge is restored. Hence the length of each segment is derived from the bridge at the end of the preceeding segment. The length of the first segment is stored in the image header as firstSegmentBytes. The start of each segment is also derived from the bridge as a delta from the start of the previous segment. The start of The first segment is stored in the image header as startOfMemory.
On load all segments are read into one single segment, eliminating the bridge objects, and computing the swizzle distance for each segment, based on where the segments were in memory when the image file was written, and where the coalesced segment ends up on load. Then the segment is traversed, swizzling pointers by selecting the relevant swizzle for each oop's segment.
Instance Variables
+ manager <SpurMemoryManager>
+ numSegments <Integer>
+ numSegInfos <Integer>
+ segments <Array of SpurSegmentInfo>
+ firstSegmentSize <Integer>
+ canSwizzle <Boolean>
- manager <SpurMemoryManager>
- numSegments <Integer>
- numSegInfos <Integer>
- segments <Array of SpurSegmentInfo>
- firstSegmentSize <Integer>
- canSwizzle <Boolean>
- sweepIndex <Integer>
- preferredPinningSegment <SpurSegmentInfo>
totalHeapSizeIncludingBridges <integer>
canSwizzle
- a flag set and cleared during initialization to validate that swizzling is only performed at the right time
firstSegmentSize
- the size of the first segment when loading an image
manager
- the memory manager the receiver manages segments for (simulation only)
numSegInfos
- the size of the segments array in units of SpurSegmentInfo size
numSegments
- the number of segments (the number of used entries in segments, <= numSegInfos)
- preferredPinningSegment
- - the segment in which objects should be copied when pinned, so as to cluster pinned objects in as few segments as possible. As yet unimplemented.
-
segments
- the start addresses, lengths and offsets to adjust oops on image load, for each segment
- sweepIndex
- - a segment index used to optimize setting the containsPinned flag on segments during freeUnmarkedObjectsAndSortAndCoalesceFreeSpace
-
totalHeapSizeIncludingBridges
- the total size of all segments, used to compute heap usage!
Item was changed:
----- Method: SpurSegmentManager>>collapseSegmentsPostSwizzle (in category 'snapshot') -----
collapseSegmentsPostSwizzle
"The image has been loaded, old segments reconstructed, and the heap
swizzled into a single contiguous segment. Collapse the segments into one."
<inline: false>
canSwizzle := false.
self cCode: []
inSmalltalk:
[segments ifNil:
[self allocateOrExtendSegmentInfos]].
numSegments := 1.
+ self computeTotalHeapSizeIncludingBridges.
(segments at: 0)
segStart: manager oldSpaceStart;
+ segSize: totalHeapSizeIncludingBridges.
- segSize: (totalHeapSizeIncludingBridges := manager endOfMemory - manager oldSpaceStart).
manager bootstrapping ifTrue:
["finally plant a bridge at the end of the coalesced segment and cut back the
manager's notion of the end of memory to immediately before the bridge."
self assert: manager endOfMemory = (segments at: 0) segLimit.
manager
initSegmentBridgeWithBytes: manager bridgeSize
at: manager endOfMemory - manager bridgeSize].
self assert: (manager isSegmentBridge: (self bridgeAt: 0)).
self assert: (manager numSlotsOfAny: (self bridgeAt: 0)) = 0!
Item was changed:
----- Method: SpurSegmentManager>>initialize (in category 'initialization') -----
initialize
+ numSegments := numSegInfos := totalHeapSizeIncludingBridges := 0.
- numSegments := numSegInfos := sweepIndex := totalHeapSizeIncludingBridges := 0.
canSwizzle := false!
Item was changed:
----- Method: SpurSegmentManager>>notePinned: (in category 'pinning') -----
notePinned: objOop
"Let the segmentManager mark which segments contain pinned objects"
+ | seg |
self assert: (manager isPinned: objOop).
+ self deny: (manager isSegmentBridge: objOop).
+ seg := self segmentContainingObj: objOop.
+ seg containsPinned: true!
- (manager isSegmentBridge: objOop)
- ifTrue:
- [manager setIsMarkedOf: objOop to: true]
- ifFalse:
- [[self oop: (segments at: sweepIndex) segLimit isLessThan: objOop] whileTrue:
- [sweepIndex := sweepIndex + 1].
- (segments at: sweepIndex) containsPinned: true]!
Item was changed:
----- Method: SpurSegmentManager>>prepareForGlobalSweep (in category 'pinning') -----
prepareForGlobalSweep
"Let the segmentManager mark which segments contain pinned objects via notePinned:.
For coallesceFreeChunk:, ensure that the last bridge is marked."
- sweepIndex := 0.
0 to: numSegments - 1 do:
[:i| (segments at: i) containsPinned: false].
manager
setIsMarkedOf: (self bridgeAt: numSegments - 1)
to: true!
Item was changed:
----- Method: SpurSegmentManager>>totalBytesInSegments (in category 'accessing') -----
totalBytesInSegments
+ "This ``slow'' count is for asserts only."
| total |
<var: #total type: #usqInt>
total := 0.
0 to: numSegments - 1 do:
[:i|
total := total + (segments at: i) segSize].
- self assert: totalHeapSizeIncludingBridges = total.
^total!
Item was changed:
----- Method: SpurSegmentManager>>totalOldSpaceCapacity (in category 'accessing') -----
totalOldSpaceCapacity
+ self assert: self totalBytesInSegments = totalHeapSizeIncludingBridges.
^totalHeapSizeIncludingBridges - (numSegments * manager bridgeSize)!
Item was changed:
----- Method: SpurSegmentManager>>totalOldSpaceSize (in category 'accessing') -----
totalOldSpaceSize
+ self assert: self totalBytesInSegments = totalHeapSizeIncludingBridges.
^totalHeapSizeIncludingBridges!
Item was changed:
----- Method: SpurSweeper>>unmark: (in category 'sweep phase') -----
unmark: objOop
self assert: ((manager isMarked: objOop) and: [(manager isFreeObject: objOop) not]).
+ (manager isSegmentBridge: objOop)
+ ifTrue: [self assert: (manager isMarked: objOop)]
+ ifFalse:
+ [manager setIsMarkedOf: objOop to: false.
+ (manager isPinned: objOop) ifTrue:
+ [manager segmentManager notePinned: objOop]]!
- (manager isSegmentBridge: objOop) ifFalse: [manager setIsMarkedOf: objOop to: false].
- (manager isPinned: objOop) ifTrue: [manager segmentManager notePinned: objOop]!
Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog.seperateMarking-eem.3325.mcz
==================== Summary ====================
Name: VMMaker.oscog.seperateMarking-eem.3325
Author: eem
Time: 23 April 2023, 11:35:43.748429 am
UUID: 288bab1e-a92e-4eed-8e94-7c8c0138b5b3
Ancestors: VMMaker.oscog.seperateMarking-eem.3324
Merge VMMaker.oscog-eem.3319. Remove a break.
=============== Diff against VMMaker.oscog.seperateMarking-eem.3324 ===============
Item was changed:
----- Method: SpurIncrementalMarker>>markInactiveEphemerons (in category 'weaklings and ephemerons') -----
markInactiveEphemerons
"Go through the unscanned ephemerons, marking the inactive ones, and
removing them from the unscanned ephemerons. Answer if any inactive
ones were found. We cannot fire the ephemerons until all are found to
be active since scan-marking an inactive ephemeron later in the set may
render a previously-observed active ephemeron as inactive."
| foundInactive ptr |
- self break.
foundInactive := false.
ptr := manager unscannedEphemerons start.
[ptr < manager unscannedEphemerons top] whileTrue:
[| ephemeron key |
key := manager followedKeyOfEphemeron: (ephemeron := manager longAt: ptr).
((manager isImmediate: key) or: [manager isMarked: key])
ifTrue:
[foundInactive := true.
"Now remove the inactive ephemeron from the set, and scan-mark it.
Scan-marking it may add more ephemerons to the set."
manager unscannedEphemerons top: manager unscannedEphemerons top - manager bytesPerOop.
manager unscannedEphemerons top > ptr ifTrue:
[manager longAt: ptr put: (manager longAt: manager unscannedEphemerons top)].
self markAndTrace: ephemeron]
ifFalse:
[ptr := ptr + manager bytesPerOop]].
^foundInactive!
Item was changed:
----- Method: SpurMemoryManager>>num32BitUnitsOf: (in category 'object access') -----
num32BitUnitsOf: objOop
"Answer the number of 32-bit units in the given non-immediate object.
N..B. Rounds down 8-bit units, so a 7 byte object has 1 32-bit unit.
Does not adjust the size of contexts by stackPointer."
^(self numBytesOf: objOop) >> 2!
Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3319.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.3319
Author: eem
Time: 23 April 2023, 11:34:33.682706 am
UUID: bac60f14-fec1-4f97-a9c6-218b9d0c2885
Ancestors: VMMaker.oscog-eem.3318
Fix a timestamp
=============== Diff against VMMaker.oscog-eem.3318 ===============
Item was changed:
----- Method: SpurMemoryManager>>num32BitUnitsOf: (in category 'object access') -----
num32BitUnitsOf: objOop
"Answer the number of 32-bit units in the given non-immediate object.
N..B. Rounds down 8-bit units, so a 7 byte object has 1 32-bit unit.
Does not adjust the size of contexts by stackPointer."
^(self numBytesOf: objOop) >> 2!
Hi Marcel, this is cool. Annoyingly gmail sent the commit message to my spam folder so I've only just noticed it. Forgive me.
I would make two minor mods. One, declare static int sqFormatToPlatformFormat & platformFormatToSqFormat as private: two, use squeak, not sq, to differentiate from the public C plugin source/internal pl=ugin implementation API:
static int sqqueakFormatToPlatformFormat(int sqFormat);
static int platformFormatToSqueakFormat(int platformFormat);
And arguably using sqInt instead of int will generate better code across 32 & 64 bits.
--
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/ed1f7431a09f044488…
You are receiving this because you are subscribed to this thread.
Message ID: <OpenSmalltalk/opensmalltalk-vm/commit/ed1f7431a09f044488a85f9d275994d6def7b670/110159333(a)github.com>