Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3320.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.3320 Author: eem Time: 24 April 2023, 12:42:06.749576 pm UUID: 43db9142-e5a3-41f0-b78e-ba9f0165183d Ancestors: VMMaker.oscog-eem.3319
SpurSegmentManager>>totalBytesInSegments do the counting. Use this in the accessors for total memory size, SpurSegmentManager>>totalOldSpaceCapacity/totalOldSpaceSize.
Fix a disagreement between the IGC and the non-IGC in SpurSegmentManager>>collapseSegmentsPostSwizzle (in the IGC the slot size of the last bridge is 1, whereas here it is 0) and make sure that the bridge at the end of memory prints correctly.
=============== Diff against VMMaker.oscog-eem.3319 ===============
Item was changed: ----- Method: SpurMemoryManager>>growOldSpaceByAtLeast: (in category 'growing/shrinking memory') ----- growOldSpaceByAtLeast: minAmmount "Attempt to grow memory by at least minAmmount. Answer the size of the new segment, or nil if the attempt failed." | ammount headroom total start interval | <var: #segInfo type: #'SpurSegmentInfo *'> "statGrowMemory counts attempts, not successes." + statGrowMemory := statGrowMemory + 1."we need to include overhead for a new object header plus the segment bridge." ammount := minAmmount + (self baseHeaderSize * 2 + self bridgeSize). "round up to the nearest power of two." ammount := 1 << (ammount - 1) highBit. "and grow by at least growHeadroom." ammount := ammount max: growHeadroom.
"Now apply the maxOldSpaceSize limit, if one is in effect." maxOldSpaceSize > 0 ifTrue: + [total := segmentManager totalOldSpaceSize. - [total := segmentManager totalBytesInSegments. total >= maxOldSpaceSize ifTrue: [needGCFlag := true. ^nil]. headroom := maxOldSpaceSize - total. headroom < ammount ifTrue: [headroom < (minAmmount + (self baseHeaderSize * 2 + self bridgeSize)) ifTrue: [needGCFlag := true. ^nil]. ammount := headroom]]. start := coInterpreter ioUTCMicrosecondsNow. ^(segmentManager addSegmentOfSize: ammount) ifNil: [needGCFlag := true. nil] ifNotNil: [:segInfo| self assimilateNewSegment: segInfo. "and add the new free chunk to the free list; done here instead of in assimilateNewSegment: for the assert" self addFreeChunkWithBytes: segInfo segSize - self bridgeSize at: segInfo segStart. self assert: (self addressAfter: (self objectStartingAt: segInfo segStart)) = (segInfo segLimit - self bridgeSize). self checkFreeSpace: GCCheckFreeSpace. segmentManager checkSegments. interval := coInterpreter ioUTCMicrosecondsNow - start. interval > statMaxAllocSegmentTime ifTrue: [statMaxAllocSegmentTime := interval]. segInfo segSize]!
Item was changed: ----- Method: SpurMemoryManager>>oldSpaceSize (in category 'accessing') ----- oldSpaceSize + ^segmentManager totalOldSpaceSize! - ^segmentManager totalBytesInSegments!
Item was added: + ----- Method: SpurMemoryManager>>printBridge:on: (in category 'debug printing interpreter support') ----- + printBridge: oop on: aStream + <var: 'aStream' type: #'FILE *'> + <inline: false> + '%P is a bridge hdr%d slot size %ud\n' + f: aStream + printf: { oop asVoidPointer. + (self hasOverflowHeader: oop) ifTrue: [16] ifFalse: [8]. + self numSlotsOfAny: oop}!
Item was changed: ----- Method: SpurMemoryManager>>printCantBeObject:on: (in category 'debug printing interpreter support') ----- printCantBeObject: oop on: aStream <var: 'aStream' type: #'FILE *'> + (oop bitAnd: self allocationUnit - 1) = 0 ifTrue: + [((self isInNewSpace: oop) + and: [self isForwarded: oop]) ifTrue: + [^self printForwarder: oop on: aStream]. + oop = (segmentManager bridgeAt: segmentManager numSegments - 1) ifTrue: + [^self printBridge: oop on: aStream]]. - ((oop bitAnd: self allocationUnit - 1) = 0 - and: [(self isInNewSpace: oop) - and: [self isForwarded: oop]]) ifTrue: - [^self printForwarder: oop on: aStream]. '%P%s\n' f: aStream printf: {oop asVoidPointer. ((oop bitAnd: self allocationUnit - 1) ~= 0 ifTrue: [' is misaligned'] ifFalse: [coInterpreter whereIs: oop])}!
Item was changed: ----- Method: SpurMemoryManager>>totalMemorySize (in category 'accessing') ----- totalMemorySize + ^scavenger newSpaceCapacity + segmentManager totalOldSpaceSize! - ^scavenger newSpaceCapacity + segmentManager totalBytesInSegments!
Item was changed: ----- Method: SpurPlanningCompactor>>unmarkPinned: (in category 'private') ----- unmarkPinned: pinnedObj <inline: true> + (manager isSegmentBridge: pinnedObj) + ifTrue: + [self assert: (manager isMarked: pinnedObj)] + ifFalse: + [manager setIsMarkedOf: pinnedObj to: false. + manager segmentManager notePinned: pinnedObj]! - (manager isSegmentBridge: pinnedObj) ifFalse: - [manager setIsMarkedOf: pinnedObj to: false. - manager segmentManager notePinned: pinnedObj]!
Item was changed: CogClass subclass: #SpurSegmentManager + instanceVariableNames: 'manager numSegments numSegInfos segments firstSegmentSize canSwizzle totalHeapSizeIncludingBridges' - instanceVariableNames: 'manager numSegments numSegInfos segments firstSegmentSize canSwizzle sweepIndex preferredPinningSegment totalHeapSizeIncludingBridges' classVariableNames: '' poolDictionaries: '' category: 'VMMaker-SpurMemoryManager'!
+ !SpurSegmentManager commentStamp: 'eem 4/23/2023 15:19' prior: 0! - !SpurSegmentManager commentStamp: 'eem 6/7/2017 14:00' prior: 0! Instances of SpurSegmentManager manage oldSpace, which is organized as a sequence of segments. Segments can be obtained from the operating system and returned to the operating system when empty and shrinkage is required. Segments are kept invisible from the SpurMemoryManager by using "bridge" objects, "fake" pinned objects to bridge the gaps between segments. A pinned object header occupies the last 16 bytes of each segment, and the pinned object's size is the distance to the start of the next segment. So when the memory manager enumerates objects it skips over these bridges and memory appears linear. The constraint is that segments obtained from the operating system must be at a higher address than the first segment. The maximum size of large objects, being an overflow slot size, should be big enough to bridge the gaps, because in 32-bits the maximum size is 2^32 slots. In 64-bits the maximum size of large objects is 2^56 slots, or 2^59 bits, which we hope will suffice.
When an image is written to a snapshot file the second word of the header of the bridge at the end of each segment is replaced by the size of the following segment, the segments are written to the file, and the second word of each bridge is restored. Hence the length of each segment is derived from the bridge at the end of the preceeding segment. The length of the first segment is stored in the image header as firstSegmentBytes. The start of each segment is also derived from the bridge as a delta from the start of the previous segment. The start of The first segment is stored in the image header as startOfMemory.
On load all segments are read into one single segment, eliminating the bridge objects, and computing the swizzle distance for each segment, based on where the segments were in memory when the image file was written, and where the coalesced segment ends up on load. Then the segment is traversed, swizzling pointers by selecting the relevant swizzle for each oop's segment.
Instance Variables + manager <SpurMemoryManager> + numSegments <Integer> + numSegInfos <Integer> + segments <Array of SpurSegmentInfo> + firstSegmentSize <Integer> + canSwizzle <Boolean> - manager <SpurMemoryManager> - numSegments <Integer> - numSegInfos <Integer> - segments <Array of SpurSegmentInfo> - firstSegmentSize <Integer> - canSwizzle <Boolean> - sweepIndex <Integer> - preferredPinningSegment <SpurSegmentInfo> totalHeapSizeIncludingBridges <integer>
canSwizzle - a flag set and cleared during initialization to validate that swizzling is only performed at the right time
firstSegmentSize - the size of the first segment when loading an image
manager - the memory manager the receiver manages segments for (simulation only) numSegInfos - the size of the segments array in units of SpurSegmentInfo size numSegments - the number of segments (the number of used entries in segments, <= numSegInfos)
- preferredPinningSegment - - the segment in which objects should be copied when pinned, so as to cluster pinned objects in as few segments as possible. As yet unimplemented. - segments - the start addresses, lengths and offsets to adjust oops on image load, for each segment
- sweepIndex - - a segment index used to optimize setting the containsPinned flag on segments during freeUnmarkedObjectsAndSortAndCoalesceFreeSpace - totalHeapSizeIncludingBridges - the total size of all segments, used to compute heap usage!
Item was changed: ----- Method: SpurSegmentManager>>collapseSegmentsPostSwizzle (in category 'snapshot') ----- collapseSegmentsPostSwizzle "The image has been loaded, old segments reconstructed, and the heap swizzled into a single contiguous segment. Collapse the segments into one." <inline: false> canSwizzle := false. self cCode: [] inSmalltalk: [segments ifNil: [self allocateOrExtendSegmentInfos]]. numSegments := 1. + self computeTotalHeapSizeIncludingBridges. (segments at: 0) segStart: manager oldSpaceStart; + segSize: totalHeapSizeIncludingBridges. - segSize: (totalHeapSizeIncludingBridges := manager endOfMemory - manager oldSpaceStart). manager bootstrapping ifTrue: ["finally plant a bridge at the end of the coalesced segment and cut back the manager's notion of the end of memory to immediately before the bridge." self assert: manager endOfMemory = (segments at: 0) segLimit. manager initSegmentBridgeWithBytes: manager bridgeSize at: manager endOfMemory - manager bridgeSize]. self assert: (manager isSegmentBridge: (self bridgeAt: 0)). self assert: (manager numSlotsOfAny: (self bridgeAt: 0)) = 0!
Item was changed: ----- Method: SpurSegmentManager>>initialize (in category 'initialization') ----- initialize + numSegments := numSegInfos := totalHeapSizeIncludingBridges := 0. - numSegments := numSegInfos := sweepIndex := totalHeapSizeIncludingBridges := 0. canSwizzle := false!
Item was changed: ----- Method: SpurSegmentManager>>notePinned: (in category 'pinning') ----- notePinned: objOop "Let the segmentManager mark which segments contain pinned objects" + | seg | self assert: (manager isPinned: objOop). + self deny: (manager isSegmentBridge: objOop). + seg := self segmentContainingObj: objOop. + seg containsPinned: true! - (manager isSegmentBridge: objOop) - ifTrue: - [manager setIsMarkedOf: objOop to: true] - ifFalse: - [[self oop: (segments at: sweepIndex) segLimit isLessThan: objOop] whileTrue: - [sweepIndex := sweepIndex + 1]. - (segments at: sweepIndex) containsPinned: true]!
Item was changed: ----- Method: SpurSegmentManager>>prepareForGlobalSweep (in category 'pinning') ----- prepareForGlobalSweep "Let the segmentManager mark which segments contain pinned objects via notePinned:. For coallesceFreeChunk:, ensure that the last bridge is marked." - sweepIndex := 0. 0 to: numSegments - 1 do: [:i| (segments at: i) containsPinned: false]. manager setIsMarkedOf: (self bridgeAt: numSegments - 1) to: true!
Item was changed: ----- Method: SpurSegmentManager>>totalBytesInSegments (in category 'accessing') ----- totalBytesInSegments + "This ``slow'' count is for asserts only." | total | <var: #total type: #usqInt> total := 0. 0 to: numSegments - 1 do: [:i| total := total + (segments at: i) segSize]. - self assert: totalHeapSizeIncludingBridges = total. ^total!
Item was changed: ----- Method: SpurSegmentManager>>totalOldSpaceCapacity (in category 'accessing') ----- totalOldSpaceCapacity + self assert: self totalBytesInSegments = totalHeapSizeIncludingBridges. ^totalHeapSizeIncludingBridges - (numSegments * manager bridgeSize)!
Item was changed: ----- Method: SpurSegmentManager>>totalOldSpaceSize (in category 'accessing') ----- totalOldSpaceSize + self assert: self totalBytesInSegments = totalHeapSizeIncludingBridges. ^totalHeapSizeIncludingBridges!
Item was changed: ----- Method: SpurSweeper>>unmark: (in category 'sweep phase') ----- unmark: objOop self assert: ((manager isMarked: objOop) and: [(manager isFreeObject: objOop) not]). + (manager isSegmentBridge: objOop) + ifTrue: [self assert: (manager isMarked: objOop)] + ifFalse: + [manager setIsMarkedOf: objOop to: false. + (manager isPinned: objOop) ifTrue: + [manager segmentManager notePinned: objOop]]! - (manager isSegmentBridge: objOop) ifFalse: [manager setIsMarkedOf: objOop to: false]. - (manager isPinned: objOop) ifTrue: [manager segmentManager notePinned: objOop]!
vm-dev@lists.squeakfoundation.org