Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog.seperateMarking-eem.3326.mcz
==================== Summary ====================
Name: VMMaker.oscog.seperateMarking-eem.3326 Author: eem Time: 24 April 2023, 3:01:21.399043 pm UUID: 6b815055-42dc-44ac-bf20-b35bbfd84204 Ancestors: VMMaker.oscog.seperateMarking-eem.3325
Add setIsMarkedAndIsGreyOf:to: for single write update of mark/grey bits.
Merge VMMaker.oscog-eem.3320 through VMMaker.oscog-eem.3324
=============== Diff against VMMaker.oscog.seperateMarking-eem.3325 ===============
Item was changed: ----- Method: CoInterpreter>>printMethodFieldForPrintContext: (in category 'debug printing') ----- printMethodFieldForPrintContext: aContext <inline: true> | meth | meth := objectMemory fetchPointer: MethodIndex ofObject: aContext. + '%P: ' f: transcript printf: meth asVoidPointer. - '%P: ' f: transcript printf: meth. self printOopShortInner: meth. (self methodHasCogMethod: meth) ifTrue: [' (%P)' f: transcript printf: (self cogMethodOf: meth)]. self cr!
Item was added: + ----- Method: Spur32BitMemoryManager>>setIsMarkedAndIsGreyOf:to: (in category 'header access') ----- + setIsMarkedAndIsGreyOf: objOop to: aBoolean + self assert: (self isFreeObject: objOop) not. + gc assertSettingGCFlagsIsOk: objOop. + + self flag: #endianness. + self longAt: objOop + put: (aBoolean + ifTrue: [(self longAt: objOop) bitOr: 1 << self greyBitShift] + ifFalse: [(self longAt: objOop) bitAnd: (1 << self greyBitShift) bitInvert32]). + self longAt: objOop + 4 + put: (aBoolean + ifTrue: [(self longAt: objOop + 4) bitOr: 1 << self markedBitHalfShift] + ifFalse: [(self longAt: objOop + 4) bitAnd: (1 << self markedBitHalfShift) bitInvert32])!
Item was added: + ----- Method: Spur64BitMemoryManager>>setIsMarkedAndIsGreyOf:to: (in category 'header access') ----- + setIsMarkedAndIsGreyOf: objOop to: aBoolean + self deny: (self isFreeObject: objOop). + gc assertSettingGCFlagsIsOk: objOop. + + self longAt: objOop + put: (aBoolean + ifTrue: [(self longAt: objOop) bitOr: (1 << self markedBitFullShift) + (1 << self greyBitShift)] + ifFalse: [(self longAt: objOop) bitAnd: ((1 << self markedBitFullShift) + (1 << self greyBitShift)) bitInvert64])!
Item was changed: ----- Method: SpurIncrementalCompactingSweeper>>unmark: (in category 'incremental sweeping') ----- unmark: objOop
self assert: ((manager isMarked: objOop) and: [(manager isFreeObject: objOop) not]). + (manager isSegmentBridge: objOop) + ifTrue: [self assert: (manager isMarked: objOop)] + ifFalse: + [manager setIsMarkedAndIsGreyOf: objOop to: false. + (manager isPinned: objOop) ifTrue: + [manager segmentManager notePinned: objOop]]! - (manager isSegmentBridge: objOop) - ifFalse: [manager - setIsMarkedOf: objOop to: false; - setIsGreyOf: objOop to: false]. - - (manager isPinned: objOop) - ifTrue: [manager segmentManager notePinned: objOop]!
Item was changed: ----- Method: SpurIncrementalSweeper>>unmark: (in category 'api') ----- unmark: objOop
self assert: ((manager isMarked: objOop) and: [(manager isFreeObject: objOop) not]). + (manager isSegmentBridge: objOop) + ifTrue: + [self assert: (manager isMarked: objOop)] + ifFalse: + [manager setIsMarkedAndIsGreyOf: objOop to: false. + (manager isPinned: objOop) ifTrue: + [manager segmentManager notePinned: objOop]]! - (manager isSegmentBridge: objOop) ifFalse: [ - manager - setIsMarkedOf: objOop to: false; - setIsGreyOf: objOop to: false]. - (manager isPinned: objOop) ifTrue: [manager segmentManager notePinned: objOop]!
Item was changed: ----- Method: SpurMemoryManager>>growOldSpaceByAtLeast: (in category 'growing/shrinking memory') ----- growOldSpaceByAtLeast: minAmmount "Attempt to grow memory by at least minAmmount. Answer the size of the new segment, or nil if the attempt failed." | ammount headroom total start interval | <var: #segInfo type: #'SpurSegmentInfo *'> "statGrowMemory counts attempts, not successes." - "self debugger." statGrowMemory := statGrowMemory + 1."we need to include overhead for a new object header plus the segment bridge." ammount := minAmmount + (self baseHeaderSize * 2 + self bridgeSize). "round up to the nearest power of two." ammount := 1 << (ammount - 1) highBit. "and grow by at least growHeadroom." ammount := ammount max: growHeadroom.
"Now apply the maxOldSpaceSize limit, if one is in effect." maxOldSpaceSize > 0 ifTrue: + [total := segmentManager totalOldSpaceSize. - [total := segmentManager totalBytesInSegments. total >= maxOldSpaceSize ifTrue: [needGCFlag := true. ^nil]. headroom := maxOldSpaceSize - total. headroom < ammount ifTrue: [headroom < (minAmmount + (self baseHeaderSize * 2 + self bridgeSize)) ifTrue: [needGCFlag := true. ^nil]. ammount := headroom]]. start := coInterpreter ioUTCMicrosecondsNow. ^(segmentManager addSegmentOfSize: ammount) ifNil: [needGCFlag := true. nil] ifNotNil: [:segInfo| self assimilateNewSegment: segInfo. "and add the new free chunk to the free list; done here instead of in assimilateNewSegment: for the assert" self addFreeChunkWithBytes: segInfo segSize - self bridgeSize at: segInfo segStart. self assert: (self addressAfter: (self objectStartingAt: segInfo segStart)) = (segInfo segLimit - self bridgeSize). self checkFreeSpace: GCCheckFreeSpace. segmentManager checkSegments. interval := coInterpreter ioUTCMicrosecondsNow - start. interval > statMaxAllocSegmentTime ifTrue: [statMaxAllocSegmentTime := interval]. segInfo segSize]!
Item was changed: ----- Method: SpurMemoryManager>>objectsAccessibleFromRoots: (in category 'image segment in/out') ----- objectsAccessibleFromRoots: arrayOfRootsArg "This primitive is called from Squeak as... arrayOfRoots uniquelyAccessibleObjects
This primitive answers an array of the receiver and every object in its proper tree of subParts (ie, that is not refered to from anywhere else outside the tree).
The primitive can fail for the following reasons with the specified failure codes: PrimErrNoMemory: additional allocations failed"
<inline: false> | arrayOfRoots arrayOfObjects | - <var: 'segAddr' type: #usqInt>
self runLeakCheckerFor: GCCheckImageSegment.
"First scavenge to collect any new space garbage that refers to the graph." self scavengingGC. arrayOfRoots := self updatePostScavenge: arrayOfRootsArg. "Now compute the transitive closure, collecting the sequence of objects to be stored in the arrayOfObjects array. Included in arrayOfObjects are the arrayOfRoots and all its contents. All objects have been unmarked." arrayOfObjects := self objectsReachableFromRoots: arrayOfRoots. arrayOfObjects ifNil: [^self integerObjectOf: PrimErrNoMemory]. "If objectsReachableFromRoots: answers an integer there is not enough continuous free space in which to allocate the reachable objects. If there is sufficient free space then answer an error code to prompt a compacting GC and a retry." (self isIntegerObject: arrayOfObjects) ifTrue: [totalFreeOldSpace - self allocationUnit >= (self integerValueOf: arrayOfObjects) ifTrue: [^self integerObjectOf: PrimErrNeedCompaction]. ^self integerObjectOf: PrimErrNoMemory].
self assert: self allObjectsUnmarked. "work to be done when the incremental GC is written" self deny: (self forwardersIn: arrayOfObjects).
^arrayOfObjects!
Item was changed: ----- Method: SpurMemoryManager>>oldSpaceSize (in category 'accessing') ----- oldSpaceSize + ^segmentManager totalOldSpaceSize! - ^segmentManager totalBytesInSegments!
Item was added: + ----- Method: SpurMemoryManager>>printBridge:on: (in category 'debug printing interpreter support') ----- + printBridge: oop on: aStream + <var: 'aStream' type: #'FILE *'> + <inline: false> + '%P is a bridge hdr%d slot size %ul\n' + f: aStream + printf: { oop asVoidPointer. + (self hasOverflowHeader: oop) ifTrue: [16] ifFalse: [8]. + self numSlotsOfAny: oop}!
Item was changed: ----- Method: SpurMemoryManager>>printCantBeObject:on: (in category 'debug printing interpreter support') ----- printCantBeObject: oop on: aStream <var: 'aStream' type: #'FILE *'> + (oop bitAnd: self allocationUnit - 1) = 0 ifTrue: + [((self isInNewSpace: oop) + and: [self isForwarded: oop]) ifTrue: + [^self printForwarder: oop on: aStream]. + oop = (segmentManager bridgeAt: segmentManager numSegments - 1) ifTrue: + [^self printBridge: oop on: aStream]]. - ((oop bitAnd: self allocationUnit - 1) = 0 - and: [(self isInNewSpace: oop) - and: [self isForwarded: oop]]) ifTrue: - [^self printForwarder: oop on: aStream]. '%P%s\n' f: aStream printf: {oop asVoidPointer. ((oop bitAnd: self allocationUnit - 1) ~= 0 ifTrue: [' is misaligned'] ifFalse: [coInterpreter whereIs: oop])}!
Item was changed: ----- Method: SpurMemoryManager>>printForwarder:on: (in category 'debug printing interpreter support') ----- printForwarder: oop on: aStream <var: 'aStream' type: #'FILE *'> <inline: false> + '%P is a forwarded hdr%d slot size %ul object to %P\n' - '%P is a forwarded hdr%d slot size %ud object to %P\n' f: aStream printf: { oop asVoidPointer. (self hasOverflowHeader: oop) ifTrue: [16] ifFalse: [8]. self numSlotsOfAny: oop. (self followForwarded: oop) asVoidPointer}!
Item was added: + ----- Method: SpurMemoryManager>>setIsMarkedAndIsGreyOf:to: (in category 'header access') ----- + setIsMarkedAndIsGreyOf: objOop to: aBoolean + self subclassResponsibility!
Item was changed: ----- Method: SpurMemoryManager>>totalMemorySize (in category 'accessing') ----- totalMemorySize + ^scavenger newSpaceCapacity + segmentManager totalOldSpaceSize! - ^scavenger newSpaceCapacity + segmentManager totalBytesInSegments!
Item was changed: ----- Method: SpurPlanningCompactor>>unmarkPinned: (in category 'private') ----- unmarkPinned: pinnedObj <inline: true> + (manager isSegmentBridge: pinnedObj) + ifTrue: + [self assert: (manager isMarked: pinnedObj)] + ifFalse: + [manager setIsMarkedOf: pinnedObj to: false. + manager segmentManager notePinned: pinnedObj]! - (manager isSegmentBridge: pinnedObj) ifFalse: - [manager setIsMarkedOf: pinnedObj to: false. - manager segmentManager notePinned: pinnedObj]!
Item was changed: CogClass subclass: #SpurSegmentManager + instanceVariableNames: 'manager numSegments numSegInfos segments firstSegmentSize canSwizzle totalHeapSizeIncludingBridges' - instanceVariableNames: 'manager numSegments numSegInfos segments firstSegmentSize canSwizzle sweepIndex preferredPinningSegment totalHeapSizeIncludingBridges' classVariableNames: '' poolDictionaries: '' category: 'VMMaker-SpurMemoryManager'!
+ !SpurSegmentManager commentStamp: 'eem 4/23/2023 15:19' prior: 0! - !SpurSegmentManager commentStamp: '' prior: 0! Instances of SpurSegmentManager manage oldSpace, which is organized as a sequence of segments. Segments can be obtained from the operating system and returned to the operating system when empty and shrinkage is required. Segments are kept invisible from the SpurMemoryManager by using "bridge" objects, "fake" pinned objects to bridge the gaps between segments. A pinned object header occupies the last 16 bytes of each segment, and the pinned object's size is the distance to the start of the next segment. So when the memory manager enumerates objects it skips over these bridges and memory appears linear. The constraint is that segments obtained from the operating system must be at a higher address than the first segment. The maximum size of large objects, being an overflow slot size, should be big enough to bridge the gaps, because in 32-bits the maximum size is 2^32 slots. In 64-bits the maximum size of large objects is 2^56 slots, or 2^59 bits, which we hope will suffice.
When an image is written to a snapshot file the second word of the header of the bridge at the end of each segment is replaced by the size of the following segment, the segments are written to the file, and the second word of each bridge is restored. Hence the length of each segment is derived from the bridge at the end of the preceeding segment. The length of the first segment is stored in the image header as firstSegmentBytes. The start of each segment is also derived from the bridge as a delta from the start of the previous segment. The start of The first segment is stored in the image header as startOfMemory.
On load all segments are read into one single segment, eliminating the bridge objects, and computing the swizzle distance for each segment, based on where the segments were in memory when the image file was written, and where the coalesced segment ends up on load. Then the segment is traversed, swizzling pointers by selecting the relevant swizzle for each oop's segment.
Instance Variables + manager <SpurMemoryManager> + numSegments <Integer> + numSegInfos <Integer> + segments <Array of SpurSegmentInfo> + firstSegmentSize <Integer> + canSwizzle <Boolean> - manager <SpurMemoryManager> - numSegments <Integer> - numSegInfos <Integer> - segments <Array of SpurSegmentInfo> - firstSegmentSize <Integer> - canSwizzle <Boolean> - sweepIndex <Integer> - preferredPinningSegment <SpurSegmentInfo> totalHeapSizeIncludingBridges <integer>
canSwizzle - a flag set and cleared during initialization to validate that swizzling is only performed at the right time
firstSegmentSize - the size of the first segment when loading an image
manager - the memory manager the receiver manages segments for (simulation only) numSegInfos - the size of the segments array in units of SpurSegmentInfo size numSegments - the number of segments (the number of used entries in segments, <= numSegInfos)
- preferredPinningSegment - - the segment in which objects should be copied when pinned, so as to cluster pinned objects in as few segments as possible. As yet unimplemented. - segments - the start addresses, lengths and offsets to adjust oops on image load, for each segment
- sweepIndex - - a segment index used to optimize setting the containsPinned flag on segments during freeUnmarkedObjectsAndSortAndCoalesceFreeSpace - totalHeapSizeIncludingBridges - the total size of all segments, used to compute heap usage!
Item was changed: ----- Method: SpurSegmentManager>>collapseSegmentsPostSwizzle (in category 'snapshot') ----- collapseSegmentsPostSwizzle "The image has been loaded, old segments reconstructed, and the heap swizzled into a single contiguous segment. Collapse the segments into one." <inline: false> canSwizzle := false. self cCode: [] inSmalltalk: [segments ifNil: [self allocateOrExtendSegmentInfos]]. numSegments := 1. + self computeTotalHeapSizeIncludingBridges. (segments at: 0) segStart: manager oldSpaceStart; + segSize: totalHeapSizeIncludingBridges. - segSize: (totalHeapSizeIncludingBridges := manager endOfMemory - manager oldSpaceStart). manager bootstrapping ifTrue: ["finally plant a bridge at the end of the coalesced segment and cut back the manager's notion of the end of memory to immediately before the bridge." self assert: manager endOfMemory = (segments at: 0) segLimit. manager initSegmentBridgeWithBytes: manager bridgeSize at: manager endOfMemory - manager bridgeSize]. self assert: (manager isSegmentBridge: (self bridgeAt: 0)). self assert: (manager numSlotsOfAny: (self bridgeAt: 0)) = 1!
Item was changed: ----- Method: SpurSegmentManager>>initialize (in category 'initialization') ----- initialize + numSegments := numSegInfos := totalHeapSizeIncludingBridges := 0. - numSegments := numSegInfos := sweepIndex := totalHeapSizeIncludingBridges := 0. canSwizzle := false!
Item was changed: ----- Method: SpurSegmentManager>>notePinned: (in category 'pinning') ----- notePinned: objOop "Let the segmentManager mark which segments contain pinned objects" + | seg | self assert: (manager isPinned: objOop). + self deny: (manager isSegmentBridge: objOop). + seg := self segmentContainingObj: objOop. + seg containsPinned: true! - (manager isSegmentBridge: objOop) - ifTrue: - [manager setIsMarkedOf: objOop to: true] - ifFalse: - [[self oop: (segments at: sweepIndex) segLimit isLessThan: objOop] whileTrue: - [sweepIndex := sweepIndex + 1]. - (segments at: sweepIndex) containsPinned: true]!
Item was changed: ----- Method: SpurSegmentManager>>prepareForGlobalSweep (in category 'pinning') ----- prepareForGlobalSweep "Let the segmentManager mark which segments contain pinned objects via notePinned:. For coallesceFreeChunk:, ensure that the last bridge is marked." - sweepIndex := 0. 0 to: numSegments - 1 do: [:i| (segments at: i) containsPinned: false]. manager setIsMarkedOf: (self bridgeAt: numSegments - 1) to: true!
Item was changed: ----- Method: SpurSegmentManager>>segmentContainingObj: (in category 'accessing') ----- segmentContainingObj: objOop "Answer the segment containing an object. This is mostly for assert checking, but variations on the incremental GC may use it in anger. Binary search is (of course) marginally slower than linear search for a single segment (e.g. in a 720k object heap, 67.1ms vs 61.3ms, or 9.5% slower to derive the segment containing every old space entity), but usefully faster for many segments (e.g. 92.7ms vs 116ms, or 20% faster in the same heap extended with enough large arrays to require 11 segments; and this is pessimal; there are fewer objects at high addresses since the large arrays are there)." <export: true> <returnTypeC: #'SpurSegmentInfo *'> | high low mid seg | low := 0. mid := numSegments // 2. high := numSegments - 1. + [seg := self addressOf: (segments at: mid). - [seg := segments at: mid. (self oop: objOop isGreaterThanOrEqualTo: seg segStart) ifTrue: [mid = high ifTrue: [^(self oop: objOop isLessThan: seg segLimit) ifTrue: [seg]] ifFalse: [low := mid. mid := mid + high + 1 // 2]] ifFalse: [high := mid - 1. mid := low + mid // 2]. low <= high] whileTrue. ^nil!
Item was changed: ----- Method: SpurSegmentManager>>totalBytesInSegments (in category 'accessing') ----- totalBytesInSegments + "This ``slow'' count is for asserts only." | total | <var: #total type: #usqInt> total := 0. 0 to: numSegments - 1 do: [:i| total := total + (segments at: i) segSize]. - self assert: totalHeapSizeIncludingBridges = total. ^total!
Item was changed: ----- Method: SpurSegmentManager>>totalOldSpaceCapacity (in category 'accessing') ----- totalOldSpaceCapacity + self assert: self totalBytesInSegments = totalHeapSizeIncludingBridges. ^totalHeapSizeIncludingBridges - (numSegments * manager bridgeSize)!
Item was changed: ----- Method: SpurSegmentManager>>totalOldSpaceSize (in category 'accessing') ----- totalOldSpaceSize + self assert: self totalBytesInSegments = totalHeapSizeIncludingBridges. ^totalHeapSizeIncludingBridges!
Item was changed: ----- Method: SpurSweeper>>unmark: (in category 'sweep phase') ----- unmark: objOop self assert: ((manager isMarked: objOop) and: [(manager isFreeObject: objOop) not]). + (manager isSegmentBridge: objOop) + ifTrue: [self assert: (manager isMarked: objOop)] + ifFalse: + [manager setIsMarkedOf: objOop to: false. + (manager isPinned: objOop) ifTrue: + [manager segmentManager notePinned: objOop]]! - (manager isSegmentBridge: objOop) ifFalse: [manager setIsMarkedOf: objOop to: false]. - (manager isPinned: objOop) ifTrue: [manager segmentManager notePinned: objOop]!
Item was changed: ----- Method: StackInterpreter>>printFrameOop:index:at: (in category 'debug printing') ----- printFrameOop: name index: idx at: address <var: #name type: #'char *'> <var: #address type: #'char *'> <inline: false> | it | it := stackPages longAt: address. self printFrameAddress: address. + (self pst: '%s%10s%ld: %WP\t') - (self pst: '%s%10s%d: %WP\t') f: transcript printf: { idx > 9 ifTrue: [''] ifFalse: [' ']. name. idx. it asVoidPointer }. self printOopShortInner: it; cr!
vm-dev@lists.squeakfoundation.org