Vm-dev April 2023

vm-dev@lists.squeakfoundation.org

10 participants
73 discussions

[OpenSmalltalk/opensmalltalk-vm] BitBlt rgbMul is broken for depth 1 and not correctly rounded above depth 4 (Issue #651)
by Nicolas Cellier 24 Nov '23

24 Nov '23

In depth 1, the resulting bits will always be 0. It's not a big problem because rgbMul is just a bitAnd operation at this depth. So a quick workaround would be to detect the case in BitBltSimulation destDepth = 1 ifTrue: [^self bitAnd: sourceWord with: destinationWord]. That would also accelerate the Bit BLock Transfer operation, so it's a good hack. But there is more. What we want is multiply ratios in interval [0,1]. dstRatio * srcRatio Our implementation is scaled ratio (scaled by `1 << nBits - 1`): src := (srcRatio * scale) rounded. dst := (dstRatio * scale) rounded. So what we want is: ((dst/scale) * (src/scale) * scale) rounded that is: (dst*src / (1<<nBits-1)) rounded Unfortunately, that's the other grief with the current implementation used for rounding: (dst+1)*(src+1) - 1 >> nBits It only equals correctly rounded operation for depths 2 and 4. For rounding we might use: (((dst/scale) * (src/scale) + 0.5) * scale) truncated. that is expressed with truncated division: dst*src + (scale+1//2) // scale So here is a nicer formulation for doing the job at any depth (including 5bits rgb channels for 16 bits depth) with correctly rounded division: aux := src * dst + (1 << (nBits - 1)). "add mid-scale for rounding" result := aux << (nBits - 1) + aux << (nBits -1). "divide by scale" This is because instead of dividing by scale, we can multiply by shifted inverse (sort of double precision), then shift right. (2 to: 32) allSatisfy: [:nBits | (1 << (nBits * 2) / (1 << nBits - 1)) rounded = (1 << nBits + 1)]. Multiplying by this inverse is easy and cheap: x * (1 << nBits + 1) = (x << nBits + x). And then applying the right shift `>> (2 * nBits)` is equivalent to: x >> nBits + x >> nBits. We must first add 0.5 (scaled), that is `src * dst + (1 << (nBits -1))` - our formulation of aux, and we're done. We verify: { (0 to: 1<<20-1) allSatisfy: [:i | (1<<9+i)>>10+ (1<<9+i)>>10 = (i/1023) rounded]. (0 to: 1<<18-1) allSatisfy: [:i | (1<<8+i)>>9+ (1<<8+i)>>9 = (i/511) rounded]. (0 to: 1<<16-1) allSatisfy: [:i | (1<<7+i)>>8+ (1<<7+i)>>8 = (i/255) rounded]. (0 to: 1<<14-1) allSatisfy: [:i | (1<<6+i)>>7+ (1<<6+i)>>7 = (i/127) rounded]. (0 to: 1<<12-1) allSatisfy: [:i | (1<<5+i)>>6+ (1<<5+i)>>6 = (i/63) rounded]. (0 to: 1<<10-1) allSatisfy: [:i | (1<<4+i)>>5+ (1<<4+i)>>5 = (i/31) rounded]. (0 to: 1<<8-1) allSatisfy: [:i | (1<<3+i)>>4+ (1<<3+i)>>4 = (i/15) rounded]. (0 to: 1<<6-1) allSatisfy: [:i | (1<<2+i)>>3+ (1<<2+i)>>3 = (i/7) rounded]. (0 to: 1<<4-1) allSatisfy: [:i | (1<<1+i)>>2+ (1<<1+i)>>2 = (i/3) rounded]. } allSatisfy: #yourself. The nice thing is that above down-scaling operation can be multiplexed. Suppose that we have p groups of 2*nBits `M` holding square-scale multiplication of each channel concatenated in a double-Word-Mul. doubleWordMul = Mp .... M5 M3 M1 Note we arrange to have odd channels in low word, and even channels in high word. We first form a `groupMask` on a word with (p+1)/2 groups of nBits alternating all one `i` and all zero `o`, `oioi...ioi`. channelMask := 1 << nBits - 1. groupMask := 0. 0 to: wordBits // (2 * nBits) do: [:i | groupMask = groupMask << (2 * nBits) + channelMask]. Where wordBits is the number of bits in a word (usually we want to operate on 32 bits words in BitBlt). We form the `doubleGroupMask` on a double-word with p groups of 2*nBits `oi`: doubleGroupMask := groupMask >> nBits. doubleGroupMask := doubleGroupMask << wordBits + groupMask. And we perform the division by scale: doubleWordMul := (doubleWordMul >> nBits bitAnd: doubleGroupMask) + doubleWord >> nBits bitAnd: doubleGroupMask. At this stage we obtain a double word containing scaled multiplicands interleaved with groups of nBits zeros: o mp ... o m3 o m1 Now the final result can be obtained by shifting back: doubleWordMul >> (wordBits - nBits) + (doubleWordMul bitAnd: groupMask) The only problem remaining is how to obtain the squared-scale multiplicands. It would be easy to form the alternate even-odd channels for each src and dst operands: doubleWordSrc := src >> nBits bitAnd: groupMask. doubleWordSrc := doubleWordSrc << wordBits + (src bitAnd: groupMask). doubleWordDst := dst >> nBits bitAnd: groupMask. doubleWordDst := doubleWordDst << wordBits + (dst bitAnd: groupMask). we now get `o sp ... o s3 o s1` and `0 dp ... o d3 o d1`, but we would now need a SIMD integer multiplication operating on groups of 2*nBits in parallel... We don't have that, at least in portable C code. So we still have to emulate it with a loop. half := 1 << (nBits - 1). shift := 0. doubleWordMul := 0 0 to: nChannels - 1 do: [:i | doubleWordMul := doubleWordMul + (((doubleWordSrc >> shift bitAnd: channelMask) * (doubleWordSrc >> shift bitAnd: channelMask) + half) << shift). shift := shift + nBits + nBits]. We know that each operation cannot overflow on upper neighbour group of 2*nBits, because the maximum value is: (1<<nBits-1) squared + (1 << (nBits-1)) = 1 << (2*nBits) - (2*(1<<nBits)) + (1 << (nBits-1)) - 1 < (1 << (2*nBits) - 1) It remains the odd case of 16 bits depth, which has 3 groups of 5 bits and a leading zero. I believe that above algorithm works without splitting in two half-words... To be tested. We have gathered the pieces for a correctly rounded almost-multiplexed rgbMul. Somehow have our cake and eat it too. -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/651 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/651(a)github.com>

2 4

Re: [Newbies] FreeBSD VM
by Tobias Pape 30 Sep '23

30 Sep '23

Hi David > On 30. Mar 2023, at 16:09, David Gray <dgray(a)iesl.forth.gr> wrote: > > I’ve been trying to get a working VM for FreeBSD. However even though I manage to get the system to start and update itself it gives a segmentation error when I open a workspace and type any character at all. I used the latest linux64x64 cog.spur. > Both the 6.0 and 5.3 images behaved the same. To get it to compile I had the comment out the Joystick plugin and edit the baud rates inside the serial plugin. I’m a complete beginner with squeak so I don’t know how to proceed from here. Based on your comments, I went and tried to verify that the VM still builds. I have made similar adjustments and was able to build a VM with the recents commits to the Cog branch of the VM. Can you please build a debug vm and try to recreate the error? I am very interested in that. Best regards -Tobias

4 5

In reference to: Minifying Woes
by Tom Braun 28 Aug '23

28 Aug '23

Hi all, I am new to the list, so I couldn’t answer directly to Minifying Woes (or at least didn’t know how to). I identified 75 VM intern objects in Tom’s (tobe) image, by loading it in the simulator and setting a halt after loading the image. With the following code: | collection | collection := OrderedCollection new. self allOldSpaceEntitiesDo: [:obj | (((self classIndexOf: obj) < self lastClassIndexPun) and: (self isImmediate: obj) not) ifTrue: [ collection add: obj] ]. (collection sorted: [:a :b | (self bytesInBody: a) > (self bytesInBody: b)]) collect: [:ea | ea hex -> (self bytesInBody: ea)] I get the following 75 objects: 1. A free chunk after all other objects. Can be ignored for the sake of minimising image size 2. Remembered set -> 1048592 byte 3. hiddenRootsObj -> 32848 byte 4 - 61. Pages of the mark and weakling stack -> respectively 32752 byte 62 - 74. arrays of the class table -> respectively 8208 byte 75. specialObjectsOop -> 520 byte As far as I can judge the stack pages and remembered set could be removed from the image to minimize it further. If I understand correctly the StackPages could be removed by using the SpurImagePreener. The remembered set Could be removed in the SpurImagePreener too (when the VM initialises the memory it initialises a new remembered set too, if it is nil). After a quick read of the SpurImagePreener I didn’t see that the remembered set gets removed. When I tried using the preener it resulted in an unusable image (both simulator and compiled VM couldn’t load it). I tried both: SpurImagePreener new preenImage: '/Users/tombraun/Desktop/Squeak6.0-22104-64bit copy.image' SpurImagePreener new writeDefaultHeader: true; savedWindowSize: 1@1; preenImage: '/Users/tombraun/Desktop/Squeak6.0-22104-64bit copy.image' Could be a me problem, as I did some changes to the memory management in my VMMaker image, although this shouldn’t influence the preener…. On this note @Eliot: why the decision to make the object stacks and the remembered set VM managed objects instead of allocating them separately? 1. We don’t need to keep them in a snapshot. All object stacks are empty after GC and as we flushed the new space pre snapshot The remembered set shouldn't need to be persisted too. 2. During GC we simply mark all stack pages and keep them alive. When we at least freed empty pages (after a limit, to prevent the running VM from having to allocate too many pages every GC?) I would see the value. What did I overlook or does it simply have historical reasons? Best, Tom (WoC)

5 5

[OpenSmalltalk/opensmalltalk-vm] Allow choosing an image via zenity when no arguments are provided (#408)
by Leon Matthes 05 Jun '23

05 Jun '23

You can view, comment on, or merge this pull request online at: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/408 -- Commit Summary -- * Add file chooser dialog to squeak.sh.in * Add file chooser dialog to bin.squeak.sh.in -- File Changes -- M platforms/unix/config/bin.squeak.sh.in (13) M platforms/unix/config/squeak.sh.in (13) -- Patch Links -- https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/408.patch https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/408.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/408

4 8

[OpenSmalltalk/opensmalltalk-vm] 52b067: For win32, tweak ExtendedClipboard plugin as sugge...
by Marcel Taeumel 25 Apr '23

25 Apr '23

Branch: refs/heads/virtend Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: 52b067d42d29fe84500955dc59616654aa2037e2 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/52b067d42d29fe8450… Author: Marcel Taeumel <marcel.taeumel(a)hpi.de> Date: 2023-04-25 (Tue, 25 Apr 2023) Changed paths: M platforms/win32/plugins/ClipboardExtendedPlugin/sqWin32ExtendedClipboard.c Log Message: ----------- For win32, tweak ExtendedClipboard plugin as suggested by Eliot: (1) use "squeak" prefix instead of "sq" to avoid confusion with plugin-impl API, (2) make helper functions static and thus private to the compilation unit, (3) use sqInt instead of int to generate better code.

1 0

Re: [OpenSmalltalk/opensmalltalk-vm] For Windows extended clipboard plugin, add support for more format types including common MIME types. (ed1f743)
by Marcel Taeumel 25 Apr '23

25 Apr '23

Thanks for the review! Done via https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/6316198fb5b97c07b9… -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/ed1f7431a09f044488… You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/commit/ed1f7431a09f044488a85f9d275994d6def7b670/110456093(a)github.com>

1 0

[OpenSmalltalk/opensmalltalk-vm] 631619: For win32, tweak ExtendedClipboard plugin as sugge...
by Marcel Taeumel 25 Apr '23

25 Apr '23

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: 6316198fb5b97c07b9be9b49851f17d157cf756e https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/6316198fb5b97c07b9… Author: Marcel Taeumel <marcel.taeumel(a)hpi.de> Date: 2023-04-25 (Tue, 25 Apr 2023) Changed paths: M platforms/win32/plugins/ClipboardExtendedPlugin/sqWin32ExtendedClipboard.c Log Message: ----------- For win32, tweak ExtendedClipboard plugin as suggested by Eliot: (1) use "squeak" prefix instead of "sq" to avoid confusion with plugin-impl API, (2) make helper functions static and thus private to the compilation unit, (3) use sqInt instead of int to generate better code.

1 0

VM Maker: VMMaker.oscog.seperateMarking-eem.3326.mcz
by commits＠source.squeak.org 24 Apr '23

24 Apr '23

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog.seperateMarking-eem.3326.mcz ==================== Summary ==================== Name: VMMaker.oscog.seperateMarking-eem.3326 Author: eem Time: 24 April 2023, 3:01:21.399043 pm UUID: 6b815055-42dc-44ac-bf20-b35bbfd84204 Ancestors: VMMaker.oscog.seperateMarking-eem.3325 Add setIsMarkedAndIsGreyOf:to: for single write update of mark/grey bits. Merge VMMaker.oscog-eem.3320 through VMMaker.oscog-eem.3324 =============== Diff against VMMaker.oscog.seperateMarking-eem.3325 =============== Item was changed: ----- Method: CoInterpreter>>printMethodFieldForPrintContext: (in category 'debug printing') ----- printMethodFieldForPrintContext: aContext <inline: true> | meth | meth := objectMemory fetchPointer: MethodIndex ofObject: aContext. + '%P: ' f: transcript printf: meth asVoidPointer. - '%P: ' f: transcript printf: meth. self printOopShortInner: meth. (self methodHasCogMethod: meth) ifTrue: [' (%P)' f: transcript printf: (self cogMethodOf: meth)]. self cr! Item was added: + ----- Method: Spur32BitMemoryManager>>setIsMarkedAndIsGreyOf:to: (in category 'header access') ----- + setIsMarkedAndIsGreyOf: objOop to: aBoolean + self assert: (self isFreeObject: objOop) not. + gc assertSettingGCFlagsIsOk: objOop. + + self flag: #endianness. + self longAt: objOop + put: (aBoolean + ifTrue: [(self longAt: objOop) bitOr: 1 << self greyBitShift] + ifFalse: [(self longAt: objOop) bitAnd: (1 << self greyBitShift) bitInvert32]). + self longAt: objOop + 4 + put: (aBoolean + ifTrue: [(self longAt: objOop + 4) bitOr: 1 << self markedBitHalfShift] + ifFalse: [(self longAt: objOop + 4) bitAnd: (1 << self markedBitHalfShift) bitInvert32])! Item was added: + ----- Method: Spur64BitMemoryManager>>setIsMarkedAndIsGreyOf:to: (in category 'header access') ----- + setIsMarkedAndIsGreyOf: objOop to: aBoolean + self deny: (self isFreeObject: objOop). + gc assertSettingGCFlagsIsOk: objOop. + + self longAt: objOop + put: (aBoolean + ifTrue: [(self longAt: objOop) bitOr: (1 << self markedBitFullShift) + (1 << self greyBitShift)] + ifFalse: [(self longAt: objOop) bitAnd: ((1 << self markedBitFullShift) + (1 << self greyBitShift)) bitInvert64])! Item was changed: ----- Method: SpurIncrementalCompactingSweeper>>unmark: (in category 'incremental sweeping') ----- unmark: objOop self assert: ((manager isMarked: objOop) and: [(manager isFreeObject: objOop) not]). + (manager isSegmentBridge: objOop) + ifTrue: [self assert: (manager isMarked: objOop)] + ifFalse: + [manager setIsMarkedAndIsGreyOf: objOop to: false. + (manager isPinned: objOop) ifTrue: + [manager segmentManager notePinned: objOop]]! - (manager isSegmentBridge: objOop) - ifFalse: [manager - setIsMarkedOf: objOop to: false; - setIsGreyOf: objOop to: false]. - - (manager isPinned: objOop) - ifTrue: [manager segmentManager notePinned: objOop]! Item was changed: ----- Method: SpurIncrementalSweeper>>unmark: (in category 'api') ----- unmark: objOop self assert: ((manager isMarked: objOop) and: [(manager isFreeObject: objOop) not]). + (manager isSegmentBridge: objOop) + ifTrue: + [self assert: (manager isMarked: objOop)] + ifFalse: + [manager setIsMarkedAndIsGreyOf: objOop to: false. + (manager isPinned: objOop) ifTrue: + [manager segmentManager notePinned: objOop]]! - (manager isSegmentBridge: objOop) ifFalse: [ - manager - setIsMarkedOf: objOop to: false; - setIsGreyOf: objOop to: false]. - (manager isPinned: objOop) ifTrue: [manager segmentManager notePinned: objOop]! Item was changed: ----- Method: SpurMemoryManager>>growOldSpaceByAtLeast: (in category 'growing/shrinking memory') ----- growOldSpaceByAtLeast: minAmmount "Attempt to grow memory by at least minAmmount. Answer the size of the new segment, or nil if the attempt failed." | ammount headroom total start interval | <var: #segInfo type: #'SpurSegmentInfo *'> "statGrowMemory counts attempts, not successes." - "self debugger." statGrowMemory := statGrowMemory + 1."we need to include overhead for a new object header plus the segment bridge." ammount := minAmmount + (self baseHeaderSize * 2 + self bridgeSize). "round up to the nearest power of two." ammount := 1 << (ammount - 1) highBit. "and grow by at least growHeadroom." ammount := ammount max: growHeadroom. "Now apply the maxOldSpaceSize limit, if one is in effect." maxOldSpaceSize > 0 ifTrue: + [total := segmentManager totalOldSpaceSize. - [total := segmentManager totalBytesInSegments. total >= maxOldSpaceSize ifTrue: [needGCFlag := true. ^nil]. headroom := maxOldSpaceSize - total. headroom < ammount ifTrue: [headroom < (minAmmount + (self baseHeaderSize * 2 + self bridgeSize)) ifTrue: [needGCFlag := true. ^nil]. ammount := headroom]]. start := coInterpreter ioUTCMicrosecondsNow. ^(segmentManager addSegmentOfSize: ammount) ifNil: [needGCFlag := true. nil] ifNotNil: [:segInfo| self assimilateNewSegment: segInfo. "and add the new free chunk to the free list; done here instead of in assimilateNewSegment: for the assert" self addFreeChunkWithBytes: segInfo segSize - self bridgeSize at: segInfo segStart. self assert: (self addressAfter: (self objectStartingAt: segInfo segStart)) = (segInfo segLimit - self bridgeSize). self checkFreeSpace: GCCheckFreeSpace. segmentManager checkSegments. interval := coInterpreter ioUTCMicrosecondsNow - start. interval > statMaxAllocSegmentTime ifTrue: [statMaxAllocSegmentTime := interval]. segInfo segSize]! Item was changed: ----- Method: SpurMemoryManager>>objectsAccessibleFromRoots: (in category 'image segment in/out') ----- objectsAccessibleFromRoots: arrayOfRootsArg "This primitive is called from Squeak as... arrayOfRoots uniquelyAccessibleObjects This primitive answers an array of the receiver and every object in its proper tree of subParts (ie, that is not refered to from anywhere else outside the tree). The primitive can fail for the following reasons with the specified failure codes: PrimErrNoMemory: additional allocations failed" <inline: false> | arrayOfRoots arrayOfObjects | - <var: 'segAddr' type: #usqInt> self runLeakCheckerFor: GCCheckImageSegment. "First scavenge to collect any new space garbage that refers to the graph." self scavengingGC. arrayOfRoots := self updatePostScavenge: arrayOfRootsArg. "Now compute the transitive closure, collecting the sequence of objects to be stored in the arrayOfObjects array. Included in arrayOfObjects are the arrayOfRoots and all its contents. All objects have been unmarked." arrayOfObjects := self objectsReachableFromRoots: arrayOfRoots. arrayOfObjects ifNil: [^self integerObjectOf: PrimErrNoMemory]. "If objectsReachableFromRoots: answers an integer there is not enough continuous free space in which to allocate the reachable objects. If there is sufficient free space then answer an error code to prompt a compacting GC and a retry." (self isIntegerObject: arrayOfObjects) ifTrue: [totalFreeOldSpace - self allocationUnit >= (self integerValueOf: arrayOfObjects) ifTrue: [^self integerObjectOf: PrimErrNeedCompaction]. ^self integerObjectOf: PrimErrNoMemory]. self assert: self allObjectsUnmarked. "work to be done when the incremental GC is written" self deny: (self forwardersIn: arrayOfObjects). ^arrayOfObjects! Item was changed: ----- Method: SpurMemoryManager>>oldSpaceSize (in category 'accessing') ----- oldSpaceSize + ^segmentManager totalOldSpaceSize! - ^segmentManager totalBytesInSegments! Item was added: + ----- Method: SpurMemoryManager>>printBridge:on: (in category 'debug printing interpreter support') ----- + printBridge: oop on: aStream + <var: 'aStream' type: #'FILE *'> + <inline: false> + '%P is a bridge hdr%d slot size %ul\n' + f: aStream + printf: { oop asVoidPointer. + (self hasOverflowHeader: oop) ifTrue: [16] ifFalse: [8]. + self numSlotsOfAny: oop}! Item was changed: ----- Method: SpurMemoryManager>>printCantBeObject:on: (in category 'debug printing interpreter support') ----- printCantBeObject: oop on: aStream <var: 'aStream' type: #'FILE *'> + (oop bitAnd: self allocationUnit - 1) = 0 ifTrue: + [((self isInNewSpace: oop) + and: [self isForwarded: oop]) ifTrue: + [^self printForwarder: oop on: aStream]. + oop = (segmentManager bridgeAt: segmentManager numSegments - 1) ifTrue: + [^self printBridge: oop on: aStream]]. - ((oop bitAnd: self allocationUnit - 1) = 0 - and: [(self isInNewSpace: oop) - and: [self isForwarded: oop]]) ifTrue: - [^self printForwarder: oop on: aStream]. '%P%s\n' f: aStream printf: {oop asVoidPointer. ((oop bitAnd: self allocationUnit - 1) ~= 0 ifTrue: [' is misaligned'] ifFalse: [coInterpreter whereIs: oop])}! Item was changed: ----- Method: SpurMemoryManager>>printForwarder:on: (in category 'debug printing interpreter support') ----- printForwarder: oop on: aStream <var: 'aStream' type: #'FILE *'> <inline: false> + '%P is a forwarded hdr%d slot size %ul object to %P\n' - '%P is a forwarded hdr%d slot size %ud object to %P\n' f: aStream printf: { oop asVoidPointer. (self hasOverflowHeader: oop) ifTrue: [16] ifFalse: [8]. self numSlotsOfAny: oop. (self followForwarded: oop) asVoidPointer}! Item was added: + ----- Method: SpurMemoryManager>>setIsMarkedAndIsGreyOf:to: (in category 'header access') ----- + setIsMarkedAndIsGreyOf: objOop to: aBoolean + self subclassResponsibility! Item was changed: ----- Method: SpurMemoryManager>>totalMemorySize (in category 'accessing') ----- totalMemorySize + ^scavenger newSpaceCapacity + segmentManager totalOldSpaceSize! - ^scavenger newSpaceCapacity + segmentManager totalBytesInSegments! Item was changed: ----- Method: SpurPlanningCompactor>>unmarkPinned: (in category 'private') ----- unmarkPinned: pinnedObj <inline: true> + (manager isSegmentBridge: pinnedObj) + ifTrue: + [self assert: (manager isMarked: pinnedObj)] + ifFalse: + [manager setIsMarkedOf: pinnedObj to: false. + manager segmentManager notePinned: pinnedObj]! - (manager isSegmentBridge: pinnedObj) ifFalse: - [manager setIsMarkedOf: pinnedObj to: false. - manager segmentManager notePinned: pinnedObj]! Item was changed: CogClass subclass: #SpurSegmentManager + instanceVariableNames: 'manager numSegments numSegInfos segments firstSegmentSize canSwizzle totalHeapSizeIncludingBridges' - instanceVariableNames: 'manager numSegments numSegInfos segments firstSegmentSize canSwizzle sweepIndex preferredPinningSegment totalHeapSizeIncludingBridges' classVariableNames: '' poolDictionaries: '' category: 'VMMaker-SpurMemoryManager'! + !SpurSegmentManager commentStamp: 'eem 4/23/2023 15:19' prior: 0! - !SpurSegmentManager commentStamp: '' prior: 0! Instances of SpurSegmentManager manage oldSpace, which is organized as a sequence of segments. Segments can be obtained from the operating system and returned to the operating system when empty and shrinkage is required. Segments are kept invisible from the SpurMemoryManager by using "bridge" objects, "fake" pinned objects to bridge the gaps between segments. A pinned object header occupies the last 16 bytes of each segment, and the pinned object's size is the distance to the start of the next segment. So when the memory manager enumerates objects it skips over these bridges and memory appears linear. The constraint is that segments obtained from the operating system must be at a higher address than the first segment. The maximum size of large objects, being an overflow slot size, should be big enough to bridge the gaps, because in 32-bits the maximum size is 2^32 slots. In 64-bits the maximum size of large objects is 2^56 slots, or 2^59 bits, which we hope will suffice. When an image is written to a snapshot file the second word of the header of the bridge at the end of each segment is replaced by the size of the following segment, the segments are written to the file, and the second word of each bridge is restored. Hence the length of each segment is derived from the bridge at the end of the preceeding segment. The length of the first segment is stored in the image header as firstSegmentBytes. The start of each segment is also derived from the bridge as a delta from the start of the previous segment. The start of The first segment is stored in the image header as startOfMemory. On load all segments are read into one single segment, eliminating the bridge objects, and computing the swizzle distance for each segment, based on where the segments were in memory when the image file was written, and where the coalesced segment ends up on load. Then the segment is traversed, swizzling pointers by selecting the relevant swizzle for each oop's segment. Instance Variables + manager <SpurMemoryManager> + numSegments <Integer> + numSegInfos <Integer> + segments <Array of SpurSegmentInfo> + firstSegmentSize <Integer> + canSwizzle <Boolean> - manager <SpurMemoryManager> - numSegments <Integer> - numSegInfos <Integer> - segments <Array of SpurSegmentInfo> - firstSegmentSize <Integer> - canSwizzle <Boolean> - sweepIndex <Integer> - preferredPinningSegment <SpurSegmentInfo> totalHeapSizeIncludingBridges <integer> canSwizzle - a flag set and cleared during initialization to validate that swizzling is only performed at the right time firstSegmentSize - the size of the first segment when loading an image manager - the memory manager the receiver manages segments for (simulation only) numSegInfos - the size of the segments array in units of SpurSegmentInfo size numSegments - the number of segments (the number of used entries in segments, <= numSegInfos) - preferredPinningSegment - - the segment in which objects should be copied when pinned, so as to cluster pinned objects in as few segments as possible. As yet unimplemented. - segments - the start addresses, lengths and offsets to adjust oops on image load, for each segment - sweepIndex - - a segment index used to optimize setting the containsPinned flag on segments during freeUnmarkedObjectsAndSortAndCoalesceFreeSpace - totalHeapSizeIncludingBridges - the total size of all segments, used to compute heap usage! Item was changed: ----- Method: SpurSegmentManager>>collapseSegmentsPostSwizzle (in category 'snapshot') ----- collapseSegmentsPostSwizzle "The image has been loaded, old segments reconstructed, and the heap swizzled into a single contiguous segment. Collapse the segments into one." <inline: false> canSwizzle := false. self cCode: [] inSmalltalk: [segments ifNil: [self allocateOrExtendSegmentInfos]]. numSegments := 1. + self computeTotalHeapSizeIncludingBridges. (segments at: 0) segStart: manager oldSpaceStart; + segSize: totalHeapSizeIncludingBridges. - segSize: (totalHeapSizeIncludingBridges := manager endOfMemory - manager oldSpaceStart). manager bootstrapping ifTrue: ["finally plant a bridge at the end of the coalesced segment and cut back the manager's notion of the end of memory to immediately before the bridge." self assert: manager endOfMemory = (segments at: 0) segLimit. manager initSegmentBridgeWithBytes: manager bridgeSize at: manager endOfMemory - manager bridgeSize]. self assert: (manager isSegmentBridge: (self bridgeAt: 0)). self assert: (manager numSlotsOfAny: (self bridgeAt: 0)) = 1! Item was changed: ----- Method: SpurSegmentManager>>initialize (in category 'initialization') ----- initialize + numSegments := numSegInfos := totalHeapSizeIncludingBridges := 0. - numSegments := numSegInfos := sweepIndex := totalHeapSizeIncludingBridges := 0. canSwizzle := false! Item was changed: ----- Method: SpurSegmentManager>>notePinned: (in category 'pinning') ----- notePinned: objOop "Let the segmentManager mark which segments contain pinned objects" + | seg | self assert: (manager isPinned: objOop). + self deny: (manager isSegmentBridge: objOop). + seg := self segmentContainingObj: objOop. + seg containsPinned: true! - (manager isSegmentBridge: objOop) - ifTrue: - [manager setIsMarkedOf: objOop to: true] - ifFalse: - [[self oop: (segments at: sweepIndex) segLimit isLessThan: objOop] whileTrue: - [sweepIndex := sweepIndex + 1]. - (segments at: sweepIndex) containsPinned: true]! Item was changed: ----- Method: SpurSegmentManager>>prepareForGlobalSweep (in category 'pinning') ----- prepareForGlobalSweep "Let the segmentManager mark which segments contain pinned objects via notePinned:. For coallesceFreeChunk:, ensure that the last bridge is marked." - sweepIndex := 0. 0 to: numSegments - 1 do: [:i| (segments at: i) containsPinned: false]. manager setIsMarkedOf: (self bridgeAt: numSegments - 1) to: true! Item was changed: ----- Method: SpurSegmentManager>>segmentContainingObj: (in category 'accessing') ----- segmentContainingObj: objOop "Answer the segment containing an object. This is mostly for assert checking, but variations on the incremental GC may use it in anger. Binary search is (of course) marginally slower than linear search for a single segment (e.g. in a 720k object heap, 67.1ms vs 61.3ms, or 9.5% slower to derive the segment containing every old space entity), but usefully faster for many segments (e.g. 92.7ms vs 116ms, or 20% faster in the same heap extended with enough large arrays to require 11 segments; and this is pessimal; there are fewer objects at high addresses since the large arrays are there)." <export: true> <returnTypeC: #'SpurSegmentInfo *'> | high low mid seg | low := 0. mid := numSegments // 2. high := numSegments - 1. + [seg := self addressOf: (segments at: mid). - [seg := segments at: mid. (self oop: objOop isGreaterThanOrEqualTo: seg segStart) ifTrue: [mid = high ifTrue: [^(self oop: objOop isLessThan: seg segLimit) ifTrue: [seg]] ifFalse: [low := mid. mid := mid + high + 1 // 2]] ifFalse: [high := mid - 1. mid := low + mid // 2]. low <= high] whileTrue. ^nil! Item was changed: ----- Method: SpurSegmentManager>>totalBytesInSegments (in category 'accessing') ----- totalBytesInSegments + "This ``slow'' count is for asserts only." | total | <var: #total type: #usqInt> total := 0. 0 to: numSegments - 1 do: [:i| total := total + (segments at: i) segSize]. - self assert: totalHeapSizeIncludingBridges = total. ^total! Item was changed: ----- Method: SpurSegmentManager>>totalOldSpaceCapacity (in category 'accessing') ----- totalOldSpaceCapacity + self assert: self totalBytesInSegments = totalHeapSizeIncludingBridges. ^totalHeapSizeIncludingBridges - (numSegments * manager bridgeSize)! Item was changed: ----- Method: SpurSegmentManager>>totalOldSpaceSize (in category 'accessing') ----- totalOldSpaceSize + self assert: self totalBytesInSegments = totalHeapSizeIncludingBridges. ^totalHeapSizeIncludingBridges! Item was changed: ----- Method: SpurSweeper>>unmark: (in category 'sweep phase') ----- unmark: objOop self assert: ((manager isMarked: objOop) and: [(manager isFreeObject: objOop) not]). + (manager isSegmentBridge: objOop) + ifTrue: [self assert: (manager isMarked: objOop)] + ifFalse: + [manager setIsMarkedOf: objOop to: false. + (manager isPinned: objOop) ifTrue: + [manager segmentManager notePinned: objOop]]! - (manager isSegmentBridge: objOop) ifFalse: [manager setIsMarkedOf: objOop to: false]. - (manager isPinned: objOop) ifTrue: [manager segmentManager notePinned: objOop]! Item was changed: ----- Method: StackInterpreter>>printFrameOop:index:at: (in category 'debug printing') ----- printFrameOop: name index: idx at: address <var: #name type: #'char *'> <var: #address type: #'char *'> <inline: false> | it | it := stackPages longAt: address. self printFrameAddress: address. + (self pst: '%s%10s%ld: %WP\t') - (self pst: '%s%10s%d: %WP\t') f: transcript printf: { idx > 9 ifTrue: [''] ifFalse: [' ']. name. idx. it asVoidPointer }. self printOopShortInner: it; cr!

1 0

[OpenSmalltalk/opensmalltalk-vm] ca960d: CogVM source as per VMMaker.oscog-eem.3324
by Eliot Miranda 24 Apr '23

24 Apr '23

Branch: refs/heads/virtend Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: ca960dee9c9e30bc7f6ab975d410a40b8f0defa5 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/ca960dee9c9e30bc7f… Author: Eliot Miranda <eliot.miranda(a)gmail.com> Date: 2023-04-24 (Mon, 24 Apr 2023) Changed paths: M src/spur32.cog.lowcode/cogit.h M src/spur32.cog.lowcode/cointerp.c M src/spur32.cog.lowcode/cointerp.h M src/spur32.cog.lowcode/gcc3x-cointerp.c M src/spur32.cog/cogit.h M src/spur32.cog/cointerp.c M src/spur32.cog/cointerp.h M src/spur32.cog/cointerpmt.c M src/spur32.cog/cointerpmt.h M src/spur32.cog/gcc3x-cointerp.c M src/spur32.cog/gcc3x-cointerpmt.c M src/spur32.sista/cogit.h M src/spur32.sista/cointerp.c M src/spur32.sista/cointerp.h M src/spur32.sista/gcc3x-cointerp.c M src/spur32.stack.lowcode/gcc3x-interp.c M src/spur32.stack.lowcode/interp.c M src/spur32.stack/gcc3x-interp.c M src/spur32.stack/interp.c M src/spur32.stack/validImage.c M src/spur64.cog.lowcode/cogit.h M src/spur64.cog.lowcode/cointerp.c M src/spur64.cog.lowcode/cointerp.h M src/spur64.cog.lowcode/gcc3x-cointerp.c M src/spur64.cog/cogit.h M src/spur64.cog/cointerp.c M src/spur64.cog/cointerp.h M src/spur64.cog/cointerpmt.c M src/spur64.cog/cointerpmt.h M src/spur64.cog/gcc3x-cointerp.c M src/spur64.cog/gcc3x-cointerpmt.c M src/spur64.sista/cogit.h M src/spur64.sista/cointerp.c M src/spur64.sista/cointerp.h M src/spur64.sista/gcc3x-cointerp.c M src/spur64.stack.lowcode/gcc3x-interp.c M src/spur64.stack.lowcode/interp.c M src/spur64.stack/gcc3x-interp.c M src/spur64.stack/interp.c M src/spur64.stack/validImage.c M src/v3.cog/cogit.h M src/v3.cog/cointerp.c M src/v3.cog/cointerp.h M src/v3.cog/gcc3x-cointerp.c M src/v3.stack/gcc3x-interp.c M src/v3.stack/interp.c Log Message: ----------- CogVM source as per VMMaker.oscog-eem.3324 Implement SpurSegmentManager>>segmentContainingObj: via binary search for the benefit of the IGC. Access Spur oldSpaceSize via totalHeapSizeIncludingBridges. Have SpurSegmentManager>>totalBytesInSegments do the counting. Use this in the accessors for total memory size, SpurSegmentManager>>totalOldSpaceCapacity/totalOldSpaceSize. Include the receiver when short printing a context, for more information in the context parts of stack dumps (printCallStack et al). Have the ImageSegment storage primitive fail with an error code for bad arguments detected at the outermost level. Include more failure cases in SpurMemoryManager>>#storeImageSegmentInto:outPointers:roots:'s comment. Fix some printf format warnings.

1 0

[OpenSmalltalk/opensmalltalk-vm] e8b78d: CogVM source as per VMMaker.oscog-eem.3324
by Eliot Miranda 24 Apr '23

24 Apr '23

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: e8b78de60f35dee887ed4bd55991b343d1883124 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/e8b78de60f35dee887… Author: Eliot Miranda <eliot.miranda(a)gmail.com> Date: 2023-04-24 (Mon, 24 Apr 2023) Changed paths: M src/spur32.cog.lowcode/cogit.h M src/spur32.cog.lowcode/cointerp.c M src/spur32.cog.lowcode/cointerp.h M src/spur32.cog.lowcode/gcc3x-cointerp.c M src/spur32.cog/cogit.h M src/spur32.cog/cointerp.c M src/spur32.cog/cointerp.h M src/spur32.cog/cointerpmt.c M src/spur32.cog/cointerpmt.h M src/spur32.cog/gcc3x-cointerp.c M src/spur32.cog/gcc3x-cointerpmt.c M src/spur32.sista/cogit.h M src/spur32.sista/cointerp.c M src/spur32.sista/cointerp.h M src/spur32.sista/gcc3x-cointerp.c M src/spur32.stack.lowcode/gcc3x-interp.c M src/spur32.stack.lowcode/interp.c M src/spur32.stack/gcc3x-interp.c M src/spur32.stack/interp.c M src/spur32.stack/validImage.c M src/spur64.cog.lowcode/cogit.h M src/spur64.cog.lowcode/cointerp.c M src/spur64.cog.lowcode/cointerp.h M src/spur64.cog.lowcode/gcc3x-cointerp.c M src/spur64.cog/cogit.h M src/spur64.cog/cointerp.c M src/spur64.cog/cointerp.h M src/spur64.cog/cointerpmt.c M src/spur64.cog/cointerpmt.h M src/spur64.cog/gcc3x-cointerp.c M src/spur64.cog/gcc3x-cointerpmt.c M src/spur64.sista/cogit.h M src/spur64.sista/cointerp.c M src/spur64.sista/cointerp.h M src/spur64.sista/gcc3x-cointerp.c M src/spur64.stack.lowcode/gcc3x-interp.c M src/spur64.stack.lowcode/interp.c M src/spur64.stack/gcc3x-interp.c M src/spur64.stack/interp.c M src/spur64.stack/validImage.c M src/v3.cog/cogit.h M src/v3.cog/cointerp.c M src/v3.cog/cointerp.h M src/v3.cog/gcc3x-cointerp.c M src/v3.stack/gcc3x-interp.c M src/v3.stack/interp.c Log Message: ----------- CogVM source as per VMMaker.oscog-eem.3324 Implement SpurSegmentManager>>segmentContainingObj: via binary search for the benefit of the IGC. Access Spur oldSpaceSize via totalHeapSizeIncludingBridges. Have SpurSegmentManager>>totalBytesInSegments do the counting. Use this in the accessors for total memory size, SpurSegmentManager>>totalOldSpaceCapacity/totalOldSpaceSize. Include the receiver when short printing a context, for more information in the context parts of stack dumps (printCallStack et al). Have the ImageSegment storage primitive fail with an error code for bad arguments detected at the outermost level. Include more failure cases in SpurMemoryManager>>#storeImageSegmentInto:outPointers:roots:'s comment. Fix some printf format warnings.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev April 2023