Vm-dev December 2016

vm-dev@lists.squeakfoundation.org

34 participants
102 discussions

[MoreVMs'17] Call for Contributions: 1st Workshop on Modern Language Runtimes, Ecosystems, and VMs at <Programming> 2017
by Stefan Marr 19 Dec '16

19 Dec '16

============================================================================ Call for Papers: MoreVMs’17 1st Workshop on Modern Language Runtimes, Ecosystems, and VMs Co-located with <Programming> 2017 April, 2017, Brussels, Belgium http://2017.programming-conference.org/track/MoreVMs-2017-papers ============================================================================ The MoreVMs'17 workshop aims bring together programmers from industry and academy to discuss the design, implementation, and usage of modern languages and runtimes. This includes aspects such as reuse of language runtimes, modular implementation, or design and compilation strategies to target existing runtimes. The main goals of the workshop is to bring together both researchers and practitioners and facilitate effective sharing of their respective experiences and ideas on how languages and runtimes are utilized and where they need to improve further. We welcome presentation proposals in the form of extended abstracts discussing experiences, work-in-progress, as well as future visions from the academic as well as industrial perspective. Relevant topics include, but are definitely not limited to, the following: - extensible VM design (compiler- or interpreter-based VMs) - reusable runtime components (e.g. interpreters, garbage collectors, intermediate representations) - static and dynamic compiler techniques - techniques for compilation to high-level languages such as JavaScript - runtimes and mechanisms for interoperability between languages - tooling support (e.g. debugging, profiling, etc.) - programming language development environments and virtual machines - case studies of existing language implementations, virtual machines, and runtime components (e.g. design choices, tradeoffs, etc.) - language implementation challenges and trade-offs (e.g. performance, completeness, etc.) - surveys and applications usage reports to understand runtime usage in the wild - surveys on frameworks and their impact on runtime usage - new research ideas on how we want to build languages in the future ### Workshop Format and Submissions This workshop welcomes the presentation and discussion of new ideas and emerging problems to facilitate interaction among workshop participants and exchange of ideas. We accept presentation proposals in the form of extended abstracts (1-2 pages). Accepted abstracts will be published on the workshop's website before the workshop date. For preparing your abstract, please use the provided author kit: https://github.com/smarr/morevms17-author-kit. It is based on the ACM SIGPLAN Conference Format with 10 point font, and includes a Creative Commons License, which will allow us to publish the abstract on the workshop web site. Please submit abstracts through http://ssw.jku.at/morevms17/ ### Important Dates Abstract submission: 15 February 2017 Author notification: 01 March 2017 Workshop: 3 April 2017 All deadlines are Anywhere on Earth (AoE), i.e. GMT/UTC−12:00 hour ### Program Committee Matthias Grimmer, Oracle Labs Christine H. Flood, Red Hat Tony Hosking, Australian National University Hannes Payer, Google Tiark Rompf, Purdue University Jeremy Singer, University of Glasgow Mark Stoodley, IBM Canada Sam Tobin-Hochstadt, Indiana University ### Workshop Organizers Laurence Tratt, King's College London, United Kingdom Adam Welc, Oracle Labs, United States Stefan Marr, Johannes Kepler University Linz, Austria

1 0

VM Maker: VMMaker.oscog-cb.2043.mcz
by commits＠source.squeak.org 19 Dec '16

19 Dec '16

ClementBera uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-cb.2043.mcz ==================== Summary ==================== Name: VMMaker.oscog-cb.2043 Author: cb Time: 19 December 2016, 11:14:17.217588 am UUID: 62a72d10-be64-47ed-bbe5-eb745f0b7516 Ancestors: VMMaker.oscog-eem.2042 Added identityHash code =============== Diff against VMMaker.oscog-eem.2042 =============== Item was changed: ----- Method: SistaCogit>>genUnaryInlinePrimitive: (in category 'inline primitive generators') ----- genUnaryInlinePrimitive: prim "Unary inline primitives." "SistaV1: 248 11111000 iiiiiiii mjjjjjjj Call Primitive #iiiiiiii + (jjjjjjj * 256) m=1 means inlined primitive, no hard return after execution. See EncoderForSistaV1's class comment and StackInterpreter>>#unaryInlinePrimitive:" | rcvrReg resultReg | rcvrReg := self allocateRegForStackEntryAt: 0. resultReg := self allocateRegNotConflictingWith: (self registerMaskFor: rcvrReg). prim caseOf: { "00 unchecked class" [1] -> "01 unchecked pointer numSlots" [self ssTop popToReg: rcvrReg. self ssPop: 1. objectRepresentation genGetNumSlotsOf: rcvrReg into: resultReg; genConvertIntegerToSmallIntegerInReg: resultReg]. "02 unchecked pointer basicSize" [3] -> "03 unchecked byte numBytes" [self ssTop popToReg: rcvrReg. self ssPop: 1. objectRepresentation genGetNumBytesOf: rcvrReg into: resultReg; genConvertIntegerToSmallIntegerInReg: resultReg]. "04 unchecked short16Type format numShorts" "05 unchecked word32Type format numWords" "06 unchecked doubleWord64Type format numDoubleWords" [11] -> "11 unchecked fixed pointer basicNew" [self ssTop type ~= SSConstant ifTrue: [^EncounteredUnknownBytecode]. (objectRepresentation genGetInstanceOf: self ssTop constant into: resultReg initializingIf: self extBSpecifiesInitializeInstance) ~= 0 ifTrue: [^ShouldNotJIT]. "e.g. bad class" + self ssPop: 1] . + [20] -> "20 identityHash" + [self ssTop popToReg: rcvrReg. + objectRepresentation genGetHashFieldNonImmOf: rcvrReg asSmallIntegerInto: resultReg. self ssPop: 1] + "21 identityHash (SmallInteger)" + "22 identityHash (Character)" + "23 identityHash (SmallFloat64)" + "24 identityHash (Behavior)" } otherwise: [^EncounteredUnknownBytecode]. extB := 0. self ssPushRegister: resultReg. ^0! Item was changed: ----- Method: StackInterpreter>>unaryInlinePrimitive: (in category 'miscellaneous bytecodes') ----- unaryInlinePrimitive: primIndex "SistaV1: 248 11111000 iiiiiiii mjjjjjjj Call Primitive #iiiiiiii + (jjjjjjj * 256) m=1 means inlined primitive, no hard return after execution." <option: #SistaVM> | result | primIndex caseOf: { "1000 unchecked class" [0] -> [result := objectMemory fetchClassOf: self internalStackTop. self internalStackTopPut: result]. "1001 unchecked pointer numSlots" [1] -> [result := objectMemory numSlotsOf: self internalStackTop. self internalStackTopPut: (objectMemory integerObjectOf: result)]. "1002 unchecked pointer basicSize" [2] -> [result := (objectMemory numSlotsOf: self internalStackTop) - (objectMemory fixedFieldsOfClass: (objectMemory fetchClassOfNonImm: self internalStackTop)). self internalStackTopPut: (objectMemory integerObjectOf: result)]. "1003 unchecked byte8Type format numBytes (includes CompiledMethod)" [3] -> [result := objectMemory numBytesOf: self internalStackTop. self internalStackTopPut: (objectMemory integerObjectOf: result)]. "1004 unchecked short16Type format numShorts" [4] -> [result := objectMemory num16BitUnitsOf: self internalStackTop. self internalStackTopPut: (objectMemory integerObjectOf: result)]. "1005 unchecked word32Type format numWords" [5] -> [result := objectMemory num32BitUnitsOf: self internalStackTop. self internalStackTopPut: (objectMemory integerObjectOf: result)]. "1006 unchecked doubleWord64Type format numDoubleWords" [6] -> [result := objectMemory num64BitUnitsOf: self internalStackTop. self internalStackTopPut: (objectMemory integerObjectOf: result)]. "1011 unchecked fixed pointer basicNew" [11] -> [| classObj numSlots | classObj := self internalStackTop. numSlots := objectMemory instanceSizeOf: classObj. result := objectMemory eeInstantiateSmallClass: classObj numSlots: numSlots. (extB noMask: 1) ifTrue: [0 to: numSlots - 1 do: [:i| objectMemory storePointerUnchecked: i ofObject: result withValue: objectMemory nilObject]]. extB := 0. + self internalStackTopPut: result]. + "1020 identityHash" + [20] -> [result := objectMemory hashBitsOf: self internalStackTop. + self internalStackTopPut: (objectMemory integerObjectOf: result)] + "1021 identityHash (SmallInteger)" + "1022 identityHash (Character)" + "1023 identityHash (SmallFloat64)" + "1024 identityHash (Behavior)" + } - self internalStackTopPut: result] } otherwise: [localIP := localIP - 3. self respondToUnknownBytecode]!

1 0

VM Maker: BytecodeSets.spur-cb.60.mcz
by commits＠source.squeak.org 19 Dec '16

19 Dec '16

ClementBera uploaded a new version of BytecodeSets to project VM Maker: http://source.squeak.org/VMMaker/BytecodeSets.spur-cb.60.mcz ==================== Summary ==================== Name: BytecodeSets.spur-cb.60 Author: cb Time: 19 December 2016, 11:11:28.315661 am UUID: 80d34098-984f-4688-acae-978401da8a95 Ancestors: BytecodeSets.spur-cb.59 Fixed identityHash spec =============== Diff against BytecodeSets.spur-cb.59 =============== Item was changed: BytecodeEncoder subclass: #EncoderForSistaV1 (excessive size, no diff calculated)

1 0

VM Maker: BytecodeSets.spur-cb.59.mcz
by commits＠source.squeak.org 19 Dec '16

19 Dec '16

ClementBera uploaded a new version of BytecodeSets to project VM Maker: http://source.squeak.org/VMMaker/BytecodeSets.spur-cb.59.mcz ==================== Summary ==================== Name: BytecodeSets.spur-cb.59 Author: cb Time: 19 December 2016, 10:54:41.759471 am UUID: 17db338f-e79a-48df-ab56-9fc40ce845c8 Ancestors: BytecodeSets.spur-eem.58 Added spec for idenityHash =============== Diff against BytecodeSets.spur-eem.58 =============== Item was changed: BytecodeEncoder subclass: #EncoderForSistaV1 (excessive size, no diff calculated)

1 0

VM Maker: VMMaker.oscog-eem.2042.mcz
by commits＠source.squeak.org 17 Dec '16

17 Dec '16

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2042.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.2042 Author: eem Time: 17 December 2016, 12:35:26.951861 pm UUID: 6c0ea7a2-25c3-4375-9ca0-f865eb2197ed Ancestors: VMMaker.oscog-eem.2041 Refactor globalGarbageCollect to do the scavenge outside of the compactor. Provide the mechanism to specify a different compactor. =============== Diff against VMMaker.oscog-eem.2041 =============== Item was changed: ----- Method: SpurMemoryManager class>>compactorClass (in category 'accessing class hierarchy') ----- compactorClass "Answer the compaction algorithm to use." + ^Smalltalk classNamed: (initializationOptions at: #compactorClass ifAbsent: [#SpurPigCompactor])! - ^SpurPigCompactor! Item was changed: ----- Method: SpurMemoryManager>>globalGarbageCollect (in category 'gc - global') ----- globalGarbageCollect <inline: true> "inline into fullGC" self assert: self validObjStacks. self assert: (self isEmptyObjStack: markStack). self assert: (self isEmptyObjStack: weaklingStack). + "Mark objects /before/ scavenging, to empty the rememberedTable of unmarked roots." self markObjects: true. + + scavenger forgetUnmarkedRememberedObjects. + self doScavenge: MarkOnTenure. + segmentManager prepareForGlobalSweep. "for notePinned:" + compactor freeUnmarkedObjectsAndPrepareFreeSpace. "Mid-way the leak check must be more lenient. Unmarked classes will have been expunged from the table, but unmarked instances will not yet have been reclaimed." self runLeakCheckerFor: GCModeFull excludeUnmarkedNewSpaceObjs: true classIndicesShouldBeValid: true. compactor compact. self setHeapSizeAtPreviousGC. self assert: self validObjStacks. self assert: (self isEmptyObjStack: markStack). self assert: (self isEmptyObjStack: weaklingStack). self assert: self allObjectsUnmarked. self runLeakCheckerFor: GCModeFull! Item was changed: ----- Method: SpurPigCompactor>>freeUnmarkedObjectsAndSortAndCoalesceFreeSpaceForPigCompact (in category 'compaction') ----- freeUnmarkedObjectsAndSortAndCoalesceFreeSpaceForPigCompact "Sweep all of old space, freeing unmarked objects, coalescing free chunks, and sorting free space. Doubly-link the free chunks in address order through the freeChunkNextIndex field using the xor trick to use only one field, see e.g. The Art of Computer Programming, Vol 1, D.E. Knuth, 3rd Ed, Sec 2.2.4 `Circular Lists', exercise. 18 http://en.wikipedia.org/wiki/XOR_linked_list. Record the lowest free object in firstFreeChunk and the highest in lastFreeChunk. Let the segmentManager mark which segments contain pinned objects via notePinned:." | prevPrevFree prevFree | <inline: #never> "for profiling" manager checkFreeSpace: GCModeFull. - scavenger forgetUnmarkedRememberedObjects. - manager doScavenge: MarkOnTenure. - manager segmentManager prepareForGlobalSweep."for notePinned:" "throw away the list heads, including the tree." manager resetFreeListHeads. firstFreeChunk := prevPrevFree := prevFree := 0. manager allOldSpaceEntitiesForCoalescingFrom: manager firstObject do: [:o| self assert: (firstFreeChunk = 0 or: [manager isFreeObject: firstFreeChunk]). (manager isMarked: o) ifTrue: "forwarders should have been followed in markAndTrace:" [self assert: (manager isForwarded: o) not. manager setIsMarkedOf: o to: false. "this will unmark bridges. undo the damage in notePinned:" (manager isPinned: o) ifTrue: [manager segmentManager notePinned: o]] ifFalse: "unmarked; two cases, an unreachable object or a free chunk." [| here | self assert: (manager isRemembered: o) not. "scavenger should have cleared this above" here := manager coallesceFreeChunk: o. manager setObjectFree: here. self inSortedFreeListLink: prevFree to: here given: prevPrevFree. prevPrevFree := prevFree. prevFree := here]]. prevFree ~= firstFreeChunk ifTrue: [manager storePointer: manager freeChunkNextIndex ofFreeChunk: prevFree withValue: prevPrevFree]. lastFreeChunk := prevFree. self inSortedFreeListLink: lastFreeChunk to: 0 given: prevPrevFree. self assert: self checkTraversableSortedFreeList!

1 0

Re: [Vm-dev] ptsd rage
by Ben Coman 17 Dec '16

17 Dec '16

On Sat, Dec 17, 2016 at 9:57 AM, Robert Withers <robert.w.withers(a)gmail.com> wrote: > > I need to apologize for all my dread behavior. Here is the reason, and I am > on service-connected disability. Current diagnosis is Complex PTSD. Sorry > about all my sturm und drang. > > http://www.ptsd.va.gov/public/problems/anger-and-trauma.asp > Sorry to hear you are suffering with that. Hopefully a useful treatment is not too far away... F.D.A. Agrees to New Trials for Ecstasy as Relief for PTSD Patients http://www.nytimes.com/2016/11/29/us/ptsd-mdma-ecstasy.html?_r=0 cheers -ben

1 0

[OpenSmalltalk/opensmalltalk-vm] 28293a: CogVM source as per VMMaker.oscog-eem.2041
by GitHub 17 Dec '16

17 Dec '16

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: 28293aa6b85eb1903ad81d71285f5c3dcd6d8eb5 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/28293aa6b85eb1903a… Author: Eliot Miranda <eliot.miranda(a)gmail.com> Date: 2016-12-16 (Fri, 16 Dec 2016) Changed paths: M image/BuildSqueakSpurTrunkVMMakerImage.st M nsspur64src/vm/cointerp.c M nsspur64src/vm/cointerp.h M nsspur64src/vm/gcc3x-cointerp.c M nsspursrc/vm/cointerp.c M nsspursrc/vm/cointerp.h M nsspursrc/vm/gcc3x-cointerp.c M nsspurstack64src/vm/gcc3x-interp.c M nsspurstack64src/vm/interp.c M nsspurstacksrc/vm/gcc3x-interp.c M nsspurstacksrc/vm/interp.c M spur64src/vm/cointerp.c M spur64src/vm/cointerp.h M spur64src/vm/gcc3x-cointerp.c M spursista64src/vm/cointerp.c M spursista64src/vm/cointerp.h M spursista64src/vm/gcc3x-cointerp.c M spursistasrc/vm/cointerp.c M spursistasrc/vm/cointerp.h M spursistasrc/vm/gcc3x-cointerp.c M spursrc/vm/cointerp.c M spursrc/vm/cointerp.h M spursrc/vm/gcc3x-cointerp.c M spurstack64src/vm/gcc3x-interp.c M spurstack64src/vm/interp.c M spurstacksrc/vm/gcc3x-interp.c M spurstacksrc/vm/interp.c Log Message: ----------- CogVM source as per VMMaker.oscog-eem.2041 Extract the pig compaction algorithm into its own class, allowing experimentation and hopefully prompt replacement. Make sure that the RBSqueakPlatform is included when building a VMMaker image.

1 0

VM Maker: VMMaker.oscog-eem.2041.mcz
by commits＠source.squeak.org 17 Dec '16

17 Dec '16

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2041.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.2041 Author: eem Time: 16 December 2016, 5:00:36.536969 pm UUID: 89811978-7175-44e6-86c2-bdc85c2e9ecf Ancestors: VMMaker.oscog-eem.2040 Extract the pig compaction algorithm into its own class, allowing experimentation and hopefully prompt replacement. Nuke a few unused methods in SpurMemoryManager. =============== Diff against VMMaker.oscog-eem.2040 =============== Item was removed: - ----- Method: Spur32BitMMLECoSimulator>>classTableFirstPage (in category 'debug support') ----- - classTableFirstPage - ^classTableFirstPage! Item was changed: ----- Method: Spur32BitMMLECoSimulator>>coInterpreter:cogit: (in category 'initialization') ----- coInterpreter: aCoInterpreter cogit: aCogit coInterpreter := aCoInterpreter. cogit := aCogit. + scavenger coInterpreter: aCoInterpreter. + compactor coInterpreter: aCoInterpreter! - scavenger coInterpreter: aCoInterpreter! Item was removed: - ----- Method: Spur64BitMMLECoSimulator>>classTableFirstPage (in category 'debug support') ----- - classTableFirstPage - ^classTableFirstPage! Item was changed: ----- Method: Spur64BitMMLECoSimulator>>coInterpreter:cogit: (in category 'initialization') ----- coInterpreter: aCoInterpreter cogit: aCogit coInterpreter := aCoInterpreter. cogit := aCogit. + scavenger coInterpreter: aCoInterpreter. + compactor coInterpreter: aCoInterpreter! - scavenger coInterpreter: aCoInterpreter! Item was changed: CogClass subclass: #SpurMemoryManager (excessive size, no diff calculated) Item was changed: ----- Method: SpurMemoryManager class>>ancilliaryClasses: (in category 'translation') ----- ancilliaryClasses: options + ^{ SpurGenerationScavenger. SpurSegmentManager. SpurSegmentInfo. self compactorClass }, - ^{ SpurGenerationScavenger. SpurSegmentManager. SpurSegmentInfo }, SpurNewSpaceSpace withAllSubclasses! Item was added: + ----- Method: SpurMemoryManager class>>compactorClass (in category 'accessing class hierarchy') ----- + compactorClass + "Answer the compaction algorithm to use." + ^SpurPigCompactor! Item was changed: ----- Method: SpurMemoryManager class>>initialize (in category 'class initialization') ----- initialize "SpurMemoryManager initialize" BitsPerByte := 8. "Initialize at least the become constants for the Spur bootstrap where the old ObjectMemory simulator is used before a Spur simulator is created.." self initializeSpurObjectRepresentationConstants. - "Pig compact can be repeated to compact better. Experience shows that 3 times - compacts very well, desirable for snapshots. But this is overkill for normal GCs." - CompactionPassesForGC := 2. - CompactionPassesForSnapshot := 3. - "An obj stack is a stack of objects stored in a hidden root slot, such as the markStack or the ephemeronQueue. It is a linked list of segments, with the hot end at the head of the list. It is a word object. The stack pointer is in ObjStackTopx and 0 means empty. The list goes through ObjStackNextx. We don't want to shrink objStacks, since they're used in GC and it's good to keep their memory around. So unused pages created by popping emptied pages are kept on the ObjStackFreex list. ObjStackNextx must be the last field for swizzleObjStackAt:." ObjStackPageSlots := 4092. "+ double header = 16k bytes per page in 32-bits" ObjStackTopx := 0. ObjStackMyx := 1. ObjStackFreex := 2. ObjStackNextx := 3. ObjStackFixedSlots := 4. ObjStackLimit := ObjStackPageSlots - ObjStackFixedSlots. "The hiddenHootsObject contains the classTable pages and up to 8 additional objects. Currently we use four; the three objStacks, the mark stack, the weaklings and the mourn queue, and the rememberedSet." MarkStackRootIndex := self basicNew classTableRootSlots. WeaklingStackRootIndex := MarkStackRootIndex + 1. MournQueueRootIndex := MarkStackRootIndex + 2. RememberedSetRootIndex := MarkStackRootIndex + 3. MarkObjectsForEnumerationPrimitives := false. "The remap buffer support is for compatibility; Spur doesn't GC during allocation. Eventually this should die." RemapBufferSize := 25. "Extra roots are for plugin support." ExtraRootsSize := 2048 "max. # of external roots"! Item was changed: ----- Method: SpurMemoryManager class>>isNonArgumentImplicitReceiverVariableName: (in category 'translation') ----- isNonArgumentImplicitReceiverVariableName: aString + ^#('self' 'coInterpreter' 'manager' 'scavenger' 'segmentManager' 'compactor' 'heapMap') includes: aString! - ^#('self' 'coInterpreter' 'manager' 'scavenger' 'segmentManager' 'heapMap') includes: aString! Item was removed: - ----- Method: SpurMemoryManager>>abstractPigCompaction (in category 'compaction - analysis') ----- - abstractPigCompaction - "This method answers a rough estimate of compactibility using a pig (a large free chunk)." - <doNotGenerate> - | pig pork moved unmoved nmoved nunmoved | - pig := self findAPig. - pork := self bytesInObject: pig. - moved := unmoved := nmoved := nunmoved := 0. - self allOldSpaceObjectsFrom: pig do: - [:o| | bytes | - bytes := self bytesInObject: o. - bytes <= pork - ifTrue: - [moved := moved + bytes. - nmoved := nmoved + 1. - pork := pork - bytes] - ifFalse: - [unmoved := unmoved + bytes. - nunmoved := nunmoved + 1]]. - ^{ self bytesInObject: pig. pork. moved. nmoved. unmoved. nunmoved }! Item was changed: ----- Method: SpurMemoryManager>>allocateMemoryOfSize:newSpaceSize:stackSize:codeSize: (in category 'spur bootstrap') ----- allocateMemoryOfSize: memoryBytes newSpaceSize: newSpaceBytes stackSize: stackBytes codeSize: codeBytes "Intialize the receiver for bootsraping an image. Set up a large oldSpace and an empty newSpace and set-up freeStart and scavengeThreshold to allocate in oldSpace. Later on (in initializePostBootstrap) freeStart and scavengeThreshold will be set to sane values." <doNotGenerate> self assert: (memoryBytes \\ self allocationUnit = 0 and: [newSpaceBytes \\ self allocationUnit = 0 and: [codeBytes \\ self allocationUnit = 0]]). self allocateMemoryOfSize: memoryBytes + newSpaceBytes + codeBytes + stackBytes. newSpaceStart := codeBytes + stackBytes. endOfMemory := freeOldSpaceStart := memoryBytes + newSpaceBytes + codeBytes + stackBytes. "leave newSpace empty for the bootstrap" freeStart := newSpaceBytes + newSpaceStart. oldSpaceStart := newSpaceLimit := newSpaceBytes + newSpaceStart. scavengeThreshold := memory size * memory bytesPerElement. "i.e. /don't/ scavenge." scavenger := SpurGenerationScavengerSimulator new. scavenger manager: self. scavenger newSpaceStart: newSpaceStart newSpaceBytes: newSpaceBytes + survivorBytes: newSpaceBytes // self scavengerDenominator. + compactor := self class compactorClass new manager: self; yourself.! - survivorBytes: newSpaceBytes // self scavengerDenominator! Item was removed: - ----- Method: SpurMemoryManager>>checkNoForwardersBelowFirstFreeChunk (in category 'gc - global') ----- - checkNoForwardersBelowFirstFreeChunk - self allOldSpaceEntitiesDo: - [:o| - (self oop: o isGreaterThanOrEqualTo: firstFreeChunk) ifTrue: - [^true]. - (self asserta: (self isForwarded: o) not) ifFalse: - [^false]]. - ^true! Item was removed: - ----- Method: SpurMemoryManager>>checkTraversableSortedFreeList (in category 'simulation only') ----- - checkTraversableSortedFreeList - | prevFree prevPrevFree freeChunk | - <api> - <inline: false> - prevFree := prevPrevFree := 0. - firstFreeChunk = 0 ifTrue: - [^lastFreeChunk = 0]. - freeChunk := firstFreeChunk. - self allOldSpaceEntitiesDo: - [:o| | objOop next limit | - (self isFreeObject: o) ifTrue: - [self assert: o = freeChunk. - next := self nextInSortedFreeListLink: freeChunk given: prevFree. - limit := next = 0 ifTrue: [endOfMemory] ifFalse: [next]. - "coInterpreter transcript cr; print: freeChunk; tab; print: o; tab; print: prevFree; nextPutAll: '<->'; print: next; flush." - objOop := freeChunk. - [self oop: (objOop := self objectAfter: objOop) isLessThan: limit] whileTrue: - [self assert: (self isFreeObject: objOop) not]. - prevPrevFree := prevFree. - prevFree := freeChunk. - freeChunk := next]]. - self assert: prevFree = lastFreeChunk. - self assert: (self nextInSortedFreeListLink: lastFreeChunk given: 0) = prevPrevFree. - self assert: freeChunk = 0. - ^true! Item was added: + ----- Method: SpurMemoryManager>>classTableFirstPage (in category 'accessing') ----- + classTableFirstPage + <cmacro> + ^classTableFirstPage! Item was changed: ----- Method: SpurMemoryManager>>coInterpreter: (in category 'simulation') ----- coInterpreter: aCoInterpreter <doNotGenerate> coInterpreter := aCoInterpreter. scavenger ifNotNil: + [scavenger coInterpreter: aCoInterpreter]. + compactor ifNotNil: + [compactor coInterpreter: aCoInterpreter]! - [scavenger coInterpreter: aCoInterpreter]! Item was removed: - ----- Method: SpurMemoryManager>>compact (in category 'compaction') ----- - compact - "We'd like to use exact fit followed by best or first fit, but it doesn't work - well enough in practice. So use pig compact. Fill large free objects starting - from low memory with objects taken from the end of memory." - <inline: #never> "for profiling" - statCompactPassCount := statCompactPassCount + 1. - self assert: (firstFreeChunk = 0 or: [self isFreeObject: firstFreeChunk]). - 1 to: numCompactionPasses do: - [:i| - self pigCompact. - self eliminateAndFreeForwardersForPigCompact]. - - "The free lists are zeroed in freeUnmarkedObjectsAndSortAndCoalesceFreeSpaceForPigCompact. - They should still be zero here" - self assert: self freeListHeadsEmpty. - self rebuildFreeListsForPigCompact! Item was removed: - ----- Method: SpurMemoryManager>>eliminateAndFreeForwardersForPigCompact (in category 'gc - global') ----- - eliminateAndFreeForwardersForPigCompact - "As the final phase of global garbage collect, sweep the heap to follow - forwarders, then free forwarders, coalescing with free space as we go." - <inline: false> - | lowestForwarder | - <var: #lowestForwarder type: #usqInt> - self assert: (self isForwarded: nilObj) not. - self assert: (self isForwarded: falseObj) not. - self assert: (self isForwarded: trueObj) not. - self assert: (self isForwarded: self freeListsObj) not. - self assert: (self isForwarded: hiddenRootsObj) not. - self assert: (self isForwarded: classTableFirstPage) not. - self followSpecialObjectsOop. - self followForwardedObjStacks. - coInterpreter mapInterpreterOops. - scavenger followRememberedForwardersAndForgetFreeObjectsForPigCompact. - self unmarkSurvivingObjectsForPigCompact. - lowestForwarder := self sweepToFollowForwardersForPigCompact. - self sweepToCoallesceFreeSpaceForPigCompactFrom: lowestForwarder. - self assert: self numberOfForwarders = 0! Item was removed: - ----- Method: SpurMemoryManager>>findAPig (in category 'compaction - analysis') ----- - findAPig - "Answer a large low free chuink." - <doNotGenerate> - | pig | - self allObjectsInFreeTreeDo: - [:f| - (self bytesInObject: f) >= 1000000 ifTrue: - [(pig isNil or: [pig > f]) ifTrue: - [pig := f]]]. - ^pig! Item was removed: - ----- Method: SpurMemoryManager>>freeUnmarkedObjectsAndSortAndCoalesceFreeSpaceForPigCompact (in category 'gc - global') ----- - freeUnmarkedObjectsAndSortAndCoalesceFreeSpaceForPigCompact - "Sweep all of old space, freeing unmarked objects, coalescing free chunks, and sorting free space. - - Doubly-link the free chunks in address order through the freeChunkNextIndex field using the - xor trick to use only one field, see e.g. - The Art of Computer Programming, Vol 1, D.E. Knuth, 3rd Ed, Sec 2.2.4 `Circular Lists', exercise. 18 - http://en.wikipedia.org/wiki/XOR_linked_list. - Record the lowest free object in firstFreeChunk and the highest in lastFreeChunk. - - Let the segmentManager mark which segments contain pinned objects via notePinned:." - - | prevPrevFree prevFree | - <inline: #never> "for profiling" - self checkFreeSpace: GCModeFull. - scavenger forgetUnmarkedRememberedObjects. - self doScavenge: MarkOnTenure. - segmentManager prepareForGlobalSweep."for notePinned:" - "throw away the list heads, including the tree." - self resetFreeListHeads. - firstFreeChunk := prevPrevFree := prevFree := 0. - self allOldSpaceEntitiesForCoalescingFrom: self firstObject do: - [:o| - self assert: (firstFreeChunk = 0 or: [self isFreeObject: firstFreeChunk]). - (self isMarked: o) - ifTrue: "forwarders should have been followed in markAndTrace:" - [self assert: (self isForwarded: o) not. - self setIsMarkedOf: o to: false. "this will unmark bridges. undo the damage in notePinned:" - (self isPinned: o) ifTrue: - [segmentManager notePinned: o]] - ifFalse: "unmarked; two cases, an unreachable object or a free chunk." - [| here | - self assert: (self isRemembered: o) not. "scavenger should have cleared this above" - here := self coallesceFreeChunk: o. - self setObjectFree: here. - self inSortedFreeListLink: prevFree to: here given: prevPrevFree. - prevPrevFree := prevFree. - prevFree := here]]. - prevFree ~= firstFreeChunk ifTrue: - [self storePointer: self freeChunkNextIndex - ofFreeChunk: prevFree - withValue: prevPrevFree]. - lastFreeChunk := prevFree. - self inSortedFreeListLink: lastFreeChunk to: 0 given: prevPrevFree. - self assert: self checkTraversableSortedFreeList! Item was changed: ----- Method: SpurMemoryManager>>garbageCollectForSnapshot (in category 'snapshot') ----- garbageCollectForSnapshot self flushNewSpace. "There is no place to put newSpace in the snapshot file." self flag: 'If we wanted to shrink the rememberedSet prior to snapshot this is the place to do it.'. + compactor biasForSnapshot. - numCompactionPasses := CompactionPassesForSnapshot. self fullGC. + compactor biasForGC. - numCompactionPasses := CompactionPassesForGC. segmentManager prepareForSnapshot. self checkFreeSpace: GCModeFull! Item was changed: ----- Method: SpurMemoryManager>>globalGarbageCollect (in category 'gc - global') ----- globalGarbageCollect <inline: true> "inline into fullGC" self assert: self validObjStacks. self assert: (self isEmptyObjStack: markStack). self assert: (self isEmptyObjStack: weaklingStack). self markObjects: true. + compactor freeUnmarkedObjectsAndPrepareFreeSpace. - self freeUnmarkedObjectsAndSortAndCoalesceFreeSpaceForPigCompact. "Mid-way the leak check must be more lenient. Unmarked classes will have been expunged from the table, but unmarked instances will not yet have been reclaimed." self runLeakCheckerFor: GCModeFull excludeUnmarkedNewSpaceObjs: true classIndicesShouldBeValid: true. + compactor compact. - self compact. self setHeapSizeAtPreviousGC. self assert: self validObjStacks. self assert: (self isEmptyObjStack: markStack). self assert: (self isEmptyObjStack: weaklingStack). self assert: self allObjectsUnmarked. self runLeakCheckerFor: GCModeFull! Item was added: + ----- Method: SpurMemoryManager>>hiddenRootsObject (in category 'accessing') ----- + hiddenRootsObject + ^hiddenRootsObj! Item was removed: - ----- Method: SpurMemoryManager>>inSortedFreeListLink:to:given: (in category 'compaction') ----- - inSortedFreeListLink: freeChunk to: nextFree given: prevFree - "Doubly-link the free chunk in address order through the freeChunkNextIndex field using the - xor trick to use only one field, see e.g. - The Art of Computer Programming, Vol 1, D.E. Knuth, 3rd Ed, Sec 2.2.4 `Circular Lists', exercise. 18 - http://en.wikipedia.org/wiki/XOR_linked_list." - freeChunk = 0 - ifTrue: - [firstFreeChunk := nextFree] - ifFalse: - [self storePointer: self freeChunkNextIndex - ofFreeChunk: freeChunk - withUncheckedValue: (prevFree bitXor: nextFree)]! Item was changed: ----- Method: SpurMemoryManager>>initialize (in category 'initialization') ----- initialize "We can put all initializations that set something to 0 or to false here. In C all global variables are initialized to 0, and 0 is false." remapBuffer := Array new: RemapBufferSize. remapBufferCount := extraRootCount := 0. "see below" freeListsMask := totalFreeOldSpace := lowSpaceThreshold := 0. checkForLeaks := 0. needGCFlag := signalLowSpace := scavengeInProgress := marking := false. becomeEffectsFlags := 0. statScavenges := statIncrGCs := statFullGCs := 0. statScavengeGCUsecs := statIncrGCUsecs := statFullGCUsecs := statGCEndUsecs := 0. statSGCDeltaUsecs := statIGCDeltaUsecs := statFGCDeltaUsecs := 0. statGrowMemory := statShrinkMemory := statRootTableCount := 0. statRootTableOverflows := statMarkCount := statCompactPassCount := statCoalesces := 0. "We can initialize things that are allocated but are lazily initialized." unscannedEphemerons := SpurContiguousObjStack new. "we can initialize things that are virtual in C." scavenger := SpurGenerationScavengerSimulator new manager: self; yourself. segmentManager := SpurSegmentManager new manager: self; yourself. + compactor := self class compactorClass new manager: self; yourself. "We can also initialize here anything that is only for simulation." heapMap := CogCheck32BitHeapMap new. "N.B. We *don't* initialize extraRoots because we don't simulate it." maxOldSpaceSize := self class initializationOptions ifNotNil: [:initOpts| initOpts at: #maxOldSpaceSize ifAbsent: [0]] ifNil: [0]! Item was changed: ----- Method: SpurMemoryManager>>initializeObjectMemory: (in category 'initialization') ----- initializeObjectMemory: bytesToShift "Initialize object memory variables at startup time. Assume endOfMemory at al are initialised by the image-reading code via setHeapBase:memoryLimit:endOfMemory:. endOfMemory is assumed to point to the end of the last object in the image. Assume: image reader also initializes the following variables: specialObjectsOop lastHash" <inline: false> | freeListObj | "Catch mis-initializations leading to bad translations to C" self assert: self baseHeaderSize = self baseHeaderSize. self assert: (self maxSlotsForAlloc * self wordSize) asInteger > 0. self bootstrapping ifFalse: [self initSegmentBridgeWithBytes: self bridgeSize at: endOfMemory - self bridgeSize]. segmentManager adjustSegmentSwizzlesBy: bytesToShift. "image may be at a different address; adjust oops for new location" self adjustAllOopsBy: bytesToShift. specialObjectsOop := segmentManager swizzleObj: specialObjectsOop. "heavily used special objects" nilObj := self splObj: NilObject. falseObj := self splObj: FalseObject. trueObj := self splObj: TrueObject. "In Cog we insist that nil, true & false are next to each other (Cogit generates tighter conditional branch code as a result). In addition, Spur places the free lists and class table root page immediately following them." self assert: nilObj = oldSpaceStart. self assert: falseObj = (self objectAfter: nilObj). self assert: trueObj = (self objectAfter: falseObj). freeListObj := self objectAfter: trueObj. self setHiddenRootsObj: (self objectAfter: freeListObj). markStack := self swizzleObjStackAt: MarkStackRootIndex. weaklingStack := self swizzleObjStackAt: WeaklingStackRootIndex. mournQueue := self swizzleObjStackAt: MournQueueRootIndex. self assert: self validObjStacks. self assert: (self isEmptyObjStack: markStack). self assert: (self isEmptyObjStack: weaklingStack). self initializeFreeSpacePostLoad: freeListObj. segmentManager collapseSegmentsPostSwizzle. self computeFreeSpacePostSwizzle. self initializeOldSpaceFirstFree: freeOldSpaceStart. "initializes endOfMemory, freeStart, free space" self initializeNewSpaceVariables. scavenger initializeRememberedSet. segmentManager checkSegments. + compactor biasForGC. - numCompactionPasses := CompactionPassesForGC. - "These defaults should depend on machine size; e.g. too small on a powerful laptop, too big on a Pi." growHeadroom := 16*1024*1024. "headroom when growing" shrinkThreshold := 32*1024*1024. "free space before shrinking" self setHeapSizeAtPreviousGC. heapGrowthToSizeGCRatio := 0.333333. "By default GC after scavenge if heap has grown by a third since the last GC"! Item was removed: - ----- Method: SpurMemoryManager>>moveARunOfObjectsStartingAt:upTo: (in category 'compaction') ----- - moveARunOfObjectsStartingAt: startAddress upTo: limit - "Move the sequence of movable objects starting at startAddress. Answer the start - of the next sequence of movable objects after a possible run of unmovable objects, - or the limit, if there are no more movable objects, or 0 if no more compaction can be - done. Compaction is done when the search through the freeList has reached the - address from which objects are being moved from. - - There are two broad cases to be dealt with here. One is a run of smallish objects - that can easily be moved into free chunks. The other is a large object that is unlikely - to fit in the typical free chunk. This second pig needs careful handling; it needs to be - moved to the lowest place it will fit and not cause the scan to skip lots of smaller - free chunks looking in vain for somewhere to put it." - <var: #startAddress type: #usqInt> - <var: #limit type: #usqInt> - <inline: false> - | here hereObj hereObjHeader prevPrevFreeChunk prevFreeChunk thisFreeChunk maxFreeChunk | - <var: #here type: #usqInt> - <var: #there type: #usqInt> - <var: #nextFree type: #usqInt> - <var: #endOfFree type: #usqInt> - <var: #destination type: #usqInt> - <var: #maxFreeChunk type: #usqInt> - here := startAddress. - hereObj := self objectStartingAt: startAddress. - hereObjHeader := self atLeastClassIndexHalfHeader: hereObj. - prevPrevFreeChunk := prevFreeChunk := 0. - thisFreeChunk := maxFreeChunk := firstFreeChunk. - [thisFreeChunk ~= 0] whileTrue: - [| freeBytes endOfFree nextFree destination there moved | - - "skip any initial immobile objects" - [(self isMobileObjectHeader: hereObjHeader)] whileFalse: - [here := self addressAfter: hereObj. - here >= limit ifTrue: - [^maxFreeChunk >= startAddress ifTrue: [0] ifFalse: [limit]]. - hereObj := self objectStartingAt: here. - hereObjHeader := self atLeastClassIndexHalfHeader: hereObj]. - - "grab a free chunk, and the following one, because we want to overwrite this one." - self assert: ((self isFreeObject: firstFreeChunk) and: [self isFreeObject: thisFreeChunk]). - freeBytes := self bytesInObject: thisFreeChunk. - nextFree := self nextInSortedFreeListLink: thisFreeChunk given: prevFreeChunk. - destination := self startOfObject: thisFreeChunk. - endOfFree := destination + freeBytes. - moved := false. - maxFreeChunk := maxFreeChunk max: nextFree. - self assert: (nextFree = 0 or: [self isFreeObject: nextFree]). - - "move as many objects as will fit in freeBytes..." - [there := self addressAfter: hereObj. - "N.B. *must* add allocationUnit, not subtract, to avoid unsigned arithmetic issues when freeBytes = 0" - (self isMobileObjectHeader: hereObjHeader) - and: [freeBytes > (there - here + self allocationUnit) - or: [freeBytes = (there - here)]]] whileTrue: - [moved := true. - self mem: destination asVoidPointer cp: here asVoidPointer y: there - here. - self forwardUnchecked: hereObj to: destination + (hereObj - here). - destination := destination + (there - here). - freeBytes := freeBytes - (there - here). - hereObj := self objectStartingAt: there. - here := there. - hereObjHeader := self atLeastClassIndexHalfHeader: hereObj]. - - moved - ifTrue: "we did overwrite it; we need to repair the free list" - [| nextNextFree | - nextFree ~= 0 ifTrue: - [nextNextFree := self nextInSortedFreeListLink: nextFree given: thisFreeChunk. - self assert: (self isFreeObject: nextFree)]. - (destination > thisFreeChunk "if false couldn't move anything" - and: [destination < endOfFree]) "if false, filled entire free chunk" - ifTrue: - [thisFreeChunk := self initFreeChunkWithBytes: endOfFree - destination at: destination. - self inSortedFreeListLink: prevFreeChunk to: thisFreeChunk given: prevPrevFreeChunk. - self inSortedFreeListLink: thisFreeChunk to: nextFree given: prevFreeChunk. - nextFree ~= 0 ifTrue: - [self inSortedFreeListLink: nextFree to: nextNextFree given: thisFreeChunk]. - prevPrevFreeChunk := prevFreeChunk. - prevFreeChunk := thisFreeChunk. - thisFreeChunk := nextFree] - ifFalse: - [self inSortedFreeListLink: prevFreeChunk to: nextFree given: prevPrevFreeChunk. - nextFree ~= 0 ifTrue: - [self inSortedFreeListLink: nextFree to: nextNextFree given: prevFreeChunk]. - thisFreeChunk := nextFree]] - ifFalse: "out of space (or immobile object); move on up the free list..." - [prevPrevFreeChunk := prevFreeChunk. - prevFreeChunk := thisFreeChunk. - thisFreeChunk := nextFree]. - - (self isMobileObjectHeader: hereObjHeader) ifFalse: - [^maxFreeChunk >= startAddress ifTrue: [0] ifFalse: [there]]. - - "Was the loop stopped by a pig? If so, try and find space for it" - there - here >= (self averageObjectSizeInBytes * 8) ifTrue: "256b in 32 bit, 512b in 64 bit" - [| usedChunk | - usedChunk := self tryToMovePig: hereObj at: here end: there. - "if it couldn't be moved we need to advance, so always - set here to there whether the pig was moved or not." - hereObj := self objectStartingAt: there. - here := there. - hereObjHeader := self atLeastClassIndexHalfHeader: hereObj. - "In general it's a bad idea to reset the enumeration; it leads to N^2 behaviour - when encountering pigs. But if the move affected the enumeration this is - simpler than resetting the list pointers." - (usedChunk = prevPrevFreeChunk - or: [usedChunk = prevFreeChunk - or: [usedChunk = thisFreeChunk]]) ifTrue: - ["reset the scan for free space back to the start of the list" - prevPrevFreeChunk := prevFreeChunk := 0. - thisFreeChunk := firstFreeChunk]]. - - ((here > startAddress and: [there >= limit]) - or: [maxFreeChunk >= startAddress]) ifTrue: - [^maxFreeChunk >= startAddress ifTrue: [0] ifFalse: [there]]]. - ^here! Item was removed: - ----- Method: SpurMemoryManager>>noForwardersBelowFirstFreeChunk (in category 'gc - global') ----- - noForwardersBelowFirstFreeChunk - self allOldSpaceEntitiesDo: - [:o| - (self oop: o isGreaterThanOrEqualTo: firstFreeChunk) ifTrue: - [^true]. - (self isForwarded: o) ifTrue: - [^false]]. - ^true! Item was removed: - ----- Method: SpurMemoryManager>>pigCompact (in category 'compaction') ----- - pigCompact - "Traverse the sorted free list, moving objects from the high-end of - memory to the free objects in the low end of memory. Return when - the address at which objects are being copiecd to meets the address - from which objects are being copied from." - self cCode: '' inSmalltalk: [coInterpreter transcript nextPutAll: 'pig compacting...'; flush]. - self sortedFreeListPairwiseReverseDo: - [:low :high| | scanAddress | - self cCode: '' inSmalltalk: [coInterpreter transcript nextPut: $.; flush]. - scanAddress := self addressAfter: low. - [self oop: scanAddress isLessThan: high] whileTrue: - [scanAddress := self moveARunOfObjectsStartingAt: scanAddress upTo: high. - scanAddress = 0 ifTrue: - [^self]]]. - self assert: self checkTraversableSortedFreeList! Item was removed: - ----- Method: SpurMemoryManager>>printSortedFreeList (in category 'debug printing') ----- - printSortedFreeList - <api> - | freeChunk prevFree nextFree | - (firstFreeChunk > 0 and: [lastFreeChunk > firstFreeChunk]) ifFalse: - [coInterpreter print: 'sorted free list empty or corrupt'; cr. - ^self]. - freeChunk := firstFreeChunk. - prevFree := 0. - [((self addressCouldBeObj: freeChunk) - and: [self isFreeObject: freeChunk]) ifFalse: - [coInterpreter printHexnp: freeChunk; print: ' is not a free chunk!!' ; cr. - ^self]. - self printFreeChunk: freeChunk printAsTreeNode: false. - freeChunk ~= lastFreeChunk] whileTrue: - [nextFree := self nextInSortedFreeListLink: freeChunk given: prevFree. - prevFree := freeChunk. - freeChunk := nextFree]! Item was removed: - ----- Method: SpurMemoryManager>>rebuildFreeListsForPigCompact (in category 'compaction') ----- - rebuildFreeListsForPigCompact - "Rebuild the free lists from the doubly-linked free list." - <inline: false> - self assert: self checkTraversableSortedFreeList. - totalFreeOldSpace := 0. - self sortedFreeListDo: - [:freeObj| | start bytes | - bytes := (self bytesInObject: freeObj). - start := self startOfObject: freeObj. - self addFreeChunkWithBytes: bytes at: start]. - self checkFreeSpace: GCModeFull! Item was removed: - ----- Method: SpurMemoryManager>>sortedFreeListDo: (in category 'compaction') ----- - sortedFreeListDo: aBlock - "Evaluate aBlock with ascending entries in the free list" - | free nextFree prevFree prevPrevFree | - <var: #free type: #usqInt> - <var: #nextFree type: #usqInt> - <var: #prevFree type: #usqInt> - <var: #prevPrevFree type: #usqInt> - <inline: true> - free := firstFreeChunk. - prevPrevFree := prevFree := 0. - [free ~= 0] whileTrue: - [nextFree := self nextInSortedFreeListLink: free given: prevFree. - self assert: (self isFreeObject: free). - self assert: (nextFree = 0 or: [nextFree > free and: [self isFreeObject: nextFree]]). - self assert: (prevFree = 0 or: [prevFree < free]). - aBlock value: free. - prevPrevFree := prevFree. - prevFree := free. - free := nextFree]! Item was removed: - ----- Method: SpurMemoryManager>>sortedFreeListPairwiseReverseDo: (in category 'compaction') ----- - sortedFreeListPairwiseReverseDo: aBinaryBlock - "Evaluate aBinaryBlock with adjacent entries in the free list, from - high address to low address. The second argument is in fact the - start of the next free chunk, not the free chunk itself. Use - endOfMemory - bridgeSize as the second argument in the first evaluation." - | free prevFree prevPrevFree | - <var: #free type: #usqInt> - <var: #prevFree type: #usqInt> - <var: #prevPrevFree type: #usqInt> - <inline: true> - free := lastFreeChunk. - prevPrevFree := prevFree := 0. - [free ~= 0] whileTrue: - [aBinaryBlock value: free value: (prevFree = 0 - ifTrue: [endOfMemory - self bridgeSize] - ifFalse: [self startOfObject: prevFree]). - "post evaluation of aBinaryBlock the value of free may be invalid - because moveARunOfObjectsStartingAt:upTo: may have filled it. - So reconstruct the position in the enumeration." - prevFree = 0 - ifTrue: - [self assert: free = lastFreeChunk. - prevFree := lastFreeChunk. - free := self nextInSortedFreeListLink: lastFreeChunk given: 0] - ifFalse: - [self assert: (self isFreeObject: prevFree). - prevPrevFree = 0 - ifTrue: - [prevPrevFree := lastFreeChunk. - prevFree := self nextInSortedFreeListLink: lastFreeChunk given: 0] - ifFalse: - [self assert: (self isFreeObject: prevPrevFree). - free := self nextInSortedFreeListLink: prevFree given: prevPrevFree. - prevPrevFree := prevFree. - prevFree := free]. - free := self nextInSortedFreeListLink: prevFree given: prevPrevFree]]! Item was removed: - ----- Method: SpurMemoryManager>>sortedFreeObject: (in category 'free space') ----- - sortedFreeObject: objOop - self subclassResponsibility! Item was added: + ----- Method: SpurMemoryManager>>statCompactPassCount (in category 'accessing') ----- + statCompactPassCount + ^statCompactPassCount! Item was added: + ----- Method: SpurMemoryManager>>statCompactPassCount: (in category 'accessing') ----- + statCompactPassCount: anInteger + ^statCompactPassCount := anInteger! Item was removed: - ----- Method: SpurMemoryManager>>sweepToCoallesceFreeSpaceForPigCompactFrom: (in category 'compaction') ----- - sweepToCoallesceFreeSpaceForPigCompactFrom: lowestForwarder - "Coallesce free chunks and forwarders, maintaining the doubly-linked free list." - | lowest firstOfFreeRun startOfFreeRun endOfFreeRun prevPrevFree prevFree | - <var: #lowestForwarder type: #usqInt> - lowest := (lowestForwarder = 0 ifTrue: [endOfMemory] ifFalse: [lowestForwarder]) - min: (firstFreeChunk = 0 ifTrue: [endOfMemory] ifFalse: [firstFreeChunk]). - firstOfFreeRun := prevPrevFree := prevFree := 0. - self allOldSpaceEntitiesFrom: lowest do: - [:o| - ((self isFreeObject: o) or: [self isForwarded: o]) - ifTrue: - [firstOfFreeRun = 0 ifTrue: - [self setObjectFree: o. - firstOfFreeRun := o. - startOfFreeRun := self startOfObject: o]. - endOfFreeRun := o] - ifFalse: - [firstOfFreeRun ~= 0 ifTrue: - [| bytes | - bytes := (self addressAfter: endOfFreeRun) - startOfFreeRun. - firstOfFreeRun := self initFreeChunkWithBytes: bytes at: startOfFreeRun. - self inSortedFreeListLink: prevFree to: firstOfFreeRun given: prevPrevFree. - prevPrevFree := prevFree. - prevFree := firstOfFreeRun. - firstOfFreeRun := 0]]]. - firstOfFreeRun ~= 0 ifTrue: - [| bytes | - bytes := (self addressAfter: endOfFreeRun) - startOfFreeRun. - firstOfFreeRun := self initFreeChunkWithBytes: bytes at: startOfFreeRun. - self inSortedFreeListLink: prevFree to: firstOfFreeRun given: prevPrevFree. - prevPrevFree := prevFree. - prevFree := firstOfFreeRun. - firstOfFreeRun := 0]. - prevFree ~= firstFreeChunk ifTrue: - [self storePointer: self freeChunkNextIndex - ofFreeChunk: prevFree - withValue: prevPrevFree]. - lastFreeChunk := prevFree. - self inSortedFreeListLink: lastFreeChunk to: 0 given: prevPrevFree. - self assert: self checkTraversableSortedFreeList! Item was removed: - ----- Method: SpurMemoryManager>>sweepToFollowForwardersForPigCompact (in category 'compaction') ----- - sweepToFollowForwardersForPigCompact - "Sweep, following forwarders in all live objects. - Answer the lowest forwarder in oldSpace." - | lowestForwarder | - <var: #lowestForwarder type: #usqInt> - self assert: (freeStart = scavenger eden start - and: [scavenger futureSurvivorStart = scavenger futureSpace start]). - self allPastSpaceObjectsDo: - [:o| - (self isForwarded: o) ifFalse: - [0 to: (self numPointerSlotsOf: o) - 1 do: - [:i| | f | - f := self fetchPointer: i ofObject: o. - (self isOopForwarded: f) ifTrue: - [f := self followForwarded: f. - self storePointerUnchecked: i ofObject: o withValue: f]]]]. - lowestForwarder := 0. - self allOldSpaceObjectsDo: - [:o| - (self isForwarded: o) - ifTrue: - [lowestForwarder = 0 ifTrue: - [lowestForwarder := o]] - ifFalse: - [0 to: (self numPointerSlotsOf: o) - 1 do: - [:i| | f | - f := self fetchPointer: i ofObject: o. - (self isOopForwarded: f) ifTrue: - [f := self followForwarded: f. - self storePointer: i ofObject: o withValue: f]]]]. - ^lowestForwarder! Item was added: + ----- Method: SpurMemoryManager>>totalFreeOldSpace: (in category 'accessing') ----- + totalFreeOldSpace: anInteger + totalFreeOldSpace := anInteger! Item was removed: - ----- Method: SpurMemoryManager>>tryToMovePig:at:end: (in category 'compaction') ----- - tryToMovePig: pigObj at: pigStart end: pigEnd - "Try to move a pig (a largish object) to a free chunk in low memory. - Answer the freeChunk that was used to house the moved pig, or - 0 if no free chunk could be found." - | freeChunk prevFree prevPrevFree pigBytes nextNext | - prevPrevFree := prevFree := 0. - freeChunk := firstFreeChunk. - pigBytes := pigEnd - pigStart. - [freeChunk ~= 0 and: [freeChunk < pigObj]] whileTrue: - [| next dest chunkBytes newChunk | - next := self nextInSortedFreeListLink: freeChunk given: prevFree. - dest := self startOfObject: freeChunk. - chunkBytes := (self addressAfter: freeChunk) - dest. - "N.B. *must* add allocationUnit, not subtract, to avoid unsigned arithmetic issues when chunkBytes = 0" - (chunkBytes = pigBytes - or: [chunkBytes > (pigBytes + self allocationUnit)]) ifTrue: - [self mem: dest asVoidPointer cp: pigStart asVoidPointer y: pigBytes. - self forwardUnchecked: pigObj to: dest + (pigObj - pigStart). - next ~= 0 ifTrue: - [nextNext := self nextInSortedFreeListLink: next given: freeChunk]. - "now either shorten the chunk, or remove it, adjusting the links to keep the list sorted." - pigBytes < chunkBytes "if false, filled entire free chunk" - ifTrue: - [newChunk := self initFreeChunkWithBytes: chunkBytes - pigBytes at: dest + pigBytes. - self inSortedFreeListLink: prevFree to: newChunk given: prevPrevFree. - self inSortedFreeListLink: newChunk to: next given: prevFree. - next ~= 0 ifTrue: - [self inSortedFreeListLink: next to: nextNext given: newChunk]] - ifFalse: - [self inSortedFreeListLink: prevFree to: next given: prevPrevFree. - next ~= 0 ifTrue: - [self inSortedFreeListLink: next to: nextNext given: prevFree]]. - "self checkTraversableSortedFreeList". - ^freeChunk]. - prevPrevFree := prevFree. - prevFree := freeChunk. - freeChunk := next]. - ^0! Item was removed: - ----- Method: SpurMemoryManager>>unmarkSurvivingObjectsForPigCompact (in category 'gc - global') ----- - unmarkSurvivingObjectsForPigCompact - self allPastSpaceObjectsDo: - [:objOop| - (self isMarked: objOop) ifTrue: - [self setIsMarkedOf: objOop to: false]]! Item was added: + CogClass subclass: #SpurPigCompactor + instanceVariableNames: 'manager scavenger coInterpreter firstFreeChunk lastFreeChunk numCompactionPasses' + classVariableNames: 'CompactionPassesForGC CompactionPassesForSnapshot' + poolDictionaries: 'SpurMemoryManagementConstants VMBasicConstants VMSpurObjectRepresentationConstants' + category: 'VMMaker-SpurMemoryManager'! + + !SpurPigCompactor commentStamp: 'eem 12/16/2016 16:20' prior: 0! + SpurPigCompactor implements the second compactioon algorithm implemented for Spur. It attempts to move ovbjects down from the end of memory to occupy free chunks in low memory. It uses Knuth's xor-encoding technique to encode a doubly-linked list in the forwarding field of each free chunk (free chunks, like Spiur objects, being known to have at least one field). This algorithm has poor performance for two reasons. First, it does not preserve object order, scrambling the order of objects as it moves the highest objects down to the lowest free chunks. Second it appears to perform badly, occasionally causing very long pauses. + + Instance Variables + coInterpreter: <StackInterpreter> + firstFreeChunk: <Integer> + lastFreeChunk: <Integer> + manager: <SpurMemoryManager> + numCompactionPasses: <Integer> + scavenger: <SpurGenerationScavenger> + + firstFreeChunk + - oop of freeChunk or 0 + + lastFreeChunk + - oop of freeChunk or 0 + + numCompactionPasses + - 2 for normal GC, 3 for snapshot! Item was added: + ----- Method: SpurPigCompactor class>>initialize (in category 'class initialization') ----- + initialize + "Pig compact can be repeated to compact better. Experience shows that 3 times + compacts very well, desirable for snapshots. But this is overkill for normal GCs." + CompactionPassesForGC := 2. + CompactionPassesForSnapshot := 3! Item was added: + ----- Method: SpurPigCompactor>>abstractPigCompaction (in category 'compaction - analysis') ----- + abstractPigCompaction + "This method answers a rough estimate of compactibility using a pig (a large free chunk)." + <doNotGenerate> + | pig pork moved unmoved nmoved nunmoved | + pig := self findAPig. + pork := manager bytesInObject: pig. + moved := unmoved := nmoved := nunmoved := 0. + manager allOldSpaceObjectsFrom: pig do: + [:o| | bytes | + bytes := manager bytesInObject: o. + bytes <= pork + ifTrue: + [moved := moved + bytes. + nmoved := nmoved + 1. + pork := pork - bytes] + ifFalse: + [unmoved := unmoved + bytes. + nunmoved := nunmoved + 1]]. + ^{ manager bytesInObject: pig. pork. moved. nmoved. unmoved. nunmoved }! Item was added: + ----- Method: SpurPigCompactor>>biasForGC (in category 'compaction - api') ----- + biasForGC + numCompactionPasses := CompactionPassesForGC! Item was added: + ----- Method: SpurPigCompactor>>biasForSnapshot (in category 'compaction - api') ----- + biasForSnapshot + numCompactionPasses := CompactionPassesForSnapshot! Item was added: + ----- Method: SpurPigCompactor>>checkNoForwardersBelowFirstFreeChunk (in category 'compaction - asserts') ----- + checkNoForwardersBelowFirstFreeChunk + manager allOldSpaceEntitiesDo: + [:o| + (self oop: o isGreaterThanOrEqualTo: firstFreeChunk) ifTrue: + [^true]. + (self asserta: (manager isForwarded: o) not) ifFalse: + [^false]]. + ^true! Item was added: + ----- Method: SpurPigCompactor>>checkTraversableSortedFreeList (in category 'compaction - asserts') ----- + checkTraversableSortedFreeList + | prevFree prevPrevFree freeChunk | + <api> + <inline: false> + prevFree := prevPrevFree := 0. + firstFreeChunk = 0 ifTrue: + [^lastFreeChunk = 0]. + freeChunk := firstFreeChunk. + manager allOldSpaceEntitiesDo: + [:o| | objOop next limit | + (manager isFreeObject: o) ifTrue: + [self assert: o = freeChunk. + next := manager nextInSortedFreeListLink: freeChunk given: prevFree. + limit := next = 0 ifTrue: [manager endOfMemory] ifFalse: [next]. + "coInterpreter transcript cr; print: freeChunk; tab; print: o; tab; print: prevFree; nextPutAll: '<->'; print: next; flush." + objOop := freeChunk. + [self oop: (objOop := manager objectAfter: objOop) isLessThan: limit] whileTrue: + [self assert: (manager isFreeObject: objOop) not]. + prevPrevFree := prevFree. + prevFree := freeChunk. + freeChunk := next]]. + self assert: prevFree = lastFreeChunk. + self assert: (manager nextInSortedFreeListLink: lastFreeChunk given: 0) = prevPrevFree. + self assert: freeChunk = 0. + ^true! Item was added: + ----- Method: SpurPigCompactor>>coInterpreter: (in category 'initialization') ----- + coInterpreter: aVMSimulator + <doNotGenerate> + coInterpreter := aVMSimulator! Item was added: + ----- Method: SpurPigCompactor>>compact (in category 'compaction - api') ----- + compact + "We'd like to use exact fit followed by best or first fit, but it doesn't work + well enough in practice. So use pig compact. Fill large free objects starting + from low memory with objects taken from the end of memory." + <inline: #never> "for profiling" + manager statCompactPassCount: manager statCompactPassCount + 1. + self assert: (firstFreeChunk = 0 or: [manager isFreeObject: firstFreeChunk]). + 1 to: numCompactionPasses do: + [:i| + self pigCompact. + self eliminateAndFreeForwardersForPigCompact]. + + "The free lists are zeroed in freeUnmarkedObjectsAndSortAndCoalesceFreeSpaceForPigCompact. + They should still be zero here" + self assert: manager freeListHeadsEmpty. + self rebuildFreeListsForPigCompact! Item was added: + ----- Method: SpurPigCompactor>>eliminateAndFreeForwardersForPigCompact (in category 'compaction') ----- + eliminateAndFreeForwardersForPigCompact + "As the final phase of global garbage collect, sweep the heap to follow + forwarders, then free forwarders, coalescing with free space as we go." + <inline: false> + | lowestForwarder | + <var: #lowestForwarder type: #usqInt> + self assert: (manager isForwarded: manager nilObject) not. + self assert: (manager isForwarded: manager falseObject) not. + self assert: (manager isForwarded: manager trueObject) not. + self assert: (manager isForwarded: manager freeListsObj) not. + self assert: (manager isForwarded: manager hiddenRootsObject) not. + self assert: (manager isForwarded: manager classTableFirstPage) not. + manager followSpecialObjectsOop. + manager followForwardedObjStacks. + coInterpreter mapInterpreterOops. + scavenger followRememberedForwardersAndForgetFreeObjectsForPigCompact. + self unmarkSurvivingObjectsForPigCompact. + lowestForwarder := self sweepToFollowForwardersForPigCompact. + self sweepToCoallesceFreeSpaceForPigCompactFrom: lowestForwarder. + self assert: manager numberOfForwarders = 0! Item was added: + ----- Method: SpurPigCompactor>>findAPig (in category 'compaction - analysis') ----- + findAPig + "Answer a large low free chunk." + <doNotGenerate> + | pig | + manager allObjectsInFreeTreeDo: + [:f| + (manager bytesInObject: f) >= 1000000 ifTrue: + [(pig isNil or: [pig > f]) ifTrue: + [pig := f]]]. + ^pig! Item was added: + ----- Method: SpurPigCompactor>>freeUnmarkedObjectsAndPrepareFreeSpace (in category 'compaction - api') ----- + freeUnmarkedObjectsAndPrepareFreeSpace + <inline: true> + self freeUnmarkedObjectsAndSortAndCoalesceFreeSpaceForPigCompact! Item was added: + ----- Method: SpurPigCompactor>>freeUnmarkedObjectsAndSortAndCoalesceFreeSpaceForPigCompact (in category 'compaction') ----- + freeUnmarkedObjectsAndSortAndCoalesceFreeSpaceForPigCompact + "Sweep all of old space, freeing unmarked objects, coalescing free chunks, and sorting free space. + + Doubly-link the free chunks in address order through the freeChunkNextIndex field using the + xor trick to use only one field, see e.g. + The Art of Computer Programming, Vol 1, D.E. Knuth, 3rd Ed, Sec 2.2.4 `Circular Lists', exercise. 18 + http://en.wikipedia.org/wiki/XOR_linked_list. + Record the lowest free object in firstFreeChunk and the highest in lastFreeChunk. + + Let the segmentManager mark which segments contain pinned objects via notePinned:." + + | prevPrevFree prevFree | + <inline: #never> "for profiling" + manager checkFreeSpace: GCModeFull. + scavenger forgetUnmarkedRememberedObjects. + manager doScavenge: MarkOnTenure. + manager segmentManager prepareForGlobalSweep."for notePinned:" + "throw away the list heads, including the tree." + manager resetFreeListHeads. + firstFreeChunk := prevPrevFree := prevFree := 0. + manager allOldSpaceEntitiesForCoalescingFrom: manager firstObject do: + [:o| + self assert: (firstFreeChunk = 0 or: [manager isFreeObject: firstFreeChunk]). + (manager isMarked: o) + ifTrue: "forwarders should have been followed in markAndTrace:" + [self assert: (manager isForwarded: o) not. + manager setIsMarkedOf: o to: false. "this will unmark bridges. undo the damage in notePinned:" + (manager isPinned: o) ifTrue: + [manager segmentManager notePinned: o]] + ifFalse: "unmarked; two cases, an unreachable object or a free chunk." + [| here | + self assert: (manager isRemembered: o) not. "scavenger should have cleared this above" + here := manager coallesceFreeChunk: o. + manager setObjectFree: here. + self inSortedFreeListLink: prevFree to: here given: prevPrevFree. + prevPrevFree := prevFree. + prevFree := here]]. + prevFree ~= firstFreeChunk ifTrue: + [manager storePointer: manager freeChunkNextIndex + ofFreeChunk: prevFree + withValue: prevPrevFree]. + lastFreeChunk := prevFree. + self inSortedFreeListLink: lastFreeChunk to: 0 given: prevPrevFree. + self assert: self checkTraversableSortedFreeList! Item was added: + ----- Method: SpurPigCompactor>>inSortedFreeListLink:to:given: (in category 'compaction') ----- + inSortedFreeListLink: freeChunk to: nextFree given: prevFree + "Doubly-link the free chunk in address order through the freeChunkNextIndex field using the + xor trick to use only one field, see e.g. + The Art of Computer Programming, Vol 1, D.E. Knuth, 3rd Ed, Sec 2.2.4 `Circular Lists', exercise. 18 + http://en.wikipedia.org/wiki/XOR_linked_list." + freeChunk = 0 + ifTrue: + [firstFreeChunk := nextFree] + ifFalse: + [manager storePointer: manager freeChunkNextIndex + ofFreeChunk: freeChunk + withUncheckedValue: (prevFree bitXor: nextFree)]! Item was added: + ----- Method: SpurPigCompactor>>manager: (in category 'initialization') ----- + manager: aSpurNBitMMXEndianSimulator + <doNotGenerate> + manager := aSpurNBitMMXEndianSimulator. + aSpurNBitMMXEndianSimulator coInterpreter ifNotNil: + [:coint| coInterpreter := coint]. + aSpurNBitMMXEndianSimulator scavenger ifNotNil: + [:scav| scavenger := scav]! Item was added: + ----- Method: SpurPigCompactor>>moveARunOfObjectsStartingAt:upTo: (in category 'compaction') ----- + moveARunOfObjectsStartingAt: startAddress upTo: limit + "Move the sequence of movable objects starting at startAddress. Answer the start + of the next sequence of movable objects after a possible run of unmovable objects, + or the limit, if there are no more movable objects, or 0 if no more compaction can be + done. Compaction is done when the search through the freeList has reached the + address from which objects are being moved from. + + There are two broad cases to be dealt with here. One is a run of smallish objects + that can easily be moved into free chunks. The other is a large object that is unlikely + to fit in the typical free chunk. This second pig needs careful handling; it needs to be + moved to the lowest place it will fit and not cause the scan to skip lots of smaller + free chunks looking in vain for somewhere to put it." + <var: #startAddress type: #usqInt> + <var: #limit type: #usqInt> + <inline: false> + | here hereObj hereObjHeader prevPrevFreeChunk prevFreeChunk thisFreeChunk maxFreeChunk | + <var: #here type: #usqInt> + <var: #there type: #usqInt> + <var: #nextFree type: #usqInt> + <var: #endOfFree type: #usqInt> + <var: #destination type: #usqInt> + <var: #maxFreeChunk type: #usqInt> + here := startAddress. + hereObj := manager objectStartingAt: startAddress. + hereObjHeader := manager atLeastClassIndexHalfHeader: hereObj. + prevPrevFreeChunk := prevFreeChunk := 0. + thisFreeChunk := maxFreeChunk := firstFreeChunk. + [thisFreeChunk ~= 0] whileTrue: + [| freeBytes endOfFree nextFree destination there moved | + + "skip any initial immobile objects" + [(manager isMobileObjectHeader: hereObjHeader)] whileFalse: + [here := manager addressAfter: hereObj. + here >= limit ifTrue: + [^maxFreeChunk >= startAddress ifTrue: [0] ifFalse: [limit]]. + hereObj := manager objectStartingAt: here. + hereObjHeader := manager atLeastClassIndexHalfHeader: hereObj]. + + "grab a free chunk, and the following one, because we want to overwrite this one." + self assert: ((manager isFreeObject: firstFreeChunk) and: [manager isFreeObject: thisFreeChunk]). + freeBytes := manager bytesInObject: thisFreeChunk. + nextFree := manager nextInSortedFreeListLink: thisFreeChunk given: prevFreeChunk. + destination := manager startOfObject: thisFreeChunk. + endOfFree := destination + freeBytes. + moved := false. + maxFreeChunk := maxFreeChunk max: nextFree. + self assert: (nextFree = 0 or: [manager isFreeObject: nextFree]). + + "move as many objects as will fit in freeBytes..." + [there := manager addressAfter: hereObj. + "N.B. *must* add allocationUnit, not subtract, to avoid unsigned arithmetic issues when freeBytes = 0" + (manager isMobileObjectHeader: hereObjHeader) + and: [freeBytes > (there - here + manager allocationUnit) + or: [freeBytes = (there - here)]]] whileTrue: + [moved := true. + manager mem: destination asVoidPointer cp: here asVoidPointer y: there - here. + manager forwardUnchecked: hereObj to: destination + (hereObj - here). + destination := destination + (there - here). + freeBytes := freeBytes - (there - here). + hereObj := manager objectStartingAt: there. + here := there. + hereObjHeader := manager atLeastClassIndexHalfHeader: hereObj]. + + moved + ifTrue: "we did overwrite it; we need to repair the free list" + [| nextNextFree | + nextFree ~= 0 ifTrue: + [nextNextFree := manager nextInSortedFreeListLink: nextFree given: thisFreeChunk. + self assert: (manager isFreeObject: nextFree)]. + (destination > thisFreeChunk "if false couldn't move anything" + and: [destination < endOfFree]) "if false, filled entire free chunk" + ifTrue: + [thisFreeChunk := manager initFreeChunkWithBytes: endOfFree - destination at: destination. + self inSortedFreeListLink: prevFreeChunk to: thisFreeChunk given: prevPrevFreeChunk. + self inSortedFreeListLink: thisFreeChunk to: nextFree given: prevFreeChunk. + nextFree ~= 0 ifTrue: + [self inSortedFreeListLink: nextFree to: nextNextFree given: thisFreeChunk]. + prevPrevFreeChunk := prevFreeChunk. + prevFreeChunk := thisFreeChunk. + thisFreeChunk := nextFree] + ifFalse: + [self inSortedFreeListLink: prevFreeChunk to: nextFree given: prevPrevFreeChunk. + nextFree ~= 0 ifTrue: + [self inSortedFreeListLink: nextFree to: nextNextFree given: prevFreeChunk]. + thisFreeChunk := nextFree]] + ifFalse: "out of space (or immobile object); move on up the free list..." + [prevPrevFreeChunk := prevFreeChunk. + prevFreeChunk := thisFreeChunk. + thisFreeChunk := nextFree]. + + (manager isMobileObjectHeader: hereObjHeader) ifFalse: + [^maxFreeChunk >= startAddress ifTrue: [0] ifFalse: [there]]. + + "Was the loop stopped by a pig? If so, try and find space for it" + there - here >= (manager averageObjectSizeInBytes * 8) ifTrue: "256b in 32 bit, 512b in 64 bit" + [| usedChunk | + usedChunk := self tryToMovePig: hereObj at: here end: there. + "if it couldn't be moved we need to advance, so always + set here to there whether the pig was moved or not." + hereObj := manager objectStartingAt: there. + here := there. + hereObjHeader := manager atLeastClassIndexHalfHeader: hereObj. + "In general it's a bad idea to reset the enumeration; it leads to N^2 behaviour + when encountering pigs. But if the move affected the enumeration this is + simpler than resetting the list pointers." + (usedChunk = prevPrevFreeChunk + or: [usedChunk = prevFreeChunk + or: [usedChunk = thisFreeChunk]]) ifTrue: + ["reset the scan for free space back to the start of the list" + prevPrevFreeChunk := prevFreeChunk := 0. + thisFreeChunk := firstFreeChunk]]. + + ((here > startAddress and: [there >= limit]) + or: [maxFreeChunk >= startAddress]) ifTrue: + [^maxFreeChunk >= startAddress ifTrue: [0] ifFalse: [there]]]. + ^here! Item was added: + ----- Method: SpurPigCompactor>>noForwardersBelowFirstFreeChunk (in category 'compaction - asserts') ----- + noForwardersBelowFirstFreeChunk + manager allOldSpaceEntitiesDo: + [:o| + (self oop: o isGreaterThanOrEqualTo: firstFreeChunk) ifTrue: + [^true]. + (manager isForwarded: o) ifTrue: + [^false]]. + ^true! Item was added: + ----- Method: SpurPigCompactor>>pigCompact (in category 'compaction') ----- + pigCompact + "Traverse the sorted free list, moving objects from the high-end of + memory to the free objects in the low end of memory. Return when + the address at which objects are being copiecd to meets the address + from which objects are being copied from." + self cCode: '' inSmalltalk: [coInterpreter transcript nextPutAll: 'pig compacting...'; flush]. + self sortedFreeListPairwiseReverseDo: + [:low :high| | scanAddress | + self cCode: '' inSmalltalk: [coInterpreter transcript nextPut: $.; flush]. + scanAddress := manager addressAfter: low. + [self oop: scanAddress isLessThan: high] whileTrue: + [scanAddress := self moveARunOfObjectsStartingAt: scanAddress upTo: high. + scanAddress = 0 ifTrue: + [^self]]]. + self assert: self checkTraversableSortedFreeList! Item was added: + ----- Method: SpurPigCompactor>>printSortedFreeList (in category 'debug printing') ----- + printSortedFreeList + <api> + | freeChunk prevFree nextFree | + (firstFreeChunk > 0 and: [lastFreeChunk > firstFreeChunk]) ifFalse: + [coInterpreter print: 'sorted free list empty or corrupt'; cr. + ^self]. + freeChunk := firstFreeChunk. + prevFree := 0. + [((manager addressCouldBeObj: freeChunk) + and: [manager isFreeObject: freeChunk]) ifFalse: + [coInterpreter printHexnp: freeChunk; print: ' is not a free chunk!!' ; cr. + ^self]. + manager printFreeChunk: freeChunk printAsTreeNode: false. + freeChunk ~= lastFreeChunk] whileTrue: + [nextFree := manager nextInSortedFreeListLink: freeChunk given: prevFree. + prevFree := freeChunk. + freeChunk := nextFree]! Item was added: + ----- Method: SpurPigCompactor>>rebuildFreeListsForPigCompact (in category 'compaction') ----- + rebuildFreeListsForPigCompact + "Rebuild the free lists from the doubly-linked free list." + <inline: false> + self assert: self checkTraversableSortedFreeList. + manager totalFreeOldSpace: 0. + self sortedFreeListDo: + [:freeObj| | start bytes | + bytes := (manager bytesInObject: freeObj). + start := manager startOfObject: freeObj. + manager addFreeChunkWithBytes: bytes at: start]. + manager checkFreeSpace: GCModeFull! Item was added: + ----- Method: SpurPigCompactor>>sortedFreeListDo: (in category 'compaction') ----- + sortedFreeListDo: aBlock + "Evaluate aBlock with ascending entries in the free list" + | free nextFree prevFree prevPrevFree | + <var: #free type: #usqInt> + <var: #nextFree type: #usqInt> + <var: #prevFree type: #usqInt> + <var: #prevPrevFree type: #usqInt> + <inline: true> + free := firstFreeChunk. + prevPrevFree := prevFree := 0. + [free ~= 0] whileTrue: + [nextFree := manager nextInSortedFreeListLink: free given: prevFree. + self assert: (manager isFreeObject: free). + self assert: (nextFree = 0 or: [nextFree > free and: [manager isFreeObject: nextFree]]). + self assert: (prevFree = 0 or: [prevFree < free]). + aBlock value: free. + prevPrevFree := prevFree. + prevFree := free. + free := nextFree]! Item was added: + ----- Method: SpurPigCompactor>>sortedFreeListPairwiseReverseDo: (in category 'compaction') ----- + sortedFreeListPairwiseReverseDo: aBinaryBlock + "Evaluate aBinaryBlock with adjacent entries in the free list, from + high address to low address. The second argument is in fact the + start of the next free chunk, not the free chunk itself. Use + endOfMemory - bridgeSize as the second argument in the first evaluation." + | free prevFree prevPrevFree | + <var: #free type: #usqInt> + <var: #prevFree type: #usqInt> + <var: #prevPrevFree type: #usqInt> + <inline: true> + free := lastFreeChunk. + prevPrevFree := prevFree := 0. + [free ~= 0] whileTrue: + [aBinaryBlock value: free value: (prevFree = 0 + ifTrue: [manager endOfMemory - manager bridgeSize] + ifFalse: [manager startOfObject: prevFree]). + "post evaluation of aBinaryBlock the value of free may be invalid + because moveARunOfObjectsStartingAt:upTo: may have filled it. + So reconstruct the position in the enumeration." + prevFree = 0 + ifTrue: + [self assert: free = lastFreeChunk. + prevFree := lastFreeChunk. + free := manager nextInSortedFreeListLink: lastFreeChunk given: 0] + ifFalse: + [self assert: (manager isFreeObject: prevFree). + prevPrevFree = 0 + ifTrue: + [prevPrevFree := lastFreeChunk. + prevFree := manager nextInSortedFreeListLink: lastFreeChunk given: 0] + ifFalse: + [self assert: (manager isFreeObject: prevPrevFree). + free := manager nextInSortedFreeListLink: prevFree given: prevPrevFree. + prevPrevFree := prevFree. + prevFree := free]. + free := manager nextInSortedFreeListLink: prevFree given: prevPrevFree]]! Item was added: + ----- Method: SpurPigCompactor>>sweepToCoallesceFreeSpaceForPigCompactFrom: (in category 'compaction') ----- + sweepToCoallesceFreeSpaceForPigCompactFrom: lowestForwarder + "Coallesce free chunks and forwarders, maintaining the doubly-linked free list." + | lowest firstOfFreeRun startOfFreeRun endOfFreeRun prevPrevFree prevFree | + <var: #lowestForwarder type: #usqInt> + lowest := (lowestForwarder = 0 ifTrue: [manager endOfMemory] ifFalse: [lowestForwarder]) + min: (firstFreeChunk = 0 ifTrue: [manager endOfMemory] ifFalse: [firstFreeChunk]). + firstOfFreeRun := prevPrevFree := prevFree := 0. + manager allOldSpaceEntitiesFrom: lowest do: + [:o| + ((manager isFreeObject: o) or: [manager isForwarded: o]) + ifTrue: + [firstOfFreeRun = 0 ifTrue: + [manager setObjectFree: o. + firstOfFreeRun := o. + startOfFreeRun := manager startOfObject: o]. + endOfFreeRun := o] + ifFalse: + [firstOfFreeRun ~= 0 ifTrue: + [| bytes | + bytes := (manager addressAfter: endOfFreeRun) - startOfFreeRun. + firstOfFreeRun := manager initFreeChunkWithBytes: bytes at: startOfFreeRun. + self inSortedFreeListLink: prevFree to: firstOfFreeRun given: prevPrevFree. + prevPrevFree := prevFree. + prevFree := firstOfFreeRun. + firstOfFreeRun := 0]]]. + firstOfFreeRun ~= 0 ifTrue: + [| bytes | + bytes := (manager addressAfter: endOfFreeRun) - startOfFreeRun. + firstOfFreeRun := manager initFreeChunkWithBytes: bytes at: startOfFreeRun. + self inSortedFreeListLink: prevFree to: firstOfFreeRun given: prevPrevFree. + prevPrevFree := prevFree. + prevFree := firstOfFreeRun. + firstOfFreeRun := 0]. + prevFree ~= firstFreeChunk ifTrue: + [manager storePointer: manager freeChunkNextIndex + ofFreeChunk: prevFree + withValue: prevPrevFree]. + lastFreeChunk := prevFree. + self inSortedFreeListLink: lastFreeChunk to: 0 given: prevPrevFree. + self assert: self checkTraversableSortedFreeList! Item was added: + ----- Method: SpurPigCompactor>>sweepToFollowForwardersForPigCompact (in category 'compaction') ----- + sweepToFollowForwardersForPigCompact + "Sweep, following forwarders in all live objects. + Answer the lowest forwarder in oldSpace." + | lowestForwarder | + <var: #lowestForwarder type: #usqInt> + self assert: (manager freeStart = scavenger eden start + and: [scavenger futureSurvivorStart = scavenger futureSpace start]). + manager allPastSpaceObjectsDo: + [:o| + (manager isForwarded: o) ifFalse: + [0 to: (manager numPointerSlotsOf: o) - 1 do: + [:i| | f | + f := manager fetchPointer: i ofObject: o. + (manager isOopForwarded: f) ifTrue: + [f := manager followForwarded: f. + manager storePointerUnchecked: i ofObject: o withValue: f]]]]. + lowestForwarder := 0. + manager allOldSpaceObjectsDo: + [:o| + (manager isForwarded: o) + ifTrue: + [lowestForwarder = 0 ifTrue: + [lowestForwarder := o]] + ifFalse: + [0 to: (manager numPointerSlotsOf: o) - 1 do: + [:i| | f | + f := manager fetchPointer: i ofObject: o. + (manager isOopForwarded: f) ifTrue: + [f := manager followForwarded: f. + manager storePointer: i ofObject: o withValue: f]]]]. + ^lowestForwarder! Item was added: + ----- Method: SpurPigCompactor>>tryToMovePig:at:end: (in category 'compaction') ----- + tryToMovePig: pigObj at: pigStart end: pigEnd + "Try to move a pig (a largish object) to a free chunk in low memory. + Answer the freeChunk that was used to house the moved pig, or + 0 if no free chunk could be found." + | freeChunk prevFree prevPrevFree pigBytes nextNext | + prevPrevFree := prevFree := 0. + freeChunk := firstFreeChunk. + pigBytes := pigEnd - pigStart. + [freeChunk ~= 0 and: [freeChunk < pigObj]] whileTrue: + [| next dest chunkBytes newChunk | + next := manager nextInSortedFreeListLink: freeChunk given: prevFree. + dest := manager startOfObject: freeChunk. + chunkBytes := (manager addressAfter: freeChunk) - dest. + "N.B. *must* add allocationUnit, not subtract, to avoid unsigned arithmetic issues when chunkBytes = 0" + (chunkBytes = pigBytes + or: [chunkBytes > (pigBytes + manager allocationUnit)]) ifTrue: + [manager mem: dest asVoidPointer cp: pigStart asVoidPointer y: pigBytes. + manager forwardUnchecked: pigObj to: dest + (pigObj - pigStart). + next ~= 0 ifTrue: + [nextNext := manager nextInSortedFreeListLink: next given: freeChunk]. + "now either shorten the chunk, or remove it, adjusting the links to keep the list sorted." + pigBytes < chunkBytes "if false, filled entire free chunk" + ifTrue: + [newChunk := manager initFreeChunkWithBytes: chunkBytes - pigBytes at: dest + pigBytes. + self inSortedFreeListLink: prevFree to: newChunk given: prevPrevFree. + self inSortedFreeListLink: newChunk to: next given: prevFree. + next ~= 0 ifTrue: + [self inSortedFreeListLink: next to: nextNext given: newChunk]] + ifFalse: + [self inSortedFreeListLink: prevFree to: next given: prevPrevFree. + next ~= 0 ifTrue: + [self inSortedFreeListLink: next to: nextNext given: prevFree]]. + "self checkTraversableSortedFreeList". + ^freeChunk]. + prevPrevFree := prevFree. + prevFree := freeChunk. + freeChunk := next]. + ^0! Item was added: + ----- Method: SpurPigCompactor>>unmarkSurvivingObjectsForPigCompact (in category 'compaction') ----- + unmarkSurvivingObjectsForPigCompact + manager allPastSpaceObjectsDo: + [:objOop| + (manager isMarked: objOop) ifTrue: + [manager setIsMarkedOf: objOop to: false]]!

1 0

VM Maker: Cog-eem.333.mcz
by commits＠source.squeak.org 17 Dec '16

17 Dec '16

Eliot Miranda uploaded a new version of Cog to project VM Maker: http://source.squeak.org/VMMaker/Cog-eem.333.mcz ==================== Summary ==================== Name: Cog-eem.333 Author: eem Time: 16 December 2016, 4:42:14.563053 pm UUID: 2df65635-9a33-4af3-90a7-dfe6de9f6480 Ancestors: Cog-eem.332 Add scripts to help extract the compaction algorithm into its own class, prior to being superceded. =============== Diff against Cog-eem.332 =============== Item was added: + ----- Method: CogScripts class>>changedSpurCompactorMethods (in category 'separate vm scripts') ----- + changedSpurCompactorMethods + "Answer the methods in SpurPigCompactor and subclasses that change if sends to self for methods implemented + in SpurMemoryManager become sends to manager." + "CogScripts changedSpurCompactorMethods" + | selectors compactorVars spurMemoryManagerSelectors rules model environment sortedChanges | + selectors := (self systemNavigation allAccessesTo: 'firstFreeChunk' from: SpurMemoryManager) asSet, + (self systemNavigation allAccessesTo: 'lastFreeChunk' from: SpurMemoryManager) + collect: [:mr| mr selector]. + selectors addAll: (SpurMemoryManager selectors select: [:s| ('*pigcompact*' match: s) or: [s endsWith: 'Pig']]). + compactorVars := #('numCompactionPasses' 'firstFreeChunk' 'lastFreeChunk' 'scavenger' 'coInterpreter'). + spurMemoryManagerSelectors := SpurMemoryManager selectors. + rules := RBParseTreeRewriter new. + rules + replace: 'self `@method: ``@args' + with: 'manager `@method: ``@args' + when: [:node| (spurMemoryManagerSelectors includes: node selector) and: [(selectors includes: node selector) not]]. + (SpurMemoryManager instVarNames reject: [:iv| compactorVars includes: iv]) do: + [:instVar| + rules + replace: instVar, ' := ``@args' with: 'manager ', instVar, ': ``@args'; + replace: instVar with: 'manager ', instVar]. + #( 'nilObj' 'trueObj' 'falseObj') + do: [:instVar| + rules + replace: instVar, ' := ``@args' with: 'manager ', instVar, 'ect: ``@args'; + replace: instVar with: 'manager ', instVar, 'ect']. + model := RBNamespace new. + environment := RBBrowserEnvironment new forClasses: { SpurMemoryManager }. + environment classesAndSelectorsDo: + [ :class :selector | | tree | + ((selectors includes: selector) + and: [rules executeTree: (tree := class parseTreeFor: selector)]) ifTrue: + [model compile: rules tree newSource in: class classified: (class whichCategoryIncludesSelector: selector)]]. + false ifTrue: [model changes inspect]. + false ifTrue: "shortest change:" + [(model changes changes inject: model changes changes first into: [:c1 :c2| c1 printString size < c2 printString size ifTrue: [c1] ifFalse: [c2]]) inspect]. + + sortedChanges := model changes changes asSortedCollection: + [:c1 :c2| + c1 changeClass == c2 changeClass + ifTrue: [c1 selector <= c2 selector] + ifFalse: [c2 changeClass inheritsFrom: c1 changeClass]]. + true ifTrue: + [MessageSetTextDifferencer + openMessageList: (sortedChanges collect: [:mr| { MethodReference class: mr changeClass selector: mr selector. TextReference new text: mr source class: mr changeClass selector: mr selector }]) + name: 'self foo <-> manager foo et al' + autoSelect: nil]. + + ^sortedChanges! Item was added: + ----- Method: CogScripts class>>defineSpurPigCompactor (in category 'separate vm scripts') ----- + defineSpurPigCompactor + "Define SpurPigCompactor" + "self defineSpurPigCompactor" + | changes pigCompactor | + pigCompactor := SpurMemoryManager superclass + subclass: #SpurPigCompactor + instanceVariableNames: 'manager scavenger coInterpreter firstFreeChunk lastFreeChunk numCompactionPasses' + classVariableNames: 'CompactionPassesForGC CompactionPassesForSnapshot' + poolDictionaries: 'SpurMemoryManagementConstants VMBasicConstants VMSpurObjectRepresentationConstants' + category: SpurMemoryManager category. + changes := CogScripts changedSpurCompactorMethods. + changes do: + [:mr| + pigCompactor compile: mr source classified: (SpurMemoryManager whichCategoryIncludesSelector: mr selector)]. + changes do: + [:mr| + SpurMemoryManager removeSelector: mr selector]. + SpurMemoryManager organization removeEmptyCategories!

1 0

VM Maker: VMMakerExtras-eem.2.mcz
by commits＠source.squeak.org 16 Dec '16

16 Dec '16

Eliot Miranda uploaded a new version of VMMakerExtras to project VM Maker: http://source.squeak.org/VMMaker/VMMakerExtras-eem.2.mcz ==================== Summary ==================== Name: VMMakerExtras-eem.2 Author: eem Time: 16 December 2016, 11:50:26.779082 am UUID: 571ec627-4e95-4373-922d-9c3052f78af9 Ancestors: VMMakerExtras-eem.1 Revive MessageSetDifferencer & MessageSetTextDifferencer =============== Diff against VMMakerExtras-eem.1 =============== Item was changed: ----- Method: MessageSetDifferencer>>contents (in category 'contents') ----- contents "Answer the contents of the receiver" | left right | left := leftSet selection. right := rightSet selection. + left ifNil: + [right ifNil: + [currentCompiledMethod := nil. + ^'']. + ^rightSet contents]. + right ifNil: - (left isNil and: [right isNil]) ifTrue: - [currentCompiledMethod := nil. - ^'']. - left isNil ifTrue: - [^rightSet contents]. - right isNil ifTrue: [^leftSet contents]. + ^left isMethodReference - left := leftSet contents. - right := rightSet contents. - ^leftSet selection isMethodReference ifTrue: [CodeDiffBuilder + buildDisplayPatchFrom: rightSet contents + to: leftSet contents + inClass: (left isMethodReference ifTrue: [left actualClass]) - buildDisplayPatchFrom: right - to: left - inClass: (leftSet selection isMethodReference ifTrue: [leftSet selection actualClass]) prettyDiffs: true] ifFalse: + [CodeDiffBuilder buildDisplayPatchFrom: rightSet contents to: leftSet contents]! - [CodeDiffBuilder buildDisplayPatchFrom: right to: left]! Item was changed: ----- Method: MessageSetDifferencer>>messageListMenu:shifted: (in category 'message list') ----- messageListMenu: aMenu shifted: shifted "Answer the message-list menu" + | newMenu | + (newMenu := self menuHook: aMenu named: #messageListMenu shifted: shifted) ~~ aMenu ifTrue:[^newMenu]. - (self menuHook: aMenu named: #messageListMenu shifted: shifted) ifTrue:[^aMenu]. shifted ifTrue: [^ self shiftedMessageListMenu: aMenu]. aMenu addList: #( ('browse full (b)' browseMethodFull) ('browse hierarchy (h)' classHierarchy) ('browse method (O)' openSingleMessageBrowser) ('browse protocol (p)' browseFullProtocol) - ('fileOut' fileOutMessage) ('printOut' printOutMessage) - ('senders of... (n)' browseSendersOfMessages) ('implementors of... (m)' browseMessages) ('inheritance (i)' methodHierarchy) ('versions (v)' browseVersions) - ('inst var refs...' browseInstVarRefs) ('inst var defs...' browseInstVarDefs) ('class var refs...' browseClassVarRefs) ('class variables' browseClassVariables) ('class refs (N)' browseClassRefs) - ('remove unchanged' removeUnchangedDefinitions) ('remove changed' removeChangedDefinitions) - ('more...' shiftedYellowButtonActivity)). ^ aMenu! Item was changed: ----- Method: MessageSetTextDifferencer>>contents (in category 'contents') ----- contents "Answer the contents of the receiver" | left right | left := leftSet selection. right := rightSet selection. + left ifNil: + [right ifNil: + [currentCompiledMethod := nil. + ^'']. + ^rightSet contents]. - (left isNil and: [right isNil]) ifTrue: - [currentCompiledMethod := nil. - ^'']. - left isNil ifTrue: - [^rightSet contents]. right isNil ifTrue: [^leftSet contents]. + ^TextDiffBuilder buildDisplayPatchFrom: left source to: right source! - ^TextDiffBuilder buildDisplayPatchFrom: rightSet contents to: leftSet contents! Item was added: + Object subclass: #TextReference + instanceVariableNames: 'text selector class' + classVariableNames: '' + poolDictionaries: '' + category: 'VMMakerExtras-Tools'! + + !TextReference commentStamp: '<historical>' prior: 0! + Hack to allow comparing method sources with specific texts! Item was added: + ----- Method: TextReference>>compiledMethodAt:ifAbsent: (in category 'hack puns') ----- + compiledMethodAt: selector ifAbsent: aBlock + ^nil! Item was added: + ----- Method: TextReference>>isMethodReference (in category 'testing') ----- + isMethodReference + ^true! Item was added: + ----- Method: TextReference>>organization (in category 'hack puns') ----- + organization + ^class organization! Item was added: + ----- Method: TextReference>>parserClass (in category 'hack puns') ----- + parserClass + ^class parserClass! Item was added: + ----- Method: TextReference>>setClassAndSelectorIn: (in category 'setting') ----- + setClassAndSelectorIn: csBlock + + ^csBlock value: class value: selector! Item was added: + ----- Method: TextReference>>source (in category 'accessing') ----- + source + ^text! Item was added: + ----- Method: TextReference>>sourceCode (in category 'accessing') ----- + sourceCode + ^text! Item was added: + ----- Method: TextReference>>sourceMethodAt:ifAbsent: (in category 'hack puns') ----- + sourceMethodAt: selector ifAbsent: aBlock + ^text! Item was added: + ----- Method: TextReference>>sourceString (in category 'accessing') ----- + sourceString + ^text asString! Item was added: + ----- Method: TextReference>>text:class:selector: (in category 'initialize-release') ----- + text: aText class: aBehavior selector: aSelector + text := aText. + class := aBehavior. + selector := aSelector! Item was added: + ----- Method: TextReference>>theNonMetaClass (in category 'hack puns') ----- + theNonMetaClass + ^self! Item was added: + ----- Method: TextReference>>ultimateSourceCodeAt:ifAbsent: (in category 'hack puns') ----- + ultimateSourceCodeAt: selector ifAbsent: aBlock + ^text!

1 0

← Newer
1
2
3
4
5
6
7
8
...
11
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev December 2016