Vm-dev September 2013

vm-dev@lists.squeakfoundation.org

31 participants
155 discussions

VM Maker: VMMaker.oscog-eem.417.mcz
by commits＠source.squeak.org 29 Sep '13

29 Sep '13

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.417.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.417 Author: eem Time: 29 September 2013, 12:03:30.837 pm UUID: e868c34a-af52-4474-bf0f-f4a52dafd767 Ancestors: VMMaker.oscog-eem.416 Fix ephemeron processing. Any ephemerons reached during scavenging should be copied to future space. Therefore their contents must not be scavenged if their keys are as yet unreached. So replace numStrongSlotsOf: with numStrongSlotsOf:ephemeronInactiveIf:. Further, move forwarding through ephemeron keys into isScavengeSurvivor:, and make ephemeron assert in scavengeReferentsOf: more accurate. Ephemeron must be either inactive or on the ephemeronList. Use numFixedSlotsOf: instead of numStrongSlotsOf: in processWeakSurvivor:. Fix tenuringThreshold computation, (pastSpace size * (1 - tenuringProportion) not pastSpace size * tenuringProportion. Provide some debugging conveniences. =============== Diff against VMMaker.oscog-eem.416 =============== Item was changed: ----- Method: SpurGenerationScavenger>>computeTenuringThreshold (in category 'scavenger') ----- computeTenuringThreshold | fractionSurvived | <var: 'fractionSurvived' type: #float> fractionSurvived := (futureSurvivorStart - futureSpace start) asFloat / (futureSpace limit - futureSpace start). tenuringThreshold := fractionSurvived > 0.9 + ifTrue: [((pastSpace limit - pastSpace start) * (1.0 - tenuringProportion)) rounded + pastSpace start] - ifTrue: [((pastSpace limit - pastSpace start) * tenuringProportion) rounded + pastSpace start] ifFalse: [0]! Item was added: + ----- Method: SpurGenerationScavenger>>futureSpaceObjectsDo: (in category 'debug support') ----- + futureSpaceObjectsDo: aBlock + | obj | + futureSurvivorStart > futureSpace start ifTrue: + [obj := manager objectStartingAt: futureSpace start. + [obj < futureSurvivorStart] whileTrue: + [aBlock value: obj. + obj := manager objectAfter: obj limit: futureSurvivorStart]]! Item was added: + ----- Method: SpurGenerationScavenger>>is:onWeaklingList: (in category 'weakness and ephemerality') ----- + is: oop onWeaklingList: listHead + | corpse | + corpse := self firstCorpse: listHead. + [corpse notNil] whileTrue: + [oop = (manager followForwarded: corpse) ifTrue: + [^true]. + corpse := self nextCorpseOrNil: corpse]. + ^false! Item was changed: ----- Method: SpurGenerationScavenger>>isScavengeSurvivor: (in category 'weakness and ephemerality') ----- isScavengeSurvivor: oop "Answer whether the oop has survived a scavenge. This is equivalent to + | target | + (manager isImmediate: oop) ifTrue: + [^true]. + target := (manager isForwarded: oop) + ifTrue: [manager followForwarded: oop] + ifFalse: [oop]. + ^((manager isInEden: target) + or: [(manager isInPastSpace: target)]) not" + | target | + (manager isImmediate: oop) ifTrue: + [^true]. + target := (manager isForwarded: oop) + ifTrue: [manager followForwarded: oop] + ifFalse: [oop]. + ^(manager isYoung: target) not + or: [manager isInFutureSpace: target]! - ^(manager isImmediate: oop) - or: [((manager isInEden: oop) or: [(manager isInPastSpace: oop)]) not]" - ^(manager isImmediate: oop) - or: [(manager isYoung: oop) not - or: [(manager isInFutureSpace: oop)]]! Item was changed: ----- Method: SpurGenerationScavenger>>noUnfiredEphemeronsAtEndOfRememberedSet (in category 'weakness and ephemerality') ----- noUnfiredEphemeronsAtEndOfRememberedSet "For assert checking only." numRememberedEphemerons to: rememberedSetSize - 1 do: + [:i| | referrer | - [:i| | referrer key | referrer := rememberedSet at: i. (manager isEphemeron: referrer) ifTrue: + [(self isScavengeSurvivor: (manager keyOfEphemeron: referrer)) ifFalse: - [key := manager keyOfEphemeron: referrer. - (manager isForwarded: key) ifTrue: - [key := manager followForwarded: key]. - (self isScavengeSurvivor: key) ifFalse: [^false]]]. ^true! Item was added: + ----- Method: SpurGenerationScavenger>>printWeaklingList: (in category 'debug support') ----- + printWeaklingList: listHead + "Print the objects on either the weakList or the ephemeronList." + | corpse | + corpse := self firstCorpse: listHead. + corpse ifNil: + [coInterpreter print: 'empty'; cr. + ^self]. + [corpse notNil] whileTrue: + [coInterpreter printHexnp: corpse; print: ' -> '; shortPrintOop: (manager followForwarded: corpse). + corpse := self nextCorpseOrNil: corpse]! Item was changed: ----- Method: SpurGenerationScavenger>>processWeakSurvivor: (in category 'weakness and ephemerality') ----- processWeakSurvivor: weakObj "Process a weak survivor on the weakList. Those of its fields which have not survived the scavenge should be nilled, and if any are, the coInterpreter should be informed via signalFinalization:." | weakObjShouldMourn | weakObjShouldMourn := false. + (manager numFixedSlotsOf: weakObj) - (manager numStrongSlotsOf: weakObj) to: (manager numSlotsOf: weakObj) - 1 do: [:i| | referent | referent := manager fetchPointer: i ofObject: weakObj. "Referent could be forwarded due to scavenging or a become:, don't assume." ((manager isNonImmediate: referent) and: [manager isForwarded: referent]) ifTrue: [referent := manager followForwarded: referent. (self isScavengeSurvivor: referent) ifTrue: [manager storePointer: i ofObject: weakObj withValue: referent]]. (self isScavengeSurvivor: referent) ifFalse: [weakObjShouldMourn := true. manager storePointerUnchecked: i ofObject: weakObj withValue: manager nilObject] ifTrue: [self assert: referent = (manager fetchPointer: i ofObject: weakObj)]]. weakObjShouldMourn ifTrue: [coInterpreter signalFinalization: weakObj]! Item was changed: ----- Method: SpurGenerationScavenger>>scavengeReferentsOf: (in category 'scavenger') ----- scavengeReferentsOf: referrer "scavengeReferentsOf: referrer inspects all the pointers in referrer. If any are new objects, it has them moved to FutureSurvivorSpace, and answers truth. If there are no new referents, it answers falsity. To handle weak arrays only scavenge string slots and answer true if the referrer is weak, so that it won't be removed from the remembered set until later." | foundNewReferent | "forwarding objects should be followed by callers, unless the forwarder is a root in the remembered table." self assert: ((manager isForwarded: referrer) not or: [manager isRemembered: referrer]). "unscanned ephemerons should be scanned later." self assert: ((manager isEphemeron: referrer) not + or: [(self isScavengeSurvivor: (manager keyOfEphemeron: referrer)) + or: [self is: referrer onWeaklingList: ephemeronList]]). - or: [| key | - key := manager keyOfEphemeron: referrer. - (manager isForwarded: key) ifTrue: - [key := manager followForwarded: key]. - self isScavengeSurvivor: key]). foundNewReferent := false. + 0 to: (manager numStrongSlotsOf: referrer ephemeronInactiveIf: #isScavengeSurvivor:) - 1 + do: [:i| | referent newLocation | - 0 to: (manager numStrongSlotsOf: referrer) - 1 do: - [:i| | referent newLocation | referent := manager fetchPointer: i ofMaybeForwardedObject: referrer. (manager isNonImmediate: referent) ifTrue: ["a forwarding pointer could be because of become: or scavenging." referent := (manager isForwarded: referent) ifTrue: [manager followForwarded: referent] ifFalse: [referent]. (manager isYoung: referent) ifTrue: ["if target is already in future space forwarding pointer was due to a become:." (manager isInFutureSpace: referent) ifTrue: [newLocation := referent] ifFalse: [(manager isForwarded: referent) ifTrue: [self halt. "can this even happen?" newLocation := manager followForwarded: referent] ifFalse: [newLocation := self copyAndForward: referent]]. (manager isYoung: newLocation) ifTrue: [foundNewReferent := true]. manager storePointerUnchecked: i ofMaybeForwardedObject: referrer withValue: newLocation] ifFalse: [manager storePointerUnchecked: i ofMaybeForwardedObject: referrer withValue: referent]]]. ^foundNewReferent or: [manager isWeakNonImm: referrer]! Item was changed: ----- Method: SpurGenerationScavenger>>scavengeUnfiredEphemeronsInRememberedSet (in category 'weakness and ephemerality') ----- scavengeUnfiredEphemeronsInRememberedSet "There are ephemerons to be scavenged in the remembered set. Scavenge any with unfired (live) keys, removing them from the unscavenged ephemerons, and from the set if they no longer refer to new objects, and answer if any with unfired keys were found." + | unfiredEphemeronsScavenged i referrer hasNewReferents | - | unfiredEphemeronsScavenged i referrer key hasNewReferents | unfiredEphemeronsScavenged := false. i := 0. [i < numRememberedEphemerons] whileTrue: [referrer := rememberedSet at: i. self assert: (manager isEphemeron: referrer). + (self isScavengeSurvivor: (manager keyOfEphemeron: referrer)) - key := manager keyOfEphemeron: referrer. - (manager isOopForwarded: key) ifTrue: - [key := manager followForwarded: key]. - (self isScavengeSurvivor: key) ifTrue: [unfiredEphemeronsScavenged := true. hasNewReferents := self scavengeReferentsOf: referrer. "remove from unscanned ephemerons in set by swapping with last ephemeron" numRememberedEphemerons := numRememberedEphemerons - 1. rememberedSet at: i put: (rememberedSet at: numRememberedEphemerons). hasNewReferents ifTrue: "keep in set" [rememberedSet at: numRememberedEphemerons put: referrer] ifFalse: "remove from set by overwriting with next-to-be scanned" [previousRememberedSetSize := previousRememberedSetSize - 1. rememberedSetSize := rememberedSetSize - 1. rememberedSet at: numRememberedEphemerons put: (rememberedSet at: previousRememberedSetSize); at: previousRememberedSetSize put: (rememberedSet at: rememberedSetSize). manager setIsRememberedOf: referrer to: false]] ifFalse: [i := i + 1]]. ^unfiredEphemeronsScavenged! Item was changed: ----- Method: SpurGenerationScavenger>>scavengeUnfiredEphemeronsOnEphemeronList (in category 'weakness and ephemerality') ----- scavengeUnfiredEphemeronsOnEphemeronList "There may be ephemerons to be scavenged on the ephemeronList. Scavenge any with unfired (live) keys, removing them from the list, and answer if any with unfired keys were found." | unfiredEphemeronsScavenged corpseOffset previousCorpse | ephemeronList ifNil: [^false]. unfiredEphemeronsScavenged := false. corpseOffset := ephemeronList. [corpseOffset ~= 0] whileTrue: + [| ephemeronCorpse ephemeron nextCorpseOffset | - [| ephemeronCorpse ephemeron nextCorpseOffset key | ephemeronCorpse := self corpseForCorpseOffset: corpseOffset. self assert: (manager isForwarded: ephemeronCorpse). ephemeron := manager followForwarded: ephemeronCorpse. nextCorpseOffset := self nextCorpseOffset: ephemeronCorpse. + (self isScavengeSurvivor: (manager keyOfEphemeron: ephemeron)) - key := manager keyOfEphemeron: ephemeron. - (manager isOopForwarded: key) ifTrue: - [key := manager followForwarded: key]. - (self isScavengeSurvivor: key) ifTrue: [corpseOffset = ephemeronList ifTrue: [ephemeronList := nextCorpseOffset ~= 0 ifTrue: [nextCorpseOffset]] ifFalse: [self setCorpseOffsetOf: previousCorpse to: nextCorpseOffset]. unfiredEphemeronsScavenged := true. self cCoerceSimple: (self scavengeReferentsOf: ephemeron) to: #void] ifFalse: [previousCorpse := ephemeronCorpse]. corpseOffset := nextCorpseOffset]. ^unfiredEphemeronsScavenged! Item was changed: ----- Method: SpurGenerationScavenger>>scavengerTenuringThreshold: (in category 'accessing') ----- scavengerTenuringThreshold: aProportion "(Slang flattens so need unique selectors)" <var: 'aProportion' type: #float> tenuringProportion := aProportion. tenuringThreshold := aProportion = 0.0 ifTrue: [0] + ifFalse: [((pastSpace limit - pastSpace start) * (1.0 - aProportion)) rounded + pastSpace start]! - ifFalse: [((pastSpace limit - pastSpace start) * aProportion) rounded + pastSpace start]! Item was added: + ----- Method: SpurMemoryManager>>isScavengeSurvivor: (in category 'generation scavenging') ----- + isScavengeSurvivor: oop + <doNotGenerate> + ^scavenger isScavengeSurvivor: oop! Item was added: + ----- Method: SpurMemoryManager>>numFixedSlotsOf: (in category 'object format') ----- + numFixedSlotsOf: objOop + <inline: true> + <asmLabel: false> + ^self fixedFieldsOfClassFormat: (self formatOfClass: (self fetchClassOfNonImm: objOop))! Item was added: + ----- Method: SpurMemoryManager>>numStrongSlotsOf:ephemeronInactiveIf: (in category 'object access') ----- + numStrongSlotsOf: objOop ephemeronInactiveIf: criterion + "Answer the number of strong pointer fields in the given object. + Works with CompiledMethods, as well as ordinary objects." + <api> + <var: 'criterion' declareC: 'int (*criterion)(sqInt key)'> + <inline: true> + <asmLabel: false> + | fmt numSlots contextSize numLiterals | + fmt := self formatOf: objOop. + fmt <= self lastPointerFormat ifTrue: + [numSlots := self numSlotsOf: objOop. + fmt <= self arrayFormat ifTrue: + [^numSlots]. + fmt = self indexablePointersFormat ifTrue: + [(self isContextNonImm: objOop) ifTrue: + ["contexts end at the stack pointer" + contextSize := coInterpreter fetchStackPointerOf: objOop. + ^CtxtTempFrameStart + contextSize]. + ^numSlots]. + fmt = self weakArrayFormat ifTrue: + [^self fixedFieldsOfClass: (self fetchClassOfNonImm: objOop)]. + self assert: fmt = self ephemeronFormat. + ^(self perform: criterion with: (self keyOfEphemeron: objOop)) + ifTrue: [numSlots] + ifFalse: [0]]. + fmt = self forwardedFormat ifTrue: [^1]. + fmt < self firstCompiledMethodFormat ifTrue: [^0]. "no pointers" + + "CompiledMethod: contains both pointers and bytes" + numLiterals := coInterpreter literalCountOf: objOop. + ^numLiterals + LiteralStart!

1 0

Re: [squeak-dev] Re: Float oddities
by David T. Lewis 28 Sep '13

28 Sep '13

On Fri, Sep 27, 2013 at 05:51:33PM -0400, Bob Arning wrote: > I thought I'd extend the test to an older squeak&vm > > ---interpreter vm (3.8.12) squeak 3.2 --------- > Float compareTiming: 1 to: 1.0 ==> 496 > Float compareTiming: 1.0 to: 1.0 ==> 761 > Float compareTiming: 1 to: 1 ==> 225 > Float compareTiming: 1.0 to: 1 ==> 494 > ---interpreter vm (5.7.4.1) squeak 4.5 --------- > Float compareTiming: 1 to: 1.0 ==> 719 > Float compareTiming: 1.0 to: 1.0 ==> 991 > Float compareTiming: 1 to: 1 ==> 312 > Float compareTiming: 1.0 to: 1 ==> 686 > > The old one wins. > > Cheers, > Bob (CC to vm-dev list) I get different results on my Linux PC. - Squeak 3.2 with interpreter VM compiled by Ian, circa 2006: Float compareTiming: 1 to: 1.0 ==> 1041 Float compareTiming: 1.0 to: 1.0 ==> 1432 Float compareTiming: 1 to: 1 ==> 390 Float compareTiming: 1.0 to: 1 ==> 1108 - Squeak 4.4 with Ian's 4.4.7-2357 VM, circa 2012: Float compareTiming: 1 to: 1.0 ==> 840 Float compareTiming: 1.0 to: 1.0 ==> 1176 Float compareTiming: 1 to: 1 ==> 420 Float compareTiming: 1.0 to: 1 ==> 836 - Squeak 4.4 with my locally compiled 4.12.6.2787 VM (because I cannot use Ian's 4.10.2.2614 from squeakvm.org due to a Linux glib compatibility problem for the outdated Linux on my PC): Float compareTiming: 1 to: 1.0 ==> 782 Float compareTiming: 1.0 to: 1.0 ==> 1077 Float compareTiming: 1 to: 1 ==> 461 Float compareTiming: 1.0 to: 1 ==> 789 Perhaps there are other factors on the Mac platform (compiler changes, etc) that are affecting the performance. Could you try Squeak 4.4 with Ian's unix Mac VM from squeakvm.org and see if you get different results? http://squeakvm.org/unix/release/Squeak-4.10.2.2614-darwin10.8.0_i386.tar.gz Dave > On 9/27/13 2:12 PM, Bob Arning wrote: > >Following up Herbert's question, I wrote: > > > >compareTiming: a to: b > >" > >---interpreter vm (5.7.4.1) --------- > >Float compareTiming: 1 to: 1.0 ==> 719 > >Float compareTiming: 1.0 to: 1.0 ==> 991 > >Float compareTiming: 1 to: 1 ==> 312 > >Float compareTiming: 1.0 to: 1 ==> 686 > >---cog vm (2749)--------- > >*Float compareTiming: 1 to: 1.0 ==> 28162* > >Float compareTiming: 1.0 to: 1.0 ==> 148 > >Float compareTiming: 1 to: 1 ==> 97 > >Float compareTiming: 1.0 to: 1 ==> 179 > >" > > > >^[1000000 timesRepeat: [ > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > > a < b. > >]] timeToRun > > > >So, some obvious questions: > >- in the interpreter vm, why is comparing two floats slower that an > >integer and a float? > >- what's up with Cog? Three tests are faster by the expected ratio and > >one took a real dive. > > > >Also, the profiler Herbert used could use some attention to the > >NewFloat printing: > > > >99.9 (898,069) Multiline1 doOneStep: > >*77.30000000000001* (694,902) Multiline1 [HkTestLine] doTests: > > 73.8 (663,438) HkRunin [HkTestStation] canStartNextTest: > > > >Cheers, > >Bob > > > >

1 0

VM Maker: VMMaker.oscog-eem.416.mcz
by commits＠source.squeak.org 28 Sep '13

28 Sep '13

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.416.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.416 Author: eem Time: 27 September 2013, 4:22:05.988 pm UUID: 5fb57fd2-e594-480f-b24c-1a329abd76bd Ancestors: VMMaker.oscog-eem.415 Add ephemeron support to the scavenger. As yet incorrect because my test only fires a single ephemeron and it should fire two. Refactor the corpse list machinery to allow access to offsets for list removal in scavengeUnfiredEphemeronsOnEphemeronList. Note that StackInterpreter>>fireEphemeron: is shouldBeImplemented. =============== Diff against VMMaker.oscog-eem.415 =============== Item was changed: CogClass subclass: #SpurGenerationScavenger + instanceVariableNames: 'coInterpreter manager eden futureSpace pastSpace futureSurvivorStart rememberedSet rememberedSetSize previousRememberedSetSize weakList ephemeronList tenuringThreshold tenuringProportion numRememberedEphemerons' - instanceVariableNames: 'coInterpreter manager eden futureSpace pastSpace futureSurvivorStart rememberedSet rememberedSetSize weakList ephemeronList tenuringThreshold tenuringProportion' classVariableNames: 'RememberedSetLimit RememberedSetRedZone' poolDictionaries: '' category: 'VMMaker-SpurMemoryManager'! + !SpurGenerationScavenger commentStamp: 'eem 9/27/2013 11:53' prior: 0! - !SpurGenerationScavenger commentStamp: 'eem 9/24/2013 13:04' prior: 0! SpurGenerationScavenger is an implementation of David Ungar's Generation Scavenging garbage collection algorithm. See Generation Scavenging, A Non-disruptive, High-Performance Storage Reclamation Algorithm David Ungar Proceeding SDE 1 Proceedings of the first ACM SIGSOFT/SIGPLAN software engineering symposium on Practical software development environments Pages 157 - 167 ACM New York, NY, USA ©1984 - ISBN:0-89791-131-8 Also relevant are An adaptive tenuring policy for generation scavengers David Ungar & Frank Jackson ACM Transactions on Programming Languages and Systems (TOPLAS) TOPLAS Homepage archive Volume 14 Issue 1, Jan. 1992 Pages 1 - 27 + ACM New York, NY, USA ©1992 + and + Ephemerons: a new finalization mechanism + Barry Hayes + Proceedings of the 12th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications + Pages 176-183 + ACM New York, NY, USA ©1997 - ACM New York, NY, USA Instance Variables coInterpreter: <StackInterpreterSimulator|CogVMSimulator> eden: <SpurNewSpaceSpace> futureSpace: <SpurNewSpaceSpace> futureSurvivorStart: <Integer address> manager: <SpurMemoryManager|Spur32BitMMLESimulator et al> pastSpace: <SpurNewSpaceSpace> rememberedSet: <CArrayAccessor on: Array> rememberedSetSize: <Integer> tenuringProportion: <Float> tenuringThreshold: <Integer address> coInterpreter - the interpreter/vm, in this context, the mutator eden - the space containing newly created objects futureSpace - the space to which surviving objects are copied during a scavenge futureSurvivorStart - the allocation pointer into futureSpace manager - the Spur memory manager pastSpace - the space surviving objects live in until the next scavenge rememberedSet - the root old space objects that refer to objects in new space; a scavenge starts form these roots and the interpreter's stack rememberedSetSize - the size of the remembered set, also the first unused index in the rememberedSet tenuringProportion - the amount of pastSpace below which the system will not tenure unless futureSpace fills up, and above which it will eagerly tenure tenuringThreshold - the pointer into pastSpace below which objects will be tenured ! Item was added: + ----- Method: SpurGenerationScavenger>>addToEphemeronList: (in category 'weakness and ephemerality') ----- + addToEphemeronList: ephemeronCorpse + "ephemeronCorpse is the corpse of an ephemeron that was copied and forwarded. + Later on its surviving copy must be scanned to nil weak references. + Thread the corpse onto the weakList. Later, the weakList can be followed, and + the forwarding pointer followed to locate the survivor." + | ephemeronListOffset | + self assert: (manager isYoung: ephemeronCorpse). + self assert: (manager isForwarded: ephemeronCorpse). + self assert: (self isScavengeSurvivor: (manager keyOfEphemeron: (manager followForwarded: ephemeronCorpse))) not. + + ephemeronListOffset := ephemeronList ifNil: 0. + self setCorpseOffsetOf: ephemeronCorpse to: ephemeronListOffset. + ephemeronList := ephemeronCorpse - manager startOfMemory >> manager shiftForWord. + self assert: (self firstCorpse: ephemeronList) = ephemeronCorpse! Item was changed: ----- Method: SpurGenerationScavenger>>addToWeakList: (in category 'weakness and ephemerality') ----- addToWeakList: weakCorpse "weakCorpse is the corpse of a weak array that was copied and forwarded. Later on its surviving copy must be scanned to nil weak references. Thread the corpse onto the weakList. Later, the weakList can be followed, and + the forwarding pointer followed to locate the survivor." + | weakListOffset | - the forwarding pointer followed to locate the survivor. Use the identityHash - and format fields to construct a 27 bit offset through non-future newSpace and - use this to implement the list. 27 bits is 2 ^ 27 slots, or at least 0.5Gb, big enough - for any newSpace size for the near future." - | weakListIndex | self assert: (manager isYoung: weakCorpse). + self assert: (manager isForwarded: weakCorpse). + + weakListOffset := weakList ifNil: 0. + self setCorpseOffsetOf: weakCorpse to: weakListOffset. - weakListIndex := weakList ifNil: 0. - manager - setHashBitsOf: weakCorpse - to: weakListIndex >> manager formatFieldWidthShift; - setFormatOf: weakCorpse - to: (weakListIndex bitAnd: manager formatMask). weakList := weakCorpse - manager startOfMemory >> manager shiftForWord. self assert: (self firstCorpse: weakList) = weakCorpse! Item was changed: ----- Method: SpurGenerationScavenger>>copyAndForward: (in category 'scavenger') ----- copyAndForward: survivor "copyAndForward: survivor copies a survivor object either to futureSurvivorSpace or, if it is to be promoted, to oldSpace. It leaves a forwarding pointer behind. If the object is weak then corpse is threaded onto the weakList for later treatment." <inline: true> + | bytesInObject newLocation hash | - | bytesInObject newLocation hash isWeak | bytesInObject := manager bytesInObject: survivor. + "Must remember hash before copying because threading + on to the weak & ephemeron lists smashes the hash field." - newLocation := ((self shouldBeTenured: survivor) - or: [futureSurvivorStart + bytesInObject > futureSpace limit]) - ifTrue: [self copyToOldSpace: survivor] - ifFalse: [self copyToFutureSpace: survivor bytes: bytesInObject]. hash := manager rawHashBitsOf: survivor. + ((self shouldBeTenured: survivor) + or: [futureSurvivorStart + bytesInObject > futureSpace limit]) + ifTrue: + [newLocation := self copyToOldSpace: survivor. + manager forward: survivor to: newLocation] + ifFalse: + [newLocation := self copyToFutureSpace: survivor bytes: bytesInObject. + manager forward: survivor to: newLocation. + "if weak or ephemeron add to the relevant lists if newLocation is young. If + old, newLocation will be remembered and dealt with in the rememberedSet." + (manager isWeakNonImm: newLocation) ifTrue: + [self addToWeakList: survivor]. + ((manager isEphemeron: newLocation) + and: [(self isScavengeSurvivor: (manager keyOfEphemeron: newLocation)) not]) ifTrue: + [self addToEphemeronList: survivor]]. hash ~= 0 ifTrue: [manager setHashBitsOf: newLocation to: hash]. - "Alas forward:to: smashes the format field and so destroys the weakness - property. Test it before forwarding." - isWeak := manager isWeakNonImm: survivor. - manager forward: survivor to: newLocation. - isWeak ifTrue: - [self addToWeakList: survivor]. ^newLocation! Item was added: + ----- Method: SpurGenerationScavenger>>corpseForCorpseOffset: (in category 'weakness and ephemerality') ----- + corpseForCorpseOffset: corpseOffset + "Use the identityHash and format fields to construct a 27 bit offset through + non-future newSpace and use this to implement lists of corpses for weak + array and ephemeron processing. 27 bits is 2 ^ 27 slots, or at least 0.5Gb; + big enough for any newSpace size for the near future." + ^corpseOffset << manager shiftForWord + manager startOfMemory! Item was added: + ----- Method: SpurGenerationScavenger>>fireEphemeronsInRememberedSet (in category 'weakness and ephemerality') ----- + fireEphemeronsInRememberedSet + "There are ephemerons to be fired in the remembered set. + Fire them and scavenge their keys. Leave it to scavengeLoop + to remove any scavenged ephemerons that no longer have + new referents." + | i | + self assert: self noUnfiredEphemeronsAtEndOfRememberedSet. + + i := 0. + [i < numRememberedEphemerons] whileTrue: + [ | ephemeron | + ephemeron := rememberedSet at: i. + self assert: (manager isEphemeron: ephemeron). + self assert: (self isScavengeSurvivor: (manager keyOfEphemeron: ephemeron)) not. + coInterpreter fireEphemeron: ephemeron. + self copyAndForward: (manager keyOfEphemeron: ephemeron). + (self scavengeReferentsOf: ephemeron) + ifTrue: "keep in set" + [i := i + 1] + ifFalse: + [manager setIsRememberedOf: ephemeron to: false. + "remove from set by overwriting with next-to-be scanned" + numRememberedEphemerons := numRememberedEphemerons - 1. + previousRememberedSetSize := previousRememberedSetSize - 1. + rememberedSetSize := rememberedSetSize - 1. + "First overwrite with last firable ephemeron (could be a noop if this is the last one). + Then overwrite last firable entry with next unscanned rememberedSet entry (could also be a noop). + Then overwrite next unscanned entry with last unscanned rememberedSet entry (could also be a noop)." + rememberedSet + at: i + put: (rememberedSet at: numRememberedEphemerons); + at: numRememberedEphemerons + put: (rememberedSet at: previousRememberedSetSize); + at: previousRememberedSetSize + put: (rememberedSet at: rememberedSetSize)]]. + + "no more firable ephemerons in this cycle. + scavengeRememberedSetStartingAt: may find new ones." + numRememberedEphemerons := 0! Item was added: + ----- Method: SpurGenerationScavenger>>fireEphemeronsOnEphemeronList (in category 'weakness and ephemerality') ----- + fireEphemeronsOnEphemeronList + "There are ephemerons to be fired in the remembered set. + Fire them and scavenge their keys. Be careful since copyAndForward: + can remember ephemerons (ephemerons pointing to ephemerons)." + | ephemeron ephemeronCorpse | + ephemeronList ifNil: + [^self]. + ephemeronCorpse := self firstCorpse: ephemeronList. + "Reset the list head so that new ephemerons will get added + to a new list, not concatenated on the one we are scanning." + ephemeronList := nil. + [ephemeronCorpse notNil] whileTrue: + [self assert: (manager isForwarded: ephemeronCorpse). + ephemeron := manager followForwarded: ephemeronCorpse. + self assert: (self isScavengeSurvivor: (manager keyOfEphemeron: ephemeron)) not.. + coInterpreter fireEphemeron: ephemeron. + self copyAndForward: (manager keyOfEphemeron: ephemeron). + self cCoerceSimple: (self scavengeReferentsOf: ephemeron) to: #void. + ephemeronCorpse := self nextCorpseOrNil: ephemeronCorpse]! Item was changed: ----- Method: SpurGenerationScavenger>>firstCorpse: (in category 'weakness and ephemerality') ----- firstCorpse: headOfCorpseList "Use the identityHash and format fields to construct a 27 bit offset through non-future newSpace and use this to implement lists of corpses for weak array and ephemeron processing. 27 bits is 2 ^ 27 slots, or at least 0.5Gb; big enough for any newSpace size for the near future." + ^self corpseForCorpseOffset: headOfCorpseList! - ^headOfCorpseList << manager shiftForWord + manager startOfMemory! Item was added: + ----- Method: SpurGenerationScavenger>>nextCorpseOffset: (in category 'weakness and ephemerality') ----- + nextCorpseOffset: corpse + "Answer the offset of the next corpse to corpse, which is zero if none. + Use the identityHash and format fields to construct a 27 bit offset through + non-future newSpace and use this to implement the list. 27 bits is 2 ^ 27 slots, + or at least 0.5Gb, big enough for any newSpace size for the near future." + ^(manager rawHashBitsOf: corpse) << manager formatFieldWidthShift + + (manager formatOf: corpse)! Item was changed: ----- Method: SpurGenerationScavenger>>nextCorpseOrNil: (in category 'weakness and ephemerality') ----- nextCorpseOrNil: weakCorpse "weakCorpse is the corpse of a weak array that has been added to the weakList. + Answer the next object on the list, or nil if none." + | weakListOffset | - Answer the next object on the list, or nil if none. - Use the identityHash and format fields to construct a 27 bit offset through - non-future newSpace and use this to implement the list. 27 bits is 2 ^ 27 slots, - or at least 0.5Gb, big enough for any newSpace size for the near future." - | weakListIndex | self assert: (manager isYoung: weakCorpse). + weakListOffset := self nextCorpseOffset: weakCorpse. + ^weakListOffset ~= 0 ifTrue: + [weakListOffset << manager shiftForWord + manager startOfMemory]! - weakListIndex := (manager rawHashBitsOf: weakCorpse) << manager formatFieldWidthShift - + (manager formatOf: weakCorpse). - ^weakListIndex ~= 0 ifTrue: - [weakListIndex << manager shiftForWord + manager startOfMemory]! Item was added: + ----- Method: SpurGenerationScavenger>>noUnfiredEphemeronsAtEndOfRememberedSet (in category 'weakness and ephemerality') ----- + noUnfiredEphemeronsAtEndOfRememberedSet + "For assert checking only." + numRememberedEphemerons to: rememberedSetSize - 1 do: + [:i| | referrer key | + referrer := rememberedSet at: i. + (manager isEphemeron: referrer) ifTrue: + [key := manager keyOfEphemeron: referrer. + (manager isForwarded: key) ifTrue: + [key := manager followForwarded: key]. + (self isScavengeSurvivor: key) ifFalse: + [^false]]]. + ^true! Item was removed: - ----- Method: SpurGenerationScavenger>>processEphemeronList (in category 'weakness and ephemerality') ----- - processEphemeronList - "There are ephemerons to be scavenged. Scavenge them - and fire any whose keys are still in pastSpace and/or eden." - ^self shouldBeImplemented! Item was added: + ----- Method: SpurGenerationScavenger>>processEphemerons (in category 'weakness and ephemerality') ----- + processEphemerons + "There are ephemerons to be scavenged. Scavenge them + and fire any whose keys are still in pastSpace and/or eden. + The ephemerons in this cycle can only be fired if all the + ephemerons in this cycle are firable, because references + to ephemeron keys from unfired ephemerons should prevent + the ephemerons with those keys from firing. So scavenge + ephemerons with surviving keys, and only if none are found, + fire ephemerons with unreferenced keys, and scavenge them." + | unfiredEphemeronsScavenged | + unfiredEphemeronsScavenged := self scavengeUnfiredEphemeronsInRememberedSet. + self scavengeUnfiredEphemeronsOnEphemeronList ifTrue: + [unfiredEphemeronsScavenged := true]. + "If no unfired ephemerons were scavenged, then all ephemerons in this cycle can be fired." + unfiredEphemeronsScavenged ifFalse: + [self fireEphemeronsInRememberedSet. + self fireEphemeronsOnEphemeronList]! Item was changed: ----- Method: SpurGenerationScavenger>>scavengeLoop (in category 'scavenger') ----- scavengeLoop "This is the inner loop of the main routine, scavenge. It first scavenges the new objects immediately reachable from old ones. Then it scavenges those that are transitively reachable. If this results in a promotion, the promotee gets remembered, and it first scavenges objects adjacent to the promotee, then scavenges the ones reachable from the promoted. This loop continues until no more reachable objects are left. At that point, pastSurvivorSpace is exchanged with futureSurvivorSpace. Notice that each pointer in a live object is inspected once and only once. The previousRememberedSetSize and previousFutureSurvivorSpaceLimit variables ensure that no object is scanned twice, as well as detecting closure. If this were not true, some pointers might get forwarded twice. An extension of the algorithm presented in David's original paper is to handle weak arrays and ephemerons. Weak arrays should not have their weak referents scavenged unless there are strong references to them. Ephemerons should fire if their key is not reachable other than from ephemerons and weak arrays. Handle this by maintaining a list for weak arrays and a list for ephemerons, which allow scavenging these objects once all other objects in new space have been scavenged, hence allowing the scavenger to detect which referents in new space of weak arrays are dead and of ephemeron keys are only live due to ephemerons." + | previousFutureSurvivorStart firstTime | - | firstTime previousRememberedSetSize previousFutureSurvivorStart | self assert: futureSurvivorStart = futureSpace start. "future space should be empty at the start" + weakList := ephemeronList := nil. + numRememberedEphemerons := 0. firstTime := true. previousRememberedSetSize := 0. previousFutureSurvivorStart := futureSurvivorStart. + [self scavengeRememberedSetStartingAt: previousRememberedSetSize. previousRememberedSetSize := rememberedSetSize. firstTime ifTrue: [coInterpreter mapInterpreterOops. firstTime := false]. + "if nothing more copied and forwarded (or remembered by mapInterpreterOops) + to scavenge, and no ephemerons to process, scavenge is done." - "nothing more copied and forwarded (or remembered by mapInterpreterOops) - to scavenge so scavenge is done." (previousRememberedSetSize = rememberedSetSize and: [previousFutureSurvivorStart = futureSurvivorStart + and: [numRememberedEphemerons = 0 + and: [ephemeronList isNil]]]) ifTrue: - and: [ephemeronList isNil]]) ifTrue: [^self]. self scavengeFutureSurvivorSpaceStartingAt: previousFutureSurvivorStart. + "if no more roots created to scavenge, and no ephemerons to process, scavenge is done." - "no more roots created to scavenge, so scavenge is done." (previousRememberedSetSize = rememberedSetSize + and: [numRememberedEphemerons = 0 + and: [ephemeronList isNil]]) ifTrue: - and: [ephemeronList isNil]) ifTrue: [^self]. previousFutureSurvivorStart := futureSurvivorStart. + (numRememberedEphemerons > 0 + or: [ephemeronList notNil]) ifTrue: + [self processEphemerons]] repeat! - ephemeronList notNil ifTrue: - [self processEphemeronList]] repeat! Item was changed: ----- Method: SpurGenerationScavenger>>scavengeReferentsOf: (in category 'scavenger') ----- scavengeReferentsOf: referrer "scavengeReferentsOf: referrer inspects all the pointers in referrer. If any are new objects, it has them moved to FutureSurvivorSpace, and answers truth. If there are no new referents, it answers falsity. To handle weak arrays only scavenge string slots and answer true if the referrer is weak, so that it won't be removed from the remembered set until later." | foundNewReferent | "forwarding objects should be followed by callers, unless the forwarder is a root in the remembered table." self assert: ((manager isForwarded: referrer) not or: [manager isRemembered: referrer]). + "unscanned ephemerons should be scanned later." + self assert: ((manager isEphemeron: referrer) not + or: [| key | + key := manager keyOfEphemeron: referrer. + (manager isForwarded: key) ifTrue: + [key := manager followForwarded: key]. + self isScavengeSurvivor: key]). foundNewReferent := false. 0 to: (manager numStrongSlotsOf: referrer) - 1 do: [:i| | referent newLocation | referent := manager fetchPointer: i ofMaybeForwardedObject: referrer. (manager isNonImmediate: referent) ifTrue: ["a forwarding pointer could be because of become: or scavenging." referent := (manager isForwarded: referent) ifTrue: [manager followForwarded: referent] ifFalse: [referent]. (manager isYoung: referent) ifTrue: ["if target is already in future space forwarding pointer was due to a become:." (manager isInFutureSpace: referent) ifTrue: [newLocation := referent] ifFalse: [(manager isForwarded: referent) ifTrue: [self halt. "can this even happen?" newLocation := manager followForwarded: referent] ifFalse: [newLocation := self copyAndForward: referent]]. (manager isYoung: newLocation) ifTrue: [foundNewReferent := true]. manager storePointerUnchecked: i ofMaybeForwardedObject: referrer withValue: newLocation] ifFalse: [manager storePointerUnchecked: i ofMaybeForwardedObject: referrer withValue: referent]]]. ^foundNewReferent or: [manager isWeakNonImm: referrer]! Item was changed: ----- Method: SpurGenerationScavenger>>scavengeRememberedSetStartingAt: (in category 'scavenger') ----- scavengeRememberedSetStartingAt: n "scavengeRememberedSetStartingAt: n traverses objects in the remembered set starting at the nth one. If the object does not refer to any new objects, it + is removed from the set. Otherwise, its new referents are scavenged. Defer + scavenging ephemerons until after a complete scavenge has been performed, + so that triggered ephemerons can be fired. Move them to the front of the set + and count them in numRememberedEphemerons for later scanning." + | destIndex sourceIndex referrer | - is removed from the set. Otherwise, its new referents are scavenged." - | destIndex sourceIndex | sourceIndex := destIndex := n. [sourceIndex < rememberedSetSize] whileTrue: + ["*Don't* follow forwarding pointers here. oldSpace objects may refer + to these roots, and so they can't be removed in the scavenge." + referrer := rememberedSet at: sourceIndex. + "Any potential firing ephemerons should not be scanned yet. + Move any to the front of the set to save time in later scanning." + ((manager isEphemeron: referrer) + and: [(self isScavengeSurvivor: ((manager keyOfEphemeron: referrer))) not]) - [| referree | - "*Don't* follow forwarding pointers here. oldSpace objects may refer - to these roots, and so they can't be removed in the scavenge." - referree := rememberedSet at: sourceIndex. - (self scavengeReferentsOf: referree) ifTrue: + [self assert: destIndex >= numRememberedEphemerons. + rememberedSet + at: destIndex put: (rememberedSet at: numRememberedEphemerons); + at: numRememberedEphemerons put: referrer. + numRememberedEphemerons := numRememberedEphemerons + 1. - [rememberedSet at: destIndex put: referree. destIndex := destIndex + 1] ifFalse: + [(self scavengeReferentsOf: referrer) + ifTrue: + [rememberedSet at: destIndex put: referrer. + destIndex := destIndex + 1] + ifFalse: + [manager setIsRememberedOf: referrer to: false]]. - [manager setIsRememberedOf: referree to: false]. sourceIndex := sourceIndex + 1]. + rememberedSetSize := destIndex. + self assert: self noUnfiredEphemeronsAtEndOfRememberedSet! - rememberedSetSize := destIndex! Item was added: + ----- Method: SpurGenerationScavenger>>scavengeUnfiredEphemeronsInRememberedSet (in category 'weakness and ephemerality') ----- + scavengeUnfiredEphemeronsInRememberedSet + "There are ephemerons to be scavenged in the remembered set. + Scavenge any with unfired (live) keys, removing them from the + unscavenged ephemerons, and from the set if they no longer refer + to new objects, and answer if any with unfired keys were found." + | unfiredEphemeronsScavenged i referrer key hasNewReferents | + unfiredEphemeronsScavenged := false. + i := 0. + [i < numRememberedEphemerons] whileTrue: + [referrer := rememberedSet at: i. + self assert: (manager isEphemeron: referrer). + key := manager keyOfEphemeron: referrer. + (manager isOopForwarded: key) ifTrue: + [key := manager followForwarded: key]. + (self isScavengeSurvivor: key) + ifTrue: + [unfiredEphemeronsScavenged := true. + hasNewReferents := self scavengeReferentsOf: referrer. + "remove from unscanned ephemerons in set by swapping with last ephemeron" + numRememberedEphemerons := numRememberedEphemerons - 1. + rememberedSet + at: i + put: (rememberedSet at: numRememberedEphemerons). + hasNewReferents + ifTrue: "keep in set" + [rememberedSet + at: numRememberedEphemerons + put: referrer] + ifFalse: "remove from set by overwriting with next-to-be scanned" + [previousRememberedSetSize := previousRememberedSetSize - 1. + rememberedSetSize := rememberedSetSize - 1. + rememberedSet + at: numRememberedEphemerons + put: (rememberedSet at: previousRememberedSetSize); + at: previousRememberedSetSize + put: (rememberedSet at: rememberedSetSize). + manager setIsRememberedOf: referrer to: false]] + ifFalse: + [i := i + 1]]. + ^unfiredEphemeronsScavenged! Item was added: + ----- Method: SpurGenerationScavenger>>scavengeUnfiredEphemeronsOnEphemeronList (in category 'weakness and ephemerality') ----- + scavengeUnfiredEphemeronsOnEphemeronList + "There may be ephemerons to be scavenged on the ephemeronList. + Scavenge any with unfired (live) keys, removing them from the + list, and answer if any with unfired keys were found." + | unfiredEphemeronsScavenged corpseOffset previousCorpse | + ephemeronList ifNil: + [^false]. + unfiredEphemeronsScavenged := false. + corpseOffset := ephemeronList. + [corpseOffset ~= 0] whileTrue: + [| ephemeronCorpse ephemeron nextCorpseOffset key | + ephemeronCorpse := self corpseForCorpseOffset: corpseOffset. + self assert: (manager isForwarded: ephemeronCorpse). + ephemeron := manager followForwarded: ephemeronCorpse. + nextCorpseOffset := self nextCorpseOffset: ephemeronCorpse. + key := manager keyOfEphemeron: ephemeron. + (manager isOopForwarded: key) ifTrue: + [key := manager followForwarded: key]. + (self isScavengeSurvivor: key) + ifTrue: + [corpseOffset = ephemeronList + ifTrue: [ephemeronList := nextCorpseOffset ~= 0 ifTrue: [nextCorpseOffset]] + ifFalse: [self setCorpseOffsetOf: previousCorpse to: nextCorpseOffset]. + unfiredEphemeronsScavenged := true. + self cCoerceSimple: (self scavengeReferentsOf: ephemeron) to: #void] + ifFalse: + [previousCorpse := ephemeronCorpse]. + corpseOffset := nextCorpseOffset]. + ^unfiredEphemeronsScavenged! Item was added: + ----- Method: SpurGenerationScavenger>>setCorpseOffsetOf:to: (in category 'weakness and ephemerality') ----- + setCorpseOffsetOf: corpse to: offset + "Set the offset of the corpse's next corpse to offset. Use the identityHash + and format fields to construct a 27 bit offset through non-future newSpace and + use this to implement the list. 27 bits is 2 ^ 27 slots, or at least 0.5Gb, big enough + for any newSpace size for the near future." + manager + setHashBitsOf: corpse + to: offset >> manager formatFieldWidthShift; + setFormatOf: corpse + to: (offset bitAnd: manager formatMask)! Item was added: + ----- Method: SpurMemoryManager>>isEphemeron: (in category 'object testing') ----- + isEphemeron: objOop + self assert: (self isNonImmediate: objOop). + ^(self formatOf: objOop) = self ephemeronFormat! Item was added: + ----- Method: SpurMemoryManager>>keyOfEphemeron: (in category 'object access') ----- + keyOfEphemeron: objOop + "Answer the object the ephemeron guards. This is its first element." + self assert: ((self isNonImmediate: objOop) and: [self isEphemeron: objOop]). + ^self fetchPointer: 0 ofObject: objOop! Item was added: + ----- Method: StackInterpreter>>fireEphemeron: (in category 'finalization') ----- + fireEphemeron: ephemeron + self shouldBeImplemented!

1 0

Re: [Pharo-dev] VM Serial plugin
by Luc Fabresse 27 Sep '13

27 Sep '13

Hi all, Here is the description of the issue: https://pharo.fogbugz.com/f/cases/edit/11698 This fix is really simple ;-) As I said, the code is on github: https://github.com/pharo-project/pharo-vm @Eliot, please find the .cs attached if it is easier for you. I did not change the platform code and this one is used: https://github.com/pharo-project/pharo-vm/blob/master/platforms/unix/plugin… Bye, #Luc 2013/9/27 Camillo Bruni <camillobruni(a)gmail.com> > > On 2013-09-27, at 12:18, Luc Fabresse <luc.fabresse(a)gmail.com> wrote: > > > Hi all, > > > > With Santiago, we made a small fix for the serial plugin on Linux and > OsX. > > We worked on a cloned repository of the PharoVM on github. > > I just sent a pull request. > > nice > > > I guess I should also fill a bug entry somewhere. > > on which bugtracker, the pharo one ? > > On http://pharo.fogbugz.com and you can categporize it under the PharoVM > project. >

1 0

VM Maker: VMMaker.oscog-eem.415.mcz
by commits＠source.squeak.org 27 Sep '13

27 Sep '13

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.415.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.415 Author: eem Time: 26 September 2013, 5:20:16.794 pm UUID: 73fb10dc-25ce-427c-b0ad-976524bc2941 Ancestors: VMMaker.oscog-eem.414 Implement weakArray processing in the scavenger (plus a skeleton for ephemeron processing). =============== Diff against VMMaker.oscog-eem.414 =============== Item was changed: CogClass subclass: #SpurGenerationScavenger + instanceVariableNames: 'coInterpreter manager eden futureSpace pastSpace futureSurvivorStart rememberedSet rememberedSetSize weakList ephemeronList tenuringThreshold tenuringProportion' - instanceVariableNames: 'coInterpreter manager eden futureSpace pastSpace futureSurvivorStart rememberedSet rememberedSetSize tenuringThreshold tenuringProportion' classVariableNames: 'RememberedSetLimit RememberedSetRedZone' poolDictionaries: '' category: 'VMMaker-SpurMemoryManager'! !SpurGenerationScavenger commentStamp: 'eem 9/24/2013 13:04' prior: 0! SpurGenerationScavenger is an implementation of David Ungar's Generation Scavenging garbage collection algorithm. See Generation Scavenging, A Non-disruptive, High-Performance Storage Reclamation Algorithm David Ungar Proceeding SDE 1 Proceedings of the first ACM SIGSOFT/SIGPLAN software engineering symposium on Practical software development environments Pages 157 - 167 ACM New York, NY, USA ©1984 ISBN:0-89791-131-8 Also relevant are An adaptive tenuring policy for generation scavengers David Ungar & Frank Jackson ACM Transactions on Programming Languages and Systems (TOPLAS) TOPLAS Homepage archive Volume 14 Issue 1, Jan. 1992 Pages 1 - 27 ACM New York, NY, USA Instance Variables coInterpreter: <StackInterpreterSimulator|CogVMSimulator> eden: <SpurNewSpaceSpace> futureSpace: <SpurNewSpaceSpace> futureSurvivorStart: <Integer address> manager: <SpurMemoryManager|Spur32BitMMLESimulator et al> pastSpace: <SpurNewSpaceSpace> rememberedSet: <CArrayAccessor on: Array> rememberedSetSize: <Integer> tenuringProportion: <Float> tenuringThreshold: <Integer address> coInterpreter - the interpreter/vm, in this context, the mutator eden - the space containing newly created objects futureSpace - the space to which surviving objects are copied during a scavenge futureSurvivorStart - the allocation pointer into futureSpace manager - the Spur memory manager pastSpace - the space surviving objects live in until the next scavenge rememberedSet - the root old space objects that refer to objects in new space; a scavenge starts form these roots and the interpreter's stack rememberedSetSize - the size of the remembered set, also the first unused index in the rememberedSet tenuringProportion - the amount of pastSpace below which the system will not tenure unless futureSpace fills up, and above which it will eagerly tenure tenuringThreshold - the pointer into pastSpace below which objects will be tenured ! Item was added: + ----- Method: SpurGenerationScavenger>>addToWeakList: (in category 'weakness and ephemerality') ----- + addToWeakList: weakCorpse + "weakCorpse is the corpse of a weak array that was copied and forwarded. + Later on its surviving copy must be scanned to nil weak references. + Thread the corpse onto the weakList. Later, the weakList can be followed, and + the forwarding pointer followed to locate the survivor. Use the identityHash + and format fields to construct a 27 bit offset through non-future newSpace and + use this to implement the list. 27 bits is 2 ^ 27 slots, or at least 0.5Gb, big enough + for any newSpace size for the near future." + | weakListIndex | + self assert: (manager isYoung: weakCorpse). + weakListIndex := weakList ifNil: 0. + manager + setHashBitsOf: weakCorpse + to: weakListIndex >> manager formatFieldWidthShift; + setFormatOf: weakCorpse + to: (weakListIndex bitAnd: manager formatMask). + weakList := weakCorpse - manager startOfMemory >> manager shiftForWord. + self assert: (self firstCorpse: weakList) = weakCorpse! Item was changed: ----- Method: SpurGenerationScavenger>>copyAndForward: (in category 'scavenger') ----- copyAndForward: survivor "copyAndForward: survivor copies a survivor object either to futureSurvivorSpace or, if it is to be promoted, to oldSpace. + It leaves a forwarding pointer behind. If the object is weak + then corpse is threaded onto the weakList for later treatment." - It leaves a forwarding pointer behind." <inline: true> + | bytesInObject newLocation hash isWeak | - | bytesInObject newLocation hash | bytesInObject := manager bytesInObject: survivor. newLocation := ((self shouldBeTenured: survivor) or: [futureSurvivorStart + bytesInObject > futureSpace limit]) ifTrue: [self copyToOldSpace: survivor] ifFalse: [self copyToFutureSpace: survivor bytes: bytesInObject]. hash := manager rawHashBitsOf: survivor. hash ~= 0 ifTrue: [manager setHashBitsOf: newLocation to: hash]. + "Alas forward:to: smashes the format field and so destroys the weakness + property. Test it before forwarding." + isWeak := manager isWeakNonImm: survivor. manager forward: survivor to: newLocation. + isWeak ifTrue: + [self addToWeakList: survivor]. ^newLocation! Item was added: + ----- Method: SpurGenerationScavenger>>firstCorpse: (in category 'weakness and ephemerality') ----- + firstCorpse: headOfCorpseList + "Use the identityHash and format fields to construct a 27 bit offset through + non-future newSpace and use this to implement lists of corpses for weak + array and ephemeron processing. 27 bits is 2 ^ 27 slots, or at least 0.5Gb; + big enough for any newSpace size for the near future." + ^headOfCorpseList << manager shiftForWord + manager startOfMemory! Item was added: + ----- Method: SpurGenerationScavenger>>isScavengeSurvivor: (in category 'weakness and ephemerality') ----- + isScavengeSurvivor: oop + "Answer whether the oop has survived a scavenge. This is equivalent to + ^(manager isImmediate: oop) + or: [((manager isInEden: oop) or: [(manager isInPastSpace: oop)]) not]" + ^(manager isImmediate: oop) + or: [(manager isYoung: oop) not + or: [(manager isInFutureSpace: oop)]]! Item was added: + ----- Method: SpurGenerationScavenger>>nextCorpseOrNil: (in category 'weakness and ephemerality') ----- + nextCorpseOrNil: weakCorpse + "weakCorpse is the corpse of a weak array that has been added to the weakList. + Answer the next object on the list, or nil if none. + Use the identityHash and format fields to construct a 27 bit offset through + non-future newSpace and use this to implement the list. 27 bits is 2 ^ 27 slots, + or at least 0.5Gb, big enough for any newSpace size for the near future." + | weakListIndex | + self assert: (manager isYoung: weakCorpse). + weakListIndex := (manager rawHashBitsOf: weakCorpse) << manager formatFieldWidthShift + + (manager formatOf: weakCorpse). + ^weakListIndex ~= 0 ifTrue: + [weakListIndex << manager shiftForWord + manager startOfMemory]! Item was added: + ----- Method: SpurGenerationScavenger>>processEphemeronList (in category 'weakness and ephemerality') ----- + processEphemeronList + "There are ephemerons to be scavenged. Scavenge them + and fire any whose keys are still in pastSpace and/or eden." + ^self shouldBeImplemented! Item was added: + ----- Method: SpurGenerationScavenger>>processWeakSurvivor: (in category 'weakness and ephemerality') ----- + processWeakSurvivor: weakObj + "Process a weak survivor on the weakList. Those of its fields + which have not survived the scavenge should be nilled, and if any + are, the coInterpreter should be informed via signalFinalization:." + | weakObjShouldMourn | + weakObjShouldMourn := false. + (manager numStrongSlotsOf: weakObj) + to: (manager numSlotsOf: weakObj) - 1 + do: [:i| | referent | + referent := manager fetchPointer: i ofObject: weakObj. + "Referent could be forwarded due to scavenging or a become:, don't assume." + ((manager isNonImmediate: referent) + and: [manager isForwarded: referent]) ifTrue: + [referent := manager followForwarded: referent. + (self isScavengeSurvivor: referent) ifTrue: + [manager storePointer: i ofObject: weakObj withValue: referent]]. + (self isScavengeSurvivor: referent) + ifFalse: + [weakObjShouldMourn := true. + manager + storePointerUnchecked: i + ofObject: weakObj + withValue: manager nilObject] + ifTrue: + [self assert: referent = (manager fetchPointer: i ofObject: weakObj)]]. + weakObjShouldMourn ifTrue: + [coInterpreter signalFinalization: weakObj]! Item was added: + ----- Method: SpurGenerationScavenger>>processWeaklings (in category 'weakness and ephemerality') ----- + processWeaklings + "Go through the remembered set and the weak list, nilling + references to any objects that didn't survive the scavenge." + | i rootObj weakCorpse weakObj | + i := 0. + [i < rememberedSetSize] whileTrue: + [rootObj := rememberedSet at: i. + (manager isWeakNonImm: rootObj) + ifTrue: + [self processWeakSurvivor: rootObj. + "If no more referents, remove by overwriting with the last element in the set." + (manager hasYoungReferents: rootObj) + ifFalse: + [manager setIsRememberedOf: rootObj to: false. + i + 1 < rememberedSetSize ifTrue: + [rememberedSet at: i put: (rememberedSet at: rememberedSetSize - 1)]. + rememberedSetSize := rememberedSetSize - 1] + ifTrue: [i := i + 1]] + ifFalse: [i := i + 1]]. + weakList ifNotNil: + [weakCorpse := self firstCorpse: weakList. + [weakCorpse notNil] whileTrue: + [self assert: (manager isForwarded: weakCorpse). + weakObj := manager followForwarded: weakCorpse. + self processWeakSurvivor: weakObj. + weakCorpse := self nextCorpseOrNil: weakCorpse]. + weakList := nil]! Item was changed: ----- Method: SpurGenerationScavenger>>scavenge (in category 'scavenger') ----- scavenge "The main routine, scavenge, scavenges young objects reachable from the roots (the stack zone and the rememberedTable). It first scavenges the new objects immediately reachable from the stack zone, then those directly from old ones (all in the remembered table). Then it scavenges those that are transitively reachable. If this results in a promotion, the promotee gets remembered, and it first scavenges objects adjacent to the promotee, then scavenges the ones reachable from the promoted. This loop continues until no more reachable objects are left. At that point, pastSurvivorSpace is exchanged with futureSurvivorSpace. Notice that each pointer in a live object is inspected once and only once. The previousRememberedSetSize and previousFutureSurvivorSpaceSize variables ensure that no object is scanned twice, as well as detecting closure. If this were not true, some pointers might get forwarded twice. Answer the limit of pastSpace, to allow the memory manager to bounds check survivors." self scavengeLoop. + self processWeaklings. self computeTenuringThreshold. self exchangeSurvivorSpaces. ^self initFutureSpaceStart! Item was changed: ----- Method: SpurGenerationScavenger>>scavengeLoop (in category 'scavenger') ----- scavengeLoop "This is the inner loop of the main routine, scavenge. It first scavenges the new objects immediately reachable from old ones. Then it scavenges those that are transitively reachable. If this results in a promotion, the promotee gets remembered, and it first scavenges objects adjacent to the promotee, then scavenges the ones reachable from the promoted. This loop continues until no more reachable objects are left. At that point, pastSurvivorSpace is exchanged with futureSurvivorSpace. Notice that each pointer in a live object is inspected once and only once. The previousRememberedSetSize and previousFutureSurvivorSpaceLimit variables ensure that no object is scanned twice, as well as + detecting closure. If this were not true, some pointers might get forwarded twice. - detecting closure. If this were not true, some pointers might get forwarded twice." + An extension of the algorithm presented in David's original paper is to handle weak arrays and ephemerons. + Weak arrays should not have their weak referents scavenged unless there are strong references to them. + Ephemerons should fire if their key is not reachable other than from ephemerons and weak arrays. + Handle this by maintaining a list for weak arrays and a list for ephemerons, which allow scavenging these + objects once all other objects in new space have been scavenged, hence allowing the scavenger to + detect which referents in new space of weak arrays are dead and of ephemeron keys are only live due to + ephemerons." + | firstTime previousRememberedSetSize previousFutureSurvivorStart | self assert: futureSurvivorStart = futureSpace start. "future space should be empty at the start" + weakList := ephemeronList := nil. firstTime := true. previousRememberedSetSize := 0. previousFutureSurvivorStart := futureSurvivorStart. [self scavengeRememberedSetStartingAt: previousRememberedSetSize. previousRememberedSetSize := rememberedSetSize. firstTime ifTrue: [coInterpreter mapInterpreterOops. firstTime := false]. "nothing more copied and forwarded (or remembered by mapInterpreterOops) to scavenge so scavenge is done." (previousRememberedSetSize = rememberedSetSize + and: [previousFutureSurvivorStart = futureSurvivorStart + and: [ephemeronList isNil]]) ifTrue: - and: [previousFutureSurvivorStart = futureSurvivorStart]) ifTrue: [^self]. self scavengeFutureSurvivorSpaceStartingAt: previousFutureSurvivorStart. "no more roots created to scavenge, so scavenge is done." + (previousRememberedSetSize = rememberedSetSize + and: [ephemeronList isNil]) ifTrue: - previousRememberedSetSize = rememberedSetSize ifTrue: [^self]. + previousFutureSurvivorStart := futureSurvivorStart. + ephemeronList notNil ifTrue: + [self processEphemeronList]] repeat! - previousFutureSurvivorStart := futureSurvivorStart] repeat! Item was changed: ----- Method: SpurGenerationScavenger>>scavengeReferentsOf: (in category 'scavenger') ----- scavengeReferentsOf: referrer "scavengeReferentsOf: referrer inspects all the pointers in referrer. If any are new objects, it has them moved to FutureSurvivorSpace, + and answers truth. If there are no new referents, it answers falsity. + To handle weak arrays only scavenge string slots and answer true + if the referrer is weak, so that it won't be removed from the + remembered set until later." - and answers truth. If there are no new referents, it answers falsity." | foundNewReferent | "forwarding objects should be followed by callers, unless the forwarder is a root in the remembered table." self assert: ((manager isForwarded: referrer) not or: [manager isRemembered: referrer]). foundNewReferent := false. + 0 to: (manager numStrongSlotsOf: referrer) - 1 do: - 0 to: (manager numPointerSlotsOf: referrer) - 1 do: [:i| | referent newLocation | referent := manager fetchPointer: i ofMaybeForwardedObject: referrer. (manager isNonImmediate: referent) ifTrue: ["a forwarding pointer could be because of become: or scavenging." referent := (manager isForwarded: referent) ifTrue: [manager followForwarded: referent] ifFalse: [referent]. (manager isYoung: referent) ifTrue: ["if target is already in future space forwarding pointer was due to a become:." (manager isInFutureSpace: referent) ifTrue: [newLocation := referent] ifFalse: [(manager isForwarded: referent) ifTrue: [self halt. "can this even happen?" newLocation := manager followForwarded: referent] ifFalse: [newLocation := self copyAndForward: referent]]. (manager isYoung: newLocation) ifTrue: [foundNewReferent := true]. manager storePointerUnchecked: i ofMaybeForwardedObject: referrer withValue: newLocation] ifFalse: [manager storePointerUnchecked: i ofMaybeForwardedObject: referrer withValue: referent]]]. + ^foundNewReferent or: [manager isWeakNonImm: referrer]! - ^foundNewReferent! Item was changed: ----- Method: SpurGenerationScavengerSimulator>>copyAndForward: (in category 'scavenger') ----- copyAndForward: survivor | newLocation | + "(#(16r167280 16r19A670) includes: survivor) ifTrue: [self halt]." true ifTrue: [^super copyAndForward: survivor.]. - "(#(16r13BC78 16r13BD68 16r1ED780 16r1FC558) includes: survivor) ifTrue: [self halt]." newLocation := super copyAndForward: survivor. comeFroms at: newLocation put: survivor. "((manager isContextNonImm: newLocation) and: [#(16r11D6988 16r11D6A48 16r11D6AC0 16r11D6B80) includes: newLocation]) ifTrue: [self halt]." ^newLocation! Item was added: + ----- Method: SpurMemoryManager>>fixedFieldsOfClass: (in category 'object format') ----- + fixedFieldsOfClass: objOop + ^self fixedFieldsOfClassFormat: (self formatOfClass: objOop)! Item was added: + ----- Method: SpurMemoryManager>>formatFieldWidthShift (in category 'header format') ----- + formatFieldWidthShift + "The format field contains 5 bits." + ^5! Item was added: + ----- Method: SpurMemoryManager>>hasYoungReferents: (in category 'object testing') ----- + hasYoungReferents: objOop + 0 to: (self numPointerSlotsOf: objOop) - 1 do: + [:i| | oop | + oop := self fetchPointer: i ofObject: objOop. + ((self isNonImmediate: oop) + and: [self isYoung: oop]) ifTrue: + [^true]]. + ^false! Item was added: + ----- Method: SpurMemoryManager>>isWeakNonImm: (in category 'object testing') ----- + isWeakNonImm: objOop + ^(self formatOf: objOop) = self weakArrayFormat! Item was changed: + ----- Method: SpurMemoryManager>>numPointerSlotsOf: (in category 'object access') ----- - ----- Method: SpurMemoryManager>>numPointerSlotsOf: (in category 'object enumeration') ----- numPointerSlotsOf: objOop "Answer the number of pointer fields in the given object. Works with CompiledMethods, as well as ordinary objects." <api> <inline: true> <asmLabel: false> | fmt contextSize numLiterals | fmt := self formatOf: objOop. fmt <= self lastPointerFormat ifTrue: [(fmt = self indexablePointersFormat and: [self isContextNonImm: objOop]) ifTrue: ["contexts end at the stack pointer" contextSize := coInterpreter fetchStackPointerOf: objOop. ^CtxtTempFrameStart + contextSize]. ^self numSlotsOf: objOop "all pointers"]. fmt = self forwardedFormat ifTrue: [^1]. fmt < self firstCompiledMethodFormat ifTrue: [^0]. "no pointers" "CompiledMethod: contains both pointers and bytes" numLiterals := coInterpreter literalCountOf: objOop. ^numLiterals + LiteralStart! Item was added: + ----- Method: SpurMemoryManager>>numStrongSlotsOf: (in category 'object access') ----- + numStrongSlotsOf: objOop + "Answer the number of strong pointer fields in the given object. + Works with CompiledMethods, as well as ordinary objects." + <api> + <inline: true> + <asmLabel: false> + | fmt contextSize numLiterals | + fmt := self formatOf: objOop. + fmt <= self lastPointerFormat ifTrue: + [fmt = self weakArrayFormat ifTrue: + [^self fixedFieldsOfClass: (self fetchClassOfNonImm: objOop)]. + (fmt = self indexablePointersFormat + and: [self isContextNonImm: objOop]) ifTrue: + ["contexts end at the stack pointer" + contextSize := coInterpreter fetchStackPointerOf: objOop. + ^CtxtTempFrameStart + contextSize]. + ^self numSlotsOf: objOop "all pointers"]. + fmt = self forwardedFormat ifTrue: [^1]. + fmt < self firstCompiledMethodFormat ifTrue: [^0]. "no pointers" + + "CompiledMethod: contains both pointers and bytes" + numLiterals := coInterpreter literalCountOf: objOop. + ^numLiterals + LiteralStart!

1 0

VM Maker: VMMaker.oscog-eem.414.mcz
by commits＠source.squeak.org 26 Sep '13

26 Sep '13

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.414.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.414 Author: eem Time: 26 September 2013, 2:38:43.606 pm UUID: a208fc7a-c1bd-4ae9-98ab-2a81e5d238fa Ancestors: VMMaker.oscog-eem.413 isWeakNonInt: => isWeakNonImm: (for Spur) =============== Diff against VMMaker.oscog-eem.413 =============== Item was changed: ----- Method: CoInterpreterPrimitives>>pathTo:using:followWeak: (in category 'object access primitives') ----- pathTo: goal using: stack followWeak: followWeak "Trace objects and frames from the root, marking visited objects, pushing the current path on stack, until goal is found. If found, unmark, leaving path in stack, and answer 0. Otherwise answer an error: PrimErrBadArgument if stack is not an Array PrimErrBadIndex if search overflows stack PrimErrNotFound if goal cannot be found" | current hdr index next stackSize stackp freeStartAtStart | (objectMemory isArray: stack) ifFalse: [^PrimErrBadArgument]. freeStartAtStart := objectMemory freeStart. "check no allocations during search" objectMemory beRootIfOld: stack. "so no store checks are necessary on stack" stackSize := objectMemory lengthOf: stack. objectMemory mark: stack. "no need. the current context is not reachable from the active process (suspendedContext is nil)" "objectMemory mark: self activeProcess." current := objectMemory specialObjectsOop. objectMemory mark: current. index := objectMemory lengthOf: current. stackp := 0. [[(index := index - 1) >= -1] whileTrue: [next := (stackPages couldBeFramePointer: current) ifTrue: [index >= 0 ifTrue: [self field: index ofFrame: (self cCoerceSimple: current to: #'char *')] ifFalse: [objectMemory nilObject]] ifFalse: [index >= 0 ifTrue: [hdr := objectMemory baseHeader: current. (objectMemory isContextHeader: hdr) ifTrue: [self fieldOrSenderFP: index ofContext: current] ifFalse: [objectMemory fetchPointer: index ofObject: current]] ifFalse: [objectMemory fetchClassOfNonImm: current]]. (stackPages couldBeFramePointer: next) ifTrue: [self assert: (self isFrame: (self cCoerceSimple: next to: #'char *') onPage: (stackPages stackPageFor: (self cCoerceSimple: next to: #'char *')))] ifFalse: [next >= heapBase ifTrue: [self assert: (self checkOkayOop: next)]]. next = goal ifTrue: [self assert: freeStartAtStart = objectMemory freeStart. self unmarkAfterPathTo. objectMemory storePointer: stackp ofObject: stack withValue: current. self pruneStack: stack stackp: stackp. ^0]. ((objectMemory isNonIntegerObject: next) and: [(stackPages couldBeFramePointer: next) ifTrue: [(self frameIsMarked: next) not] ifFalse: [next >= heapBase "exclude Cog methods" and: [(objectMemory isMarked: next) not and: [((objectMemory isPointers: next) or: [objectMemory isCompiledMethod: next]) + and: [followWeak or: [(objectMemory isWeakNonImm: next) not]]]]]]) - and: [followWeak or: [(objectMemory isWeakNonInt: next) not]]]]]]) ifTrue: [stackp + 2 > stackSize ifTrue: [self assert: freeStartAtStart = objectMemory freeStart. self unmarkAfterPathTo. objectMemory nilFieldsOf: stack. ^PrimErrBadIndex]. "PrimErrNoMemory ?" objectMemory storePointerUnchecked: stackp ofObject: stack withValue: current; storePointerUnchecked: stackp + 1 ofObject: stack withValue: (objectMemory integerObjectOf: index). stackp := stackp + 2. (stackPages couldBeFramePointer: (self cCoerceSimple: next to: #'char *')) ifTrue: [self markFrame: next. index := self fieldsInFrame: (self cCoerceSimple: next to: #'char *')] ifFalse: [hdr := objectMemory baseHeader: next. objectMemory baseHeader: next put: (hdr bitOr: MarkBit). (objectMemory isCompiledMethodHeader: hdr) ifTrue: [index := (self literalCountOf: next) + LiteralStart] ifFalse: [index := objectMemory lengthOf: next]]. current := next]]. current = objectMemory specialObjectsOop ifTrue: [self assert: freeStartAtStart = objectMemory freeStart. self unmarkAfterPathTo. objectMemory nilFieldsOf: stack. ^PrimErrNotFound]. index := objectMemory integerValueOf: (objectMemory fetchPointer: stackp - 1 ofObject: stack). current := objectMemory fetchPointer: stackp - 2 ofObject: stack. stackp := stackp - 2] repeat! Item was changed: ----- Method: NewObjectMemory>>sweepPhase (in category 'gc -- mark and sweep') ----- sweepPhase "Sweep memory from youngStart through the end of memory. Free all inaccessible objects and coalesce adjacent free chunks. Clear the mark bits of accessible objects. Compute the starting point for the first pass of incremental compaction (compStart). Return the number of surviving objects. " "Details: Each time a non-free object is encountered, decrement the number of available forward table entries. If all entries are spoken for (i.e., entriesAvailable reaches zero), set compStart to the last free chunk before that object or, if there is no free chunk before the given object, the first free chunk after it. Thus, at the end of the sweep phase, compStart through compEnd spans the highest collection of non-free objects that can be accomodated by the forwarding table. This information is used by the first pass of incremental compaction to ensure that space is initially freed at the end of memory. Note that there should always be at least one free chunk--the one at the end of the heap." | entriesAvailable survivors freeChunk firstFree oop oopHeader oopHeaderType hdrBytes oopSize freeChunkSize freeStartLocal | <inline: false> <var: #oop type: #usqInt> <var: #freeStartLocal type: #usqInt> entriesAvailable := self fwdTableInit: BytesPerWord*2. survivors := 0. freeChunk := nil. firstFree := nil. "will be updated later" freeStartLocal := freeStart. oop := self oopFromChunk: youngStart. [oop < freeStartLocal] whileTrue: ["get oop's header, header type, size, and header size" statSweepCount := statSweepCount + 1. oopHeader := self baseHeader: oop. oopHeaderType := oopHeader bitAnd: TypeMask. hdrBytes := headerTypeBytes at: oopHeaderType. (oopHeaderType bitAnd: 1) = 1 ifTrue: [oopSize := oopHeader bitAnd: SizeMask] ifFalse: [oopHeaderType = HeaderTypeSizeAndClass ifTrue: [oopSize := (self sizeHeader: oop) bitAnd: LongSizeMask] ifFalse: ["free chunk" oopSize := oopHeader bitAnd: LongSizeMask]]. (oopHeader bitAnd: MarkBit) = 0 ifTrue: ["object is not marked; free it" "<-- Finalization support: We need to mark each oop chunk as free -->" self longAt: oop - hdrBytes put: HeaderTypeFree. freeChunk ~= nil ifTrue: ["enlarge current free chunk to include this oop" freeChunkSize := freeChunkSize + oopSize + hdrBytes] ifFalse: ["start a new free chunk" freeChunk := oop - hdrBytes. "chunk may start 4 or 8 bytes before oop" freeChunkSize := oopSize + (oop - freeChunk). "adjust size for possible extra header bytes" firstFree = nil ifTrue: [firstFree := freeChunk]]] ifFalse: ["object is marked; clear its mark bit and possibly adjust the compaction start" self longAt: oop put: (oopHeader bitAnd: AllButMarkBit). "<-- Finalization support: Check if we're running about a weak class -->" + (self isWeakNonImm: oop) ifTrue: [self finalizeReference: oop]. - (self isWeakNonInt: oop) ifTrue: [self finalizeReference: oop]. entriesAvailable > 0 ifTrue: [entriesAvailable := entriesAvailable - 1] ifFalse: ["start compaction at the last free chunk before this object" firstFree := freeChunk]. freeChunk ~= nil ifTrue: ["record the size of the last free chunk" self longAt: freeChunk put: ((freeChunkSize bitAnd: LongSizeMask) bitOr: HeaderTypeFree). freeChunk := nil]. survivors := survivors + 1]. oopSize = 0 ifTrue: [self error: 'zero sized object encountered in sweep']. oop := self oopFromChunk: oop + oopSize]. freeChunk ~= nil ifTrue: ["record size of final free chunk" self longAt: freeChunk put: ((freeChunkSize bitAnd: LongSizeMask) bitOr: HeaderTypeFree)]. oop = freeStart ifFalse: [self error: 'sweep failed to find exact end of memory']. compStart := firstFree = nil ifTrue: [freeStart] ifFalse: [firstFree]. ^survivors! Item was changed: ----- Method: NewObjectMemory>>sweepPhaseForFullGC (in category 'garbage collection') ----- sweepPhaseForFullGC "Sweep memory from youngStart through the end of memory. Free all inaccessible objects and coalesce adjacent free chunks. Clear the mark bits of accessible objects. Compute the starting point for the first pass of incremental compaction (compStart). Return the number of surviving objects. Unlike sweepPhase this always leaves compStart pointing at the first free chunk." | survivors freeChunk firstFree oop oopHeader oopHeaderType hdrBytes oopSize freeChunkSize endOfMemoryLocal | <inline: false> <var: #oop type: #usqInt> <var: #endOfMemoryLocal type: #usqInt> self fwdTableInit: BytesPerWord*2. survivors := 0. freeChunk := nil. firstFree := nil. "will be updated later" endOfMemoryLocal := endOfMemory. oop := self oopFromChunk: youngStart. [oop < endOfMemoryLocal] whileTrue: ["get oop's header, header type, size, and header size" statSweepCount := statSweepCount + 1. oopHeader := self baseHeader: oop. oopHeaderType := oopHeader bitAnd: TypeMask. hdrBytes := headerTypeBytes at: oopHeaderType. (oopHeaderType bitAnd: 1) = 1 ifTrue: [oopSize := oopHeader bitAnd: SizeMask] ifFalse: [oopHeaderType = HeaderTypeSizeAndClass ifTrue: [oopSize := (self sizeHeader: oop) bitAnd: LongSizeMask] ifFalse: [self assert: (oopHeader bitAnd: MarkBit) = 0. oopSize := oopHeader bitAnd: LongSizeMask]]. (oopHeader bitAnd: MarkBit) = 0 ifTrue: ["object is not marked; free it" "<-- Finalization support: We need to mark each oop chunk as free -->" self longAt: oop - hdrBytes put: HeaderTypeFree. freeChunk ~= nil ifTrue: ["enlarge current free chunk to include this oop" freeChunkSize := freeChunkSize + oopSize + hdrBytes] ifFalse: ["start a new free chunk" freeChunk := oop - hdrBytes. "chunk may start 4 or 8 bytes before oop" freeChunkSize := oopSize + (oop - freeChunk). "adjust size for possible extra header bytes" firstFree = nil ifTrue: [firstFree := freeChunk]]] ifFalse: ["object is marked; clear its mark bit and possibly adjust the compaction start" self longAt: oop put: (oopHeader bitAnd: AllButMarkBit). "<-- Finalization support: Check if we're running about a weak class -->" + (self isWeakNonImm: oop) ifTrue: [self finalizeReference: oop]. - (self isWeakNonInt: oop) ifTrue: [self finalizeReference: oop]. freeChunk ~= nil ifTrue: ["record the size of the last free chunk" self longAt: freeChunk put: ((freeChunkSize bitAnd: LongSizeMask) bitOr: HeaderTypeFree). freeChunk := nil]. survivors := survivors + 1]. oop := self oopFromChunk: oop + oopSize]. freeChunk ~= nil ifTrue: ["record size of final free chunk" self longAt: freeChunk put: ((freeChunkSize bitAnd: LongSizeMask) bitOr: HeaderTypeFree)]. oop = endOfMemory ifFalse: [self error: 'sweep failed to find exact end of memory']. firstFree = nil ifTrue: [self error: 'expected to find at least one free object'] ifFalse: [compStart := firstFree]. ^ survivors! Item was changed: ----- Method: ObjectMemory>>isWeak: (in category 'header access') ----- isWeak: oop "Answer true if the argument has only weak fields that can hold oops. See comment in formatOf:" + ^(self isNonIntegerObject: oop) and: [self isWeakNonImm: oop]! - ^(self isNonIntegerObject: oop) and:[self isWeakNonInt: oop]! Item was added: + ----- Method: ObjectMemory>>isWeakNonImm: (in category 'header access') ----- + isWeakNonImm: oop + "Answer if the argument has only weak fields that can hold oops. See comment in formatOf:" + ^(self formatOf: oop) = self weakArrayFormat! Item was removed: - ----- Method: ObjectMemory>>isWeakNonInt: (in category 'header access') ----- - isWeakNonInt: oop - "Answer if the argument has only weak fields that can hold oops. See comment in formatOf:" - ^(self formatOf: oop) = self lastPointerFormat! Item was changed: ----- Method: ObjectMemory>>nonWeakFieldsOf: (in category 'object format') ----- nonWeakFieldsOf: oop "Return the number of non-weak fields in oop (i.e. the number of fixed fields). Note: The following is copied from fixedFieldsOf:format:length: since we do know the format of the oop (e.g. format = 4) and thus don't need the length." | class classFormat | + self assert: (self isWeakNonImm: oop). - self assert: (self isWeakNonInt: oop). "fmt = 3 or 4: mixture of fixed and indexable fields, so must look at class format word" class := self fetchClassOfNonImm: oop. classFormat := self formatOfClass: class. ^(classFormat >> 11 bitAnd: 16rC0) + (classFormat >> 2 bitAnd: 16r3F) - 1! Item was changed: ----- Method: ObjectMemory>>sweepPhase (in category 'gc -- mark and sweep') ----- sweepPhase "Sweep memory from youngStart through the end of memory. Free all inaccessible objects and coalesce adjacent free chunks. Clear the mark bits of accessible objects. Compute the starting point for the first pass of incremental compaction (compStart). Return the number of surviving objects. " "Details: Each time a non-free object is encountered, decrement the number of available forward table entries. If all entries are spoken for (i.e., entriesAvailable reaches zero), set compStart to the last free chunk before that object or, if there is no free chunk before the given object, the first free chunk after it. Thus, at the end of the sweep phase, compStart through compEnd spans the highest collection of non-free objects that can be accomodated by the forwarding table. This information is used by the first pass of incremental compaction to ensure that space is initially freed at the end of memory. Note that there should always be at least one free chunk--the one at the end of the heap." | entriesAvailable survivors freeChunk firstFree oop oopHeader oopHeaderType hdrBytes oopSize freeChunkSize endOfMemoryLocal | <inline: false> <var: #oop type: 'usqInt'> <var: #endOfMemoryLocal type: 'usqInt'> entriesAvailable := self fwdTableInit: BytesPerWord*2. survivors := 0. freeChunk := nil. firstFree := nil. "will be updated later" endOfMemoryLocal := endOfMemory. oop := self oopFromChunk: youngStart. [oop < endOfMemoryLocal] whileTrue: ["get oop's header, header type, size, and header size" statSweepCount := statSweepCount + 1. oopHeader := self baseHeader: oop. oopHeaderType := oopHeader bitAnd: TypeMask. hdrBytes := headerTypeBytes at: oopHeaderType. (oopHeaderType bitAnd: 1) = 1 ifTrue: [oopSize := oopHeader bitAnd: SizeMask] ifFalse: [oopHeaderType = HeaderTypeSizeAndClass ifTrue: [oopSize := (self sizeHeader: oop) bitAnd: LongSizeMask] ifFalse: ["free chunk" oopSize := oopHeader bitAnd: LongSizeMask]]. (oopHeader bitAnd: MarkBit) = 0 ifTrue: ["object is not marked; free it" "<-- Finalization support: We need to mark each oop chunk as free -->" self longAt: oop - hdrBytes put: HeaderTypeFree. freeChunk ~= nil ifTrue: ["enlarge current free chunk to include this oop" freeChunkSize := freeChunkSize + oopSize + hdrBytes] ifFalse: ["start a new free chunk" freeChunk := oop - hdrBytes. "chunk may start 4 or 8 bytes before oop" freeChunkSize := oopSize + (oop - freeChunk). "adjust size for possible extra header bytes" firstFree = nil ifTrue: [firstFree := freeChunk]]] ifFalse: ["object is marked; clear its mark bit and possibly adjust the compaction start" self longAt: oop put: (oopHeader bitAnd: AllButMarkBit). "<-- Finalization support: Check if we're running about a weak class -->" + (self isWeakNonImm: oop) ifTrue: [self finalizeReference: oop]. - (self isWeakNonInt: oop) ifTrue: [self finalizeReference: oop]. entriesAvailable > 0 ifTrue: [entriesAvailable := entriesAvailable - 1] ifFalse: ["start compaction at the last free chunk before this object" firstFree := freeChunk]. freeChunk ~= nil ifTrue: ["record the size of the last free chunk" self longAt: freeChunk put: ((freeChunkSize bitAnd: LongSizeMask) bitOr: HeaderTypeFree). freeChunk := nil]. survivors := survivors + 1]. oop := self oopFromChunk: oop + oopSize]. freeChunk ~= nil ifTrue: ["record size of final free chunk" self longAt: freeChunk put: ((freeChunkSize bitAnd: LongSizeMask) bitOr: HeaderTypeFree)]. oop = endOfMemory ifFalse: [self error: 'sweep failed to find exact end of memory']. firstFree = nil ifTrue: [self error: 'expected to find at least one free object'] ifFalse: [compStart := firstFree]. ^ survivors! Item was changed: ----- Method: StackInterpreterPrimitives>>pathTo:using:followWeak: (in category 'object access primitives') ----- pathTo: goal using: stack followWeak: followWeak "Trace objects and frames from the root, marking visited objects, pushing the current path on stack, until goal is found. If found, unmark, leaving path in stack, and answer 0. Otherwise answer an error: PrimErrBadArgument if stack is not an Array PrimErrBadIndex if search overflows stack PrimErrNotFound if goal cannot be found" | current hdr index next stackSize stackp freeStartAtStart | (objectMemory isArray: stack) ifFalse: [^PrimErrBadArgument]. freeStartAtStart := objectMemory freeStart. "check no allocations during search" objectMemory beRootIfOld: stack. "so no store checks are necessary on stack" stackSize := objectMemory lengthOf: stack. objectMemory mark: stack. "no need. the current context is not reachable from the active process (suspendedContext is nil)" "objectMemory mark: self activeProcess." current := objectMemory specialObjectsOop. objectMemory mark: current. index := objectMemory lengthOf: current. stackp := 0. [[(index := index - 1) >= -1] whileTrue: [next := (stackPages couldBeFramePointer: current) ifTrue: [index >= 0 ifTrue: [self field: index ofFrame: current] ifFalse: [objectMemory nilObject]] ifFalse: [index >= 0 ifTrue: [hdr := objectMemory baseHeader: current. (objectMemory isContextHeader: hdr) ifTrue: [self fieldOrSenderFP: index ofContext: current] ifFalse: [objectMemory fetchPointer: index ofObject: current]] ifFalse: [objectMemory fetchClassOfNonImm: current]]. (stackPages couldBeFramePointer: next) ifTrue: [self assert: (self isFrame: next onPage: (stackPages stackPageFor: next))] ifFalse: [self assert: (self checkOkayOop: next)]. next = goal ifTrue: [self assert: freeStartAtStart = objectMemory freeStart. self unmarkAfterPathTo. objectMemory storePointer: stackp ofObject: stack withValue: current. self pruneStack: stack stackp: stackp. ^0]. ((objectMemory isNonIntegerObject: next) and: [(stackPages couldBeFramePointer: next) ifTrue: [(self frameIsMarked: next) not] ifFalse: [(objectMemory isMarked: next) not and: [((objectMemory isPointers: next) or: [objectMemory isCompiledMethod: next]) + and: [followWeak or: [(objectMemory isWeakNonImm: next) not]]]]]) - and: [followWeak or: [(objectMemory isWeakNonInt: next) not]]]]]) ifTrue: [stackp + 2 > stackSize ifTrue: [self assert: freeStartAtStart = objectMemory freeStart. self unmarkAfterPathTo. objectMemory nilFieldsOf: stack. ^PrimErrBadIndex]. "PrimErrNoMemory ?" objectMemory storePointerUnchecked: stackp ofObject: stack withValue: current; storePointerUnchecked: stackp + 1 ofObject: stack withValue: (objectMemory integerObjectOf: index). stackp := stackp + 2. (stackPages couldBeFramePointer: (self cCoerceSimple: next to: #'char *')) ifTrue: [self markFrame: next. index := self fieldsInFrame: (self cCoerceSimple: next to: #'char *')] ifFalse: [hdr := objectMemory baseHeader: next. objectMemory baseHeader: next put: (hdr bitOr: MarkBit). (objectMemory isCompiledMethodHeader: hdr) ifTrue: [index := (self literalCountOf: next) + LiteralStart] ifFalse: [index := objectMemory lengthOf: next]]. current := next]]. current = objectMemory specialObjectsOop ifTrue: [self assert: freeStartAtStart = objectMemory freeStart. self unmarkAfterPathTo. objectMemory nilFieldsOf: stack. ^PrimErrNotFound]. index := objectMemory integerValueOf: (objectMemory fetchPointer: stackp - 1 ofObject: stack). current := objectMemory fetchPointer: stackp - 2 ofObject: stack. stackp := stackp - 2] repeat!

1 0

VM Maker: VMMaker.oscog-eem.413.mcz
by commits＠source.squeak.org 26 Sep '13

26 Sep '13

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.413.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.413 Author: eem Time: 26 September 2013, 11:55:56.418 am UUID: c9d2ca92-5900-4492-8e52-0ad86af3530f Ancestors: VMMaker.oscog-eem.412 Fix allocateOldSpaceChunkOfBytes: in two ways a) to not create slivers of 1 allocationUnit long wnen taking storage from the tree (already handled the lists). b) to nil the parent pointer of nodes stitched back into the root (this was a baaad bug) Provide allFreeObjectsDo: and isValidFreeObject: to check ammount and validity of free space. Fix bug in SpurMemMgr>>clone: that misremembered. Add assert to SpurGenerationScavenger>>remember:. Change the assert in ensureBehaviorHash:. More safety in SpurNBitMemMgr>instantiateClass:indexableSize: for the instSpec fall through case. Refactor fillObj:numSlots:with:; it differs between 32 and 64 bits. Sionara initPastSpaceForObjectEnumeration. Fix send printing in commonSend now the VM uses lkupClassTag. Use runLeakCheckerForFullGC: in loadInitialContext instead of internals. Fix simulator initialization of pluginList to include anonynous empty entry for the VM itself. Fix heap map's byteIndex calculation (!!). Nuke unused objectIsOld:. Now bootstrap can run a test for weak arrays and ephemerons through to failure (no support for weakness or ephemerality in the scavenger yet). =============== Diff against VMMaker.oscog-eem.412 =============== Item was removed: - ----- Method: CoInterpreter>>objectIsOld: (in category 'cog jit support') ----- - objectIsOld: anObject - <api> - ^self oop: anObject isLessThan: objectMemory youngStart! Item was changed: Object subclass: #CogCheck32BitHeapMap instanceVariableNames: 'pages' + classVariableNames: 'ByteShift NumPages PageMask PageShift PageSize' - classVariableNames: 'NumPages PageMask PageShift PageSize' poolDictionaries: '' category: 'VMMaker-InterpreterSimulation'! !CogCheck32BitHeapMap commentStamp: 'eem 2/27/2013 16:26' prior: 0! A CogCheckHeapMap is a simulation of the code in platforms/Cross/vm/sqHeapMap.c. This is a map for leak checking that allocates 1 bit for every 4 bytes of address space. It uses an array of pages to keep space overhead low, only allocating a page if that portion of the address space is used. So the maximum overhead is address space size / (word size * bits per byte), or (2 raisedTo: 32) / (4 * 8) or 134,217,728 bytes. Instance Variables pages: <Array of: ByteArray> pages - array of pages of bits, 1 bit per word of address space ! Item was changed: ----- Method: CogCheck32BitHeapMap class>>initialize (in category 'class initialization') ----- initialize "self initialize" | wordSize bitsPerByte | wordSize := 4. "4 bytes per bit in the map" bitsPerByte := 8. NumPages := 256. + PageShift := -24. "(32 - (NumPages log: 2)) negated asInteger" - PageShift := -24. "32 - (NumPages log: 2)" PageSize := 2 << 32 / wordSize / NumPages / bitsPerByte. + PageMask := PageSize - 1. + ByteShift := -5 "1 bit per 4 bytes, 8 bits per byte = 32 bytes of address space per map byte"! - PageMask := PageSize - 1! Item was changed: ----- Method: CogCheck32BitHeapMap>>byteIndex: (in category 'accessing') ----- byteIndex: address + ^((address bitShift: ByteShift) bitAnd: PageMask) + 1! - ^((address bitShift: -2) bitAnd: PageMask) + 1! Item was changed: ----- Method: CogCheck32BitHeapMap>>pageIndex: (in category 'accessing') ----- + pageIndex: address - pageIndex: address ^(address bitShift: PageShift) + 1 "32 - (pageSize log: 2)"! Item was changed: ----- Method: CogVMSimulator>>initialize (in category 'initialization') ----- initialize "Initialize the CogVMSimulator when running the interpreter inside Smalltalk. The primary responsibility of this method is to allocate Smalltalk Arrays for variables that will be declared as statically-allocated global arrays in the translated code." objectMemory ifNil: [objectMemory := self class objectMemoryClass simulatorClass new]. cogit ifNil: [cogit := self class cogitClass new setInterpreter: self]. objectMemory coInterpreter: self cogit: cogit. cogit numRegArgs > 0 ifTrue: [debugStackDepthDictionary := Dictionary new]. cogThreadManager ifNotNil: [super initialize]. "Note: we must initialize ConstMinusOne & HasBeenReturnedFromMCPC differently for simulation, due to the fact that the simulator works only with +ve 32-bit values" ConstMinusOne := objectMemory integerObjectOf: -1. HasBeenReturnedFromMCPC := objectMemory integerObjectOf: -1. cogMethodZone := cogit methodZone. "Because Slang can't remove intermediate implicit receivers (cogit methodZone foo doesn't reduce to foo())" enableCog := true. methodCache := Array new: MethodCacheSize. atCache := Array new: AtCacheTotalSize. self flushMethodCache. self flushAtCache. cogCompiledCodeCompactionCalledFor := false. gcSemaphoreIndex := 0. externalSemaphoreSignalRequests := externalSemaphoreSignalResponses := #(). externalPrimitiveTable := CArrayAccessor on: (Array new: MaxExternalPrimitiveTableSize). externalPrimitiveTableFirstFreeIndex := 0. primitiveTable := self class primitiveTable copy. + pluginList := {'' -> self }. - pluginList := #(). mappedPluginEntries := OrderedCollection new. desiredNumStackPages := desiredEdenBytes := desiredCogCodeSize := 0. "This is initialized on loading the image, but convenient for testing stack page values..." numStackPages := self defaultNumStackPages. startMicroseconds := Time totalSeconds * 1000000. maxLiteralCountForCompile := MaxLiteralCountForCompile. minBackwardJumpCountForCompile := MinBackwardJumpCountForCompile. flagInterpretedMethods := false. "initialize InterpreterSimulator variables used for debugging" byteCount := lastPollCount := sendCount := 0. quitBlock := [^ self]. traceOn := true. printSends := printFrameAtEachStep := printBytecodeAtEachStep := false. myBitBlt := BitBltSimulator new setInterpreter: self. transcript := Transcript. displayForm := 'Display has not yet been installed' asDisplayText form. suppressHeartbeatFlag := deferSmash := deferredSmash := false. systemAttributes := Dictionary new. primTraceLog := CArrayAccessor on: (Array new: 256 withAll: 0). primTraceLogIndex := 0. traceLog := CArrayAccessor on: (Array new: TraceBufferSize withAll: 0). traceLogIndex := 0. traceSources := TraceSources. statCodeCompactionCount := 0. statCodeCompactionUsecs := 0. extSemTabSize := 256! Item was changed: ----- Method: InterpreterSimulator>>initialize (in category 'initialization') ----- initialize "Initialize the InterpreterSimulator when running the interpreter inside Smalltalk. The primary responsibility of this method is to allocate Smalltalk Arrays for variables that will be declared as statically-allocated global arrays in the translated code." "Note: we must initialize ConstMinusOne differently for simulation, due to the fact that the simulator works only with +ve 32-bit values" ConstMinusOne := self integerObjectOf: -1. methodCache := Array new: MethodCacheSize. atCache := Array new: AtCacheTotalSize. self flushMethodCache. rootTable := Array new: RootTableSize. weakRoots := Array new: RootTableSize + RemapBufferSize + 100. remapBuffer := Array new: RemapBufferSize. gcSemaphoreIndex := 0. semaphoresUseBufferA := true. semaphoresToSignalA := Array new: SemaphoresToSignalSize. semaphoresToSignalB := Array new: SemaphoresToSignalSize. externalPrimitiveTable := CArrayAccessor on: (Array new: MaxExternalPrimitiveTableSize). primitiveTable := self class primitiveTable. + pluginList := {'' -> self }. - pluginList := #(). mappedPluginEntries := #(). "initialize InterpreterSimulator variables used for debugging" byteCount := 0. sendCount := 0. quitBlock := [^ self]. traceOn := true. printSends := "printReturns := printBytecodeAtEachStep :=" false. myBitBlt := BitBltSimulator new setInterpreter: self. headerTypeBytes := CArrayAccessor on: HeaderTypeExtraBytes. transcript := Transcript. displayForm := 'Display has not yet been installed' asDisplayText form. ! Item was changed: ----- Method: NewspeakInterpreterSimulator>>initialize (in category 'initialization') ----- initialize "Initialize the NewspeakInterpreterSimulator when running the interpreter inside Smalltalk. The primary responsibility of this method is to allocate Smalltalk Arrays for variables that will be declared as statically-allocated global arrays in the translated code." "initialize class variables" ObjectMemory initBytesPerWord: self wordSize. ObjectMemory initialize. NewspeakInterpreter initialize. super initialize. "Note: we must initialize ConstMinusOne differently for simulation, due to the fact that the simulator works only with +ve 32-bit values" ConstMinusOne := self integerObjectOf: -1. methodCache := Array new: MethodCacheSize. atCache := Array new: AtCacheTotalSize. self flushMethodCache. rootTable := Array new: RootTableSize. weakRoots := Array new: RootTableSize + RemapBufferSize + 100. remapBuffer := Array new: RemapBufferSize. gcSemaphoreIndex := 0. externalPrimitiveTable := CArrayAccessor on: (Array new: MaxExternalPrimitiveTableSize). primitiveTable := self class primitiveTable. + pluginList := {'' -> self }. - pluginList := #(). mappedPluginEntries := #(). primTraceLog := CArrayAccessor on: (Array new: 256 withAll: 0). sendTraceLog := CArrayAccessor on: (Array new: 256 withAll: 0). "initialize NewspeakInterpreterSimulator variables used for debugging" byteCount := 0. sendCount := 0. quitBlock := [^ self]. traceOn := true. printSends := "printReturns := " printBytecodeAtEachStep := printContextAtEachStep := false. myBitBlt := BitBltSimulator new setInterpreter: self. headerTypeBytes := CArrayAccessor on: HeaderTypeExtraBytes. transcript := Transcript. displayForm := 'Display has not yet been installed' asDisplayText form. ! Item was added: + ----- Method: Spur32BitMemoryManager>>fillObj:numSlots:with: (in category 'instantiation') ----- + fillObj: objOop numSlots: numSlots with: fillValue + self assert: (objOop + self baseHeaderSize + (numSlots * self bytesPerOop) - 1) + < (self addressAfter: objOop). + objOop + self baseHeaderSize + to: objOop + self baseHeaderSize + (numSlots * self wordSize) - 1 + by: self allocationUnit + do: [:p| + self longAt: p put: fillValue; + longAt: p + 4 put: fillValue]! Item was changed: + ----- Method: Spur32BitMemoryManager>>instantiateClass:indexableSize: (in category 'instantiation') ----- - ----- Method: Spur32BitMemoryManager>>instantiateClass:indexableSize: (in category 'allocation') ----- instantiateClass: classObj indexableSize: nElements | instSpec classFormat numSlots classIndex newObj fillValue | classFormat := self formatOfClass: classObj. instSpec := self instSpecOfClassFormat: classFormat. fillValue := 0. instSpec caseOf: { [self arrayFormat] -> [numSlots := nElements. fillValue := nilObj]. [self indexablePointersFormat] -> [numSlots := (self fixedFieldsOfClassFormat: classFormat) + nElements. fillValue := nilObj]. [self weakArrayFormat] -> [numSlots := (self fixedFieldsOfClassFormat: classFormat) + nElements. fillValue := nilObj]. [self sixtyFourBitIndexableFormat] -> [numSlots := nElements * 2]. [self firstLongFormat] -> [numSlots := nElements]. [self firstShortFormat] -> [numSlots := nElements + 1 // 2. instSpec := instSpec + (nElements bitAnd: 1)]. [self firstByteFormat] -> [numSlots := nElements + 3 // 4. instSpec := instSpec + (4 - nElements bitAnd: 3)]. [self firstCompiledMethodFormat] -> [numSlots := nElements + 3 // 4. instSpec := instSpec + (4 - nElements bitAnd: 3)] } otherwise: ["some Squeak images include funky fixed subclasses of abstract variable superclasses. e.g. DirectoryEntry as a subclass of ArrayedCollection. Allow fixed classes to be instantiated here iff nElements = 0." + (nElements ~= 0 or: [instSpec >= self sixtyFourBitIndexableFormat]) ifTrue: + [^nil]. + numSlots := self fixedFieldsOfClassFormat: classFormat]. "non-indexable" - numSlots := self fixedFieldsOfClassFormat: classFormat. - nElements ~= 0 ifTrue: - [^nil]]. "non-indexable" classIndex := self ensureBehaviorHash: classObj. classIndex < 0 ifTrue: [coInterpreter primitiveFailFor: classIndex negated. ^nil]. newObj := self allocateSlots: numSlots format: instSpec classIndex: classIndex. newObj ifNotNil: [self fillObj: newObj numSlots: numSlots with: fillValue]. ^newObj! Item was added: + ----- Method: Spur64BitMemoryManager>>fillObj:numSlots:with: (in category 'instantiation') ----- + fillObj: objOop numSlots: numSlots with: fillValue + self assert: (objOop + self baseHeaderSize + (numSlots * self bytesPerOop) - 1) + < (self addressAfter: objOop). + objOop + self baseHeaderSize + to: objOop + self baseHeaderSize + (numSlots * self bytesPerOop) - 1 + by: self allocationUnit + do: [:p| self longAt: p put: fillValue]! Item was changed: + ----- Method: Spur64BitMemoryManager>>instantiateClass:indexableSize: (in category 'instantiation') ----- - ----- Method: Spur64BitMemoryManager>>instantiateClass:indexableSize: (in category 'allocation') ----- instantiateClass: classObj indexableSize: nElements | instSpec classFormat numSlots classIndex newObj fillValue | classFormat := self formatOfClass: classObj. instSpec := self instSpecOfClassFormat: classFormat. fillValue := 0. instSpec caseOf: { [self arrayFormat] -> [numSlots := nElements. fillValue := nilObj]. [self indexablePointersFormat] -> [numSlots := (self fixedFieldsOfClassFormat: classFormat) + nElements. fillValue := nilObj]. [self weakArrayFormat] -> [numSlots := (self fixedFieldsOfClassFormat: classFormat) + nElements. fillValue := nilObj]. [self sixtyFourBitIndexableFormat] -> [numSlots := nElements]. [self firstLongFormat] -> [numSlots := nElements + 1 // 2. instSpec := instSpec + (nElements bitAnd: 1)]. [self firstShortFormat] -> [numSlots := nElements + 3 // 4. instSpec := instSpec + (4 - nElements bitAnd: 3)]. [self firstByteFormat] -> [numSlots := nElements + 7 // 8. instSpec := instSpec + (8 - nElements bitAnd: 7)]. [self firstCompiledMethodFormat] -> [numSlots := nElements + 7 // 8. instSpec := instSpec + (8 - nElements bitAnd: 7)] } + otherwise: ["some Squeak images include funky fixed subclasses of abstract variable + superclasses. e.g. DirectoryEntry as a subclass of ArrayedCollection. + Allow fixed classes to be instantiated here iff nElements = 0." + (nElements ~= 0 or: [instSpec >= self sixtyFourBitIndexableFormat]) ifTrue: + [^nil]. + numSlots := self fixedFieldsOfClassFormat: classFormat]. "non-indexable" - otherwise: [^nil]. "non-indexable" classIndex := self ensureBehaviorHash: classObj. classIndex < 0 ifTrue: [coInterpreter primitiveFailFor: classIndex negated. ^nil]. newObj := self allocateSlots: numSlots format: instSpec classIndex: classIndex. newObj ifNotNil: [self fillObj: newObj numSlots: numSlots with: fillValue]. ^newObj! Item was removed: - ----- Method: SpurGenerationScavenger>>initPastSpaceForObjectEnumeration (in category 'initialization') ----- - initPastSpaceForObjectEnumeration - "For SuurMemoryManager allNewSpaceObjectsDo: fill pastSpace with - a single empty object." - | objOop | - manager initFreeChunkWithBytes: pastSpace limit - pastSpace start at: pastSpace start. - objOop := manager objectStartingAt: pastSpace start. - self assert: (manager addressAfter: objOop) = pastSpace limit! Item was changed: ----- Method: SpurGenerationScavenger>>manager:newSpaceStart:newSpaceBytes:edenBytes: (in category 'initialization') ----- manager: aSpurMemoryManager newSpaceStart: startAddress newSpaceBytes: totalBytes edenBytes: requestedEdenBytes | edenBytes survivorBytes | manager := aSpurMemoryManager. edenBytes := requestedEdenBytes. survivorBytes := totalBytes - edenBytes // 2 truncateTo: manager allocationUnit. edenBytes := totalBytes - survivorBytes - survivorBytes truncateTo: manager allocationUnit. self assert: totalBytes - edenBytes - survivorBytes - survivorBytes < manager allocationUnit. "for tenuring we require older objects below younger objects. since allocation grows up this means that the survivor spaces must preceed eden." pastSpace := SpurNewSpaceSpace new. futureSpace := SpurNewSpaceSpace new. eden := SpurNewSpaceSpace new. pastSpace start: startAddress limit: startAddress + survivorBytes. futureSpace start: pastSpace limit limit: pastSpace limit + survivorBytes. eden start: futureSpace limit limit: futureSpace limit + edenBytes. self assert: futureSpace limit <= (startAddress + totalBytes). self assert: eden start \\ manager allocationUnit + (eden limit \\ manager allocationUnit) = 0. self assert: pastSpace start \\ manager allocationUnit + (pastSpace limit \\ manager allocationUnit) = 0. self assert: futureSpace start \\ manager allocationUnit + (futureSpace limit \\ manager allocationUnit) = 0. self initFutureSpaceStart. - self initPastSpaceForObjectEnumeration. manager initSpaceForAllocationCheck: eden! Item was changed: ----- Method: SpurGenerationScavenger>>remember: (in category 'store check') ----- remember: objOop + self assert: ((manager isNonImmediate: objOop) + and: [(manager isYoung: objOop) not]). rememberedSetSize < RememberedSetLimit ifTrue: [rememberedSet at: rememberedSetSize put: objOop. (rememberedSetSize := rememberedSetSize + 1) >= RememberedSetRedZone ifTrue: [manager scheduleScavenge]] ifFalse: [self error: 'remembered set overflow' "for now"]! Item was added: + ----- Method: SpurGenerationScavengerSimulator>>remember: (in category 'store check') ----- + remember: objOop + (rememberedSetSize > 0 + and: [(rememberedSet at: rememberedSetSize - 1) = objOop]) ifTrue: + [self halt]. + ^super remember: objOop! Item was added: + ----- Method: SpurMemoryManager>>allFreeObjectsDo: (in category 'free space') ----- + allFreeObjectsDo: aBlock + | obj | + 1 to: NumFreeLists - 1 do: + [:i| + obj := freeLists at: i. + [obj ~= 0] whileTrue: + [aBlock value: obj. + obj := self fetchPointer: self freeChunkNextIndex ofFreeChunk: obj]]. + self allObjectsInFreeTree: (freeLists at: 0) do: aBlock! Item was added: + ----- Method: SpurMemoryManager>>allObjectsInFreeTree:do: (in category 'free space') ----- + allObjectsInFreeTree: freeNode do: aBlock + | listNode | + freeNode = 0 ifTrue: [^0]. + listNode := freeNode. + [listNode ~= 0] whileTrue: + [aBlock value: listNode. + listNode := self fetchPointer: self freeChunkNextIndex ofFreeChunk: listNode]. + self allObjectsInFreeTree: (self fetchPointer: self freeChunkSmallerIndex ofFreeChunk: freeNode) + do: aBlock. + self allObjectsInFreeTree: (self fetchPointer: self freeChunkLargerIndex ofFreeChunk: freeNode) + do: aBlock! Item was changed: ----- Method: SpurMemoryManager>>allocateOldSpaceChunkOfBytes: (in category 'free space') ----- allocateOldSpaceChunkOfBytes: chunkBytes "Answer a chunk of oldSpace from the free lists, if available, otherwise answer nil. N.B. the chunk is simply a pointer, it has no valid header. The caller *must* fill in the header correctly." | initialIndex chunk index nodeBytes parent child smaller larger | "for debugging:" "totalFreeOldSpace := self totalFreeListBytes" totalFreeOldSpace := totalFreeOldSpace - chunkBytes. "be optimistic (& don't wait for the write)" initialIndex := chunkBytes / self allocationUnit. (initialIndex < NumFreeLists and: [1 << initialIndex <= freeListsMask]) ifTrue: [(chunk := freeLists at: initialIndex) ~= 0 ifTrue: [self assert: chunk = (self startOfObject: chunk). + self assert: (self isValidFreeObject: chunk). ^self unlinkFreeChunk: chunk atIndex: initialIndex]. freeListsMask := freeListsMask - (1 << initialIndex). "first search for free chunks of a multiple of chunkBytes in size" index := initialIndex. [(index := index + index) < NumFreeLists and: [1 << index <= freeListsMask]] whileTrue: [((freeListsMask anyMask: 1 << index) and: [(chunk := freeLists at: index) ~= 0]) ifTrue: [self assert: chunk = (self startOfObject: chunk). + self assert: (self isValidFreeObject: chunk). self unlinkFreeChunk: chunk atIndex: index. self assert: (self bytesInObject: chunk) = (index * self allocationUnit). self freeChunkWithBytes: index * self allocationUnit - chunkBytes at: (self startOfObject: chunk) + chunkBytes. ^chunk]]. "now get desperate and use the first that'll fit. Note that because the minimum free size is 16 bytes (2 * allocationUnit), to leave room for the forwarding pointer/next free link, we can only break chunks that are at least 16 bytes larger, hence start at initialIndex + 2." index := initialIndex + 1. [(index := index + 1) < NumFreeLists and: [1 << index <= freeListsMask]] whileTrue: [(freeListsMask anyMask: 1 << index) ifTrue: [(chunk := freeLists at: index) ~= 0 ifTrue: [self assert: chunk = (self startOfObject: chunk). + self assert: (self isValidFreeObject: chunk). self unlinkFreeChunk: chunk atIndex: index. self assert: (self bytesInObject: chunk) = (index * self allocationUnit). self freeChunkWithBytes: index * self allocationUnit - chunkBytes at: (self startOfObject: chunk) + chunkBytes. ^chunk]. freeListsMask := freeListsMask - (1 << index)]]]. "Large chunk, or no space on small free lists. Search the large chunk list. Large chunk list organized as a tree, each node of which is a list of chunks of the same size. Beneath the node are smaller and larger blocks. When the search ends parent should hold the smallest chunk at least as large as chunkBytes, or 0 if none." parent := 0. child := freeLists at: 0. [child ~= 0] whileTrue: [| childBytes | + self assert: (self isValidFreeObject: child). - self assert: (self isFreeObject: child). childBytes := self bytesInObject: child. childBytes = chunkBytes ifTrue: "size match; try to remove from list at node." [chunk := self fetchPointer: self freeChunkNextIndex ofFreeChunk: child. chunk ~= 0 ifTrue: + [self assert: (self isValidFreeObject: chunk). + self storePointer: self freeChunkNextIndex - [self storePointer: self freeChunkNextIndex ofFreeChunk: child withValue: (self fetchPointer: self freeChunkNextIndex ofFreeChunk: chunk). ^self startOfObject: chunk]. child := 0]. "break out of loop to remove interior node" + "Note that because the minimum free size is 16 bytes (2 * allocationUnit), to + leave room for the forwarding pointer/next free link, we can only break chunks + that are at least 16 bytes larger, hence reject chunks < 2 * allocationUnit larger." + childBytes <= (chunkBytes + self allocationUnit) - childBytes < chunkBytes ifTrue: "walk down the tree" [child := self fetchPointer: self freeChunkLargerIndex ofFreeChunk: child] ifFalse: [parent := child. nodeBytes := childBytes. child := self fetchPointer: self freeChunkSmallerIndex ofFreeChunk: child]]. parent = 0 ifTrue: [totalFreeOldSpace := totalFreeOldSpace + chunkBytes. "optimism was unfounded" + self halt. + ^nil]. - self halt]. "self printFreeChunk: parent" + self assert: (nodeBytes = chunkBytes or: [nodeBytes >= (chunkBytes + (2 * self allocationUnit))]). - self assert: nodeBytes >= chunkBytes. self assert: (self bytesInObject: parent) = nodeBytes. "attempt to remove from list" chunk := self fetchPointer: self freeChunkNextIndex ofFreeChunk: parent. chunk ~= 0 ifTrue: + [self assert: (chunkBytes = nodeBytes or: [chunkBytes + self allocationUnit < nodeBytes]). + self storePointer: self freeChunkNextIndex - [self storePointer: self freeChunkNextIndex ofFreeChunk: parent withValue: (self fetchPointer: self freeChunkNextIndex ofFreeChunk: chunk). chunkBytes ~= nodeBytes ifTrue: [self freeChunkWithBytes: nodeBytes - chunkBytes at: (self startOfObject: chunk) + chunkBytes]. ^self startOfObject: chunk]. "no list; remove an interior node; reorder tree simply. two cases (which have mirrors, for four total): case 1. interior node has one child, P = parent, N = node, S = subtree (mirrored for large vs small) ___ ___ | P | | P | _/_ _/_ | N | => | S | _/_ | S | case 2: interior node has two children, , P = parent, N = node, L = smaller, left subtree, R = larger, right subtree. add the left subtree to the bottom left of the right subtree (mirrored for large vs small) ___ ___ | P | | P | _/_ _/_ | N | => | R | _/_ _\_ _/_ | L | | R | | L |" chunk := parent. smaller := self fetchPointer: self freeChunkSmallerIndex ofFreeChunk: chunk. larger := self fetchPointer: self freeChunkLargerIndex ofFreeChunk: chunk. parent := self fetchPointer: self freeChunkParentIndex ofFreeChunk: chunk. parent = 0 ifTrue: "no parent; stitch the subnodes back into the root" [smaller = 0 + ifTrue: + [self storePointer: self freeChunkParentIndex ofFreeChunk: larger withValue: 0. + freeLists at: 0 put: larger] - ifTrue: [freeLists at: 0 put: larger] ifFalse: + [self storePointer: self freeChunkParentIndex ofFreeChunk: smaller withValue: 0. + freeLists at: 0 put: smaller. - [freeLists at: 0 put: smaller. larger ~= 0 ifTrue: [self addFreeSubTree: larger]]] ifFalse: "parent; stitch back into appropriate side of parent." [smaller = 0 ifTrue: [self storePointer: (chunk = (self fetchPointer: self freeChunkSmallerIndex ofFreeChunk: parent) ifTrue: [self freeChunkSmallerIndex] ifFalse: [self freeChunkLargerIndex]) ofFreeChunk: parent withValue: larger] ifFalse: [self storePointer: (chunk = (self fetchPointer: self freeChunkSmallerIndex ofFreeChunk: parent) ifTrue: [self freeChunkSmallerIndex] ifFalse: [self freeChunkLargerIndex]) ofFreeChunk: parent withValue: smaller. larger ~= 0 ifTrue: [self addFreeSubTree: larger]]]. "if there's space left over, add the fragment back." chunkBytes ~= nodeBytes ifTrue: [self freeChunkWithBytes: nodeBytes - chunkBytes at: (self startOfObject: chunk) + chunkBytes]. ^self startOfObject: chunk! Item was changed: ----- Method: SpurMemoryManager>>bytesInFreeTree: (in category 'free space') ----- bytesInFreeTree: freeNode | freeBytes bytesInObject listNode | freeNode = 0 ifTrue: [^0]. freeBytes := 0. bytesInObject := self bytesInObject: freeNode. self assert: bytesInObject / self allocationUnit >= NumFreeLists. - self assert: (self isFreeObject: freeNode). listNode := freeNode. [listNode ~= 0] whileTrue: + ["self printFreeChunk: listNode" + self assert: (self isValidFreeObject: listNode). + freeBytes := freeBytes + bytesInObject. - [freeBytes := freeBytes + bytesInObject. self assert: bytesInObject = (self bytesInObject: listNode). listNode := self fetchPointer: self freeChunkNextIndex ofFreeChunk: listNode]. ^freeBytes + (self bytesInFreeTree: (self fetchPointer: self freeChunkSmallerIndex ofFreeChunk: freeNode)) + (self bytesInFreeTree: (self fetchPointer: self freeChunkLargerIndex ofFreeChunk: freeNode))! Item was changed: ----- Method: SpurMemoryManager>>checkHeapIntegrity (in category 'debug support') ----- checkHeapIntegrity "Perform an integrity/leak check using the heapMap. Assume clearLeakMapAndMapAccessibleObjects has set a bit at each object's header. Scan all objects in the heap checking that every pointer points to a header. Scan the rootTable, remapBuffer and extraRootTable checking that every entry is a pointer to a header. Check that the number of roots is correct and that all rootTable entries have their rootBit set. Answer if all checks pass." | prevObj prevPrevObj ok numRememberedRootsInHeap | <inline: false> ok := true. numRememberedRootsInHeap := 0. self allHeapEntitiesDo: [:obj| | containsYoung fieldOop classIndex classOop | (self isFreeObject: obj) ifFalse: [containsYoung := false. (self isRemembered: obj) ifTrue: [numRememberedRootsInHeap := numRememberedRootsInHeap + 1. (scavenger isInRememberedTable: obj) ifFalse: [coInterpreter print: 'remembered object '; printHex: obj; print: ' is not in remembered table'; cr. self eek. ok := false]]. (self isForwarded: obj) ifTrue: [fieldOop := self fetchPointer: 0 ofMaybeForwardedObject: obj. (heapMap heapMapAtWord: (self pointerForOop: fieldOop)) = 0 ifTrue: [coInterpreter print: 'object leak in forwarder '; printHex: obj; print: ' to unmapped '; printHex: fieldOop; cr. self eek. ok := false]. (self isYoung: fieldOop) ifTrue: [containsYoung := true]] ifFalse: [classOop := self classAtIndex: (classIndex := self classIndexOf: obj). (classOop isNil or: [classOop = nilObj]) ifTrue: [coInterpreter print: 'object leak in '; printHex: obj; print: ' invalid class index '; printHex: classIndex; print: ' -> '; print: (classOop ifNil: ['nil'] ifNotNil: ['nilObj']); cr. self eek. ok := false]. self baseHeaderSize to: (self lastPointerOf: obj) by: BytesPerOop do: [:ptr| fieldOop := self longAt: obj + ptr. (self isNonImmediate: fieldOop) ifTrue: [| fi | fi := ptr - self baseHeaderSize / self wordSize. (fieldOop bitAnd: self wordSize - 1) ~= 0 ifTrue: [coInterpreter print: 'misaligned oop in '; printHex: obj; print: ' @ '; printNum: fi; print: ' = '; printHex: fieldOop; cr. self eek. ok := false] ifFalse: [(heapMap heapMapAtWord: (self pointerForOop: fieldOop)) = 0 ifTrue: [coInterpreter print: 'object leak in '; printHex: obj; print: ' @ '; printNum: fi; print: ' = '; printHex: fieldOop; cr. self eek. ok := false]. (self isYoung: fieldOop) ifTrue: [containsYoung := true]]]]]. (containsYoung and: [(self isYoung: obj) not]) ifTrue: [(self isRemembered: obj) ifFalse: [coInterpreter print: 'unremembered object '; printHex: obj; print: ' contains young oop(s)'; cr. self eek. ok := false]]]. prevPrevObj := prevObj. prevObj := obj]. numRememberedRootsInHeap ~= scavenger rememberedSetSize ifTrue: [coInterpreter print: 'root count mismatch. #heap roots '; printNum: numRememberedRootsInHeap; print: '; #roots '; printNum: scavenger rememberedSetSize; cr. + self eek. "But the system copes with overflow..." self flag: 'no support for remembered set overflow yet'. "ok := rootTableOverflowed and: [needGCFlag]"]. scavenger rememberedSetWithIndexDo: [:obj :i| (obj bitAnd: self wordSize - 1) ~= 0 ifTrue: [coInterpreter print: 'misaligned oop in rootTable @ '; printNum: i; print: ' = '; printHex: obj; cr. self eek. ok := false] ifFalse: [(heapMap heapMapAtWord: (self pointerForOop: obj)) = 0 ifTrue: + [coInterpreter print: 'object leak in remembered set @ '; printNum: i; print: ' = '; printHex: obj; cr. - [coInterpreter print: 'object leak in rootTable @ '; printNum: i; print: ' = '; printHex: obj; cr. self eek. ok := false] ifFalse: [(self isYoung: obj) ifTrue: + [coInterpreter print: 'non-root in remembered set @ '; printNum: i; print: ' = '; printHex: obj; cr. - [coInterpreter print: 'non-root in rootTable @ '; printNum: i; print: ' = '; printHex: obj; cr. self eek. ok := false]]]]. self flag: 'no support for remap buffer yet'. "1 to: remapBufferCount do: [:ri| obj := remapBuffer at: ri. (obj bitAnd: self wordSize - 1) ~= 0 ifTrue: [coInterpreter print: 'misaligned remapRoot @ '; printNum: ri; print: ' = '; printHex: obj; cr. self eek. ok := false] ifFalse: [(heapMap heapMapAtWord: (self pointerForOop: obj)) = 0 ifTrue: [coInterpreter print: 'object leak in remapRoots @ '; printNum: ri; print: ' = '; printHex: obj; cr. self eek. ok := false]]]." self flag: 'no support for extraRoots yet'. "1 to: extraRootCount do: [:ri| obj := (extraRoots at: ri) at: 0. (obj bitAnd: self wordSize - 1) ~= 0 ifTrue: [coInterpreter print: 'misaligned extraRoot @ '; printNum: ri; print: ' => '; printHex: obj; cr. self eek. ok := false] ifFalse: [(heapMap heapMapAtWord: (self pointerForOop: obj)) = 0 ifTrue: [coInterpreter print: 'object leak in extraRoots @ '; printNum: ri; print: ' => '; printHex: obj; cr. self eek. ok := false]]]." ^ok! Item was changed: ----- Method: SpurMemoryManager>>clone: (in category 'allocation') ----- clone: objOop | numSlots newObj | numSlots := self numSlotsOf: objOop. newObj := self allocateSlots: (self numSlotsOf: objOop) format: (self formatOf: objOop) classIndex: (self classIndexOf: objOop). (self isPointersNonImm: objOop) ifTrue: [0 to: numSlots - 1 do: [:i| | oop | oop := self fetchPointer: i ofObject: objOop. ((self isNonImmediate: oop) and: [self isForwarded: oop]) ifTrue: [oop := self followForwarded: oop]. self storePointerUnchecked: i ofObject: newObj withValue: oop]. ((self isRemembered: objOop) + and: [(self isYoung: newObj) not]) ifTrue: + [scavenger remember: newObj. + self setIsRememberedOf: newObj to: true]] - and: [self isYoung: newObj]) ifTrue: - [scavenger remember: objOop. - self setIsRememberedOf: objOop to: true]] ifFalse: [0 to: numSlots - 1 do: [:i| self storePointerUnchecked: i ofObject: newObj withValue: (self fetchPointer: i ofObject: objOop)]]. ^newObj! Item was changed: ----- Method: SpurMemoryManager>>ensureBehaviorHash: (in category 'class table') ----- ensureBehaviorHash: aBehavior | newHash err | <inline: true> + self assert: (coInterpreter addressCouldBeClassObj: aBehavior). - self assert: (self isImmediate: aBehavior) not. (newHash := self rawHashBitsOf: aBehavior) = 0 ifTrue: [(err := self enterIntoClassTable: aBehavior) ~= 0 ifTrue: [^err negated]. newHash := self rawHashBitsOf: aBehavior. self assert: (self classAtIndex: newHash) = aBehavior]. ^newHash! Item was changed: + ----- Method: SpurMemoryManager>>fillObj:numSlots:with: (in category 'instantiation') ----- - ----- Method: SpurMemoryManager>>fillObj:numSlots:with: (in category 'allocation') ----- fillObj: objOop numSlots: numSlots with: fillValue + self subclassResponsibility! - objOop + self baseHeaderSize - to: objOop + self baseHeaderSize + (numSlots * self bytesPerOop) - 1 - by: self allocationUnit - do: [:p| - self assert: p < (self addressAfter: objOop). - self longAt: p put: fillValue; - longAt: p + 4 put: fillValue]! Item was changed: ----- Method: SpurMemoryManager>>isNonIntegerObject: (in category 'object testing') ----- isNonIntegerObject: oop "This list records the valid senders of isNonIntegerObject: as we replace uses of isNonIntegerObject: by isNonImmediate: where appropriate." + (#( on:do: "from the dbeugger" + reverseDisplayFrom:to: - (#( reverseDisplayFrom:to: primitiveObjectAtPut) includes: thisContext sender method selector) ifFalse: [self halt]. ^(oop bitAnd: 1) = 0! Item was added: + ----- Method: SpurMemoryManager>>isValidFreeObject: (in category 'free space') ----- + isValidFreeObject: objOop + | chunk | + ^(self isFreeObject: objOop) + and: [((chunk := (self fetchPointer: self freeChunkNextIndex ofFreeChunk: objOop)) = 0 + or: [self isFreeObject: chunk]) + and: [(self bytesInObject: objOop) / self allocationUnit < NumFreeLists + or: [((chunk := (self fetchPointer: self freeChunkParentIndex ofFreeChunk: objOop)) = 0 + or: [self isFreeObject: chunk]) + and: [((chunk := (self fetchPointer: self freeChunkSmallerIndex ofFreeChunk: objOop)) = 0 + or: [self isFreeObject: chunk]) + and: [(chunk := (self fetchPointer: self freeChunkLargerIndex ofFreeChunk: objOop)) = 0 + or: [self isFreeObject: chunk]]]]]]! Item was added: + ----- Method: SpurMemoryManager>>sizeOfFree: (in category 'free space') ----- + sizeOfFree: objOop + "For compatibility with ObjectMemory, answer the size of a free chunk in bytes, + ignoring the overflow header. Do *not* use internally." + self assert: (self isFreeObject: objOop). + ^self baseHeaderSize + (self wordSize * (self numSlotsOfAny: objOop))! Item was changed: ----- Method: SpurMemoryManager>>totalFreeListBytes (in category 'free space') ----- totalFreeListBytes | freeBytes bytesInObject obj | freeBytes := 0. 1 to: NumFreeLists - 1 do: [:i| bytesInObject := i * self allocationUnit. obj := freeLists at: i. [obj ~= 0] whileTrue: [freeBytes := freeBytes + bytesInObject. self assert: bytesInObject = (self bytesInObject: obj). + self assert: (self isValidFreeObject: obj). - self assert: (self isFreeObject: obj). obj := self fetchPointer: self freeChunkNextIndex ofFreeChunk: obj]]. ^freeBytes + (self bytesInFreeTree: (freeLists at: 0))! Item was changed: ----- Method: StackInterpreter>>commonSend (in category 'send bytecodes') ----- commonSend "Send a message, starting lookup with the receiver's class." "Assume: messageSelector and argumentCount have been set, and that the receiver and arguments have been pushed onto the stack," "Note: This method is inlined into the interpreter dispatch loop." <sharedCodeNamed: 'commonSend' inCase: #singleExtendedSendBytecode> self sendBreakpoint: messageSelector receiver: (self internalStackValue: argumentCount). self printSends ifTrue: + [self printActivationNameForSelector: messageSelector startClass: (objectMemory classForClassTag: lkupClassTag); cr]. - [self printActivationNameForSelector: messageSelector startClass: lkupClass; cr]. self internalFindNewMethod. self internalExecuteNewMethod. self fetchNextBytecode! Item was changed: ----- Method: StackInterpreter>>loadInitialContext (in category 'initialization') ----- loadInitialContext <inline: false> | activeProc activeContext | self cCode: [] inSmalltalk: [self initExtensions]. + objectMemory runLeakCheckerForFullGC: true. - objectMemory leakCheckFullGC ifTrue: - [objectMemory clearLeakMapAndMapAccessibleObjects. - self assert: objectMemory checkHeapIntegrity]. activeProc := self activeProcess. activeContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: activeProc. self marryContextInNewStackPageAndInitializeInterpreterRegisters: activeContext! Item was changed: ----- Method: StackInterpreterSimulator>>initialize (in category 'initialization') ----- initialize "Initialize the StackInterpreterSimulator when running the interpreter inside Smalltalk. The primary responsibility of this method is to allocate Smalltalk Arrays for variables that will be declared as statically-allocated global arrays in the translated code." objectMemory ifNil: [objectMemory := self class objectMemoryClass simulatorClass new]. objectMemory coInterpreter: self. "Note: we must initialize ConstMinusOne differently for simulation, due to the fact that the simulator works only with +ve 32-bit values" ConstMinusOne := objectMemory integerObjectOf: -1. methodCache := Array new: MethodCacheSize. atCache := Array new: AtCacheTotalSize. self flushMethodCache. self flushAtCache. gcSemaphoreIndex := 0. externalSemaphoreSignalRequests := externalSemaphoreSignalResponses := #(). externalPrimitiveTable := CArrayAccessor on: (Array new: MaxExternalPrimitiveTableSize). externalPrimitiveTableFirstFreeIndex := 0. primitiveTable := self class primitiveTable copy. + pluginList := {'' -> self }. - pluginList := #(). mappedPluginEntries := #(). desiredNumStackPages := desiredEdenBytes := 0. "This is initialized on loading the image, but convenient for testing stack page values..." numStackPages := self defaultNumStackPages. startMicroseconds := Time totalSeconds * 1000000. "initialize InterpreterSimulator variables used for debugging" byteCount := 0. sendCount := 0. quitBlock := [^ self]. traceOn := true. printSends := printReturns := printFrameAtEachStep := printBytecodeAtEachStep := false. myBitBlt := BitBltSimulator new setInterpreter: self. transcript := Transcript. displayForm := 'Display has not yet been installed' asDisplayText form. suppressHeartbeatFlag := false. extSemTabSize := 256. disableBooleanCheat := false! Item was changed: ----- Method: StackInterpreterSimulator>>primitiveExecuteMethodArgsArray (in category 'control primitives') ----- primitiveExecuteMethodArgsArray + self halt: thisContext selector. - self halt. ^super primitiveExecuteMethodArgsArray!

1 0

Re: [Pharo-dev] [NB] Trying to implement a ReadStream on NBExternalAddress
by Eliot Miranda 26 Sep '13

26 Sep '13

On Mon, Sep 23, 2013 at 3:10 PM, Nicolas Cellier < nicolas.cellier.aka.nice(a)gmail.com> wrote: > Isn't Eliot just implementing this feature, having a segment of non > relocatable objects? > Sort of. Spur will allow any object in old space to stay still, and through become can move any object to old space very simply. So instead of having a special fixed space segment it has a best-bit compaction algorithm that doesn't slide objects, but moves them into holes. With this kind of compaction it is very easy to leave objects put. A further advantage of Spur is that objects have 64-bit alignment so passing arrays to e.g. code using sse instructions, won't cause potential alignment faults. But for NB see below on the hack that the ThreadedFFI uses. > > 2013/9/23 Igor Stasenko <siguctua(a)gmail.com> > >> >> >> >> On 23 September 2013 21:40, Igor Stasenko <siguctua(a)gmail.com> wrote: >> >>> >>> >>> >>> On 23 September 2013 16:23, Camillo Bruni <camillobruni(a)gmail.com>wrote: >>> >>>> Hi Jan, >>>> >>>> I think I will add the ByteArray accessor to NBExternalAddress today >>>> or tomorrow since I need it as well for another project. >>>> >>>> hmm, reading from memory into bytearray can be done with memory copy: >>> >>> inputs: address , offset , size to read >>> >>> newAddress := NBExternalAddress value: address value + offset. >>> buffer := ByteArray new: size. >>> NativeBoost memCopy: newAddress to: buffer size: size. >>> >>> same way, writing, just swap the source and destination: >>> >>> newAddress := NBExternalAddress value: address value + offset. >>> buffer "is given from somewhere". >>> NativeBoost memCopy: buffer to: newAddress size: size. >>> >>> but as Jan noted, you cannot tell to write starting at specified offset >>> from/to bytearray, e.g.: >>> >>> copy from: address to: buffer + someOffset >>> neither: >>> copy from: buffer + someOffset to: someAddress >>> >>> this where we need to introduce special 'field address' type, so you >>> can construct it like this: >>> >>> offsetAddress := buffer nbAddressAt: offset. >>> >>> so then you can use it to pass to any function, which expects address, >>> like memory copy >>> or any foreign function. >>> >>> Since objects are moving in memory, we cannot calculate address of field >>> before hand: >>> >>> address := NBExternalAddress value: someObject address + offset. >>> >>> because if GC will happen, after computing such address and its actual >>> use, >>> you will read/write to wrong location. >>> >> >> ** after computing and *before* actual use ** >> >> >>> Thus we should keep oop + offset up to the point of passing it to >>> external function, under >>> controllable conditions, that guarantee there's no GC is possible. >>> >>> >>> Things would be much simpler if we could have pinning, isnt? :) >> > Yes, but for the moment there is a hack one can use, a neat hack invented by Andreas Raab. The Squeak GC is a two-space GC, old space (collected by fullGC) and new space (collected by incrementalGC). An incrementalGC will move objects in new space but leave objects on old space alone. A tenuringIncrementalGC will compact new space and then make new space part of old space. Therefore one way of nearly pinning objects is to do a young GC to tenure objects into old space via tenuringIncrementalGC and then lock fullGC, prevent fullGC from running, until the external call is finished. All arguments to the call become old, and they won't be moved until the fuuGCLock is released. This doesn't help passing a buffer that will be used after the call returns, but it does help a buffer being passed to code that might callback. See uses of PrimErrObjectMayMove, e.g. ThreadedFFIPlugin>>primitiveCallout and platforms/Cross/plugins/FilePlugin/sqFilePluginBasicPrims.c>>sqFileReadIntoAt HTH eliot

6 13

[commit][2787] Include the pc in the printed register state and the backtrace, if possible.
by commits＠squeakvm.org 25 Sep '13

25 Sep '13

Revision: 2787 Author: eliot Date: 2013-09-25 14:15:45 -0700 (Wed, 25 Sep 2013) Log Message: ----------- Include the pc in the printed register state and the backtrace, if possible. Modified Paths: -------------- branches/Cog/platforms/Mac OS/vm/sqMacMain.c branches/Cog/platforms/unix/vm/sqUnixMain.c Modified: branches/Cog/platforms/Mac OS/vm/sqMacMain.c =================================================================== --- branches/Cog/platforms/Mac OS/vm/sqMacMain.c 2013-09-25 20:37:08 UTC (rev 2786) +++ branches/Cog/platforms/Mac OS/vm/sqMacMain.c 2013-09-25 21:15:45 UTC (rev 2787) @@ -161,13 +161,14 @@ /* Print an error message, possibly a stack trace, do /not/ exit. * Allows e.g. writing to a log file and stderr. */ -static void printRegisterState(ucontext_t *uap); +static void *printRegisterState(ucontext_t *uap); static void reportStackState(char *msg, char *date, int printAll, ucontext_t *uap) { #if !defined(NOEXECINFO) - void *addrs[BACKTRACE_DEPTH]; + void *addrs[BACKTRACE_DEPTH+1]; + void *pc; int depth; #endif /* flag prevents recursive error when trying to print a broken stack */ @@ -177,16 +178,18 @@ printf("%s\n\n", getVersionInfo(1)); #if !defined(NOEXECINFO) - printRegisterState(uap); - printf("C stack backtrace:\n"); + printf("C stack backtrace & registers:\n"); + pc = printRegisterState(uap); + depth = backtrace(addrs + 1, BACKTRACE_DEPTH); + addrs[0] = pc; +# if 0 /* Mac OS's backtrace_symbols_fd prints NULL byte droppings each line */ fflush(stdout); /* backtrace_symbols_fd uses unbuffered i/o */ - depth = backtrace(addrs, BACKTRACE_DEPTH); -# if 0 /* Mac OS's backtrace_symbols_fd prints NULL byte droppings each line */ - backtrace_symbols_fd(addrs, depth, fileno(stdout)); + backtrace_symbols_fd(addrs, depth + 1, fileno(stdout)); # else { int i; char **strings; - strings = backtrace_symbols(addrs, depth); - for (i = 0; i < depth; i++) + strings = backtrace_symbols(addrs, depth + 1); + printf("(%s)\n", strings[0]); + for (i = 1; i < depth; i++) printf("%s\n", strings[i]); } # endif @@ -250,23 +253,30 @@ } /* Attempt to dump the registers to stdout. Only do so if we know how. */ -static void +static void * printRegisterState(ucontext_t *uap) { #if __DARWIN_UNIX03 && __APPLE__ && __MACH__ && __i386__ _STRUCT_X86_THREAD_STATE32 *regs = &uap->uc_mcontext->__ss; - printf( "eax 0x%08x ebx 0x%08x ecx 0x%08x edx 0x%08x\n" - "edi 0x%08x esi 0x%08x ebp 0x%08x esp 0x%08x\n", + printf( "\teax 0x%08x ebx 0x%08x ecx 0x%08x edx 0x%08x\n" + "\tedi 0x%08x esi 0x%08x ebp 0x%08x esp 0x%08x\n" + "\teip 0x%08x\n", regs->__eax, regs->__ebx, regs->__ecx, regs->__edx, - regs->__edi, regs->__edi, regs->__ebp, regs->__esp); + regs->__edi, regs->__edi, regs->__ebp, regs->__esp, + regs->__eip); + return regs->__eip; #elif __APPLE__ && __MACH__ && __i386__ _STRUCT_X86_THREAD_STATE32 *regs = &uap->uc_mcontext->ss; - printf( "eax 0x%08x ebx 0x%08x ecx 0x%08x edx 0x%08x\n" - "edi 0x%08x esi 0x%08x ebp 0x%08x esp 0x%08x\n", + printf( "\teax 0x%08x ebx 0x%08x ecx 0x%08x edx 0x%08x\n" + "\tedi 0x%08x esi 0x%08x ebp 0x%08x esp 0x%08x\n" + "\teip 0x%08x\n", regs->eax, regs->ebx, regs->ecx, regs->edx, - regs->edi, regs->edi, regs->ebp, regs->esp); + regs->edi, regs->edi, regs->ebp, regs->esp, + regs->eip); + return regs->eip; #else printf("don't know how to derive register state from a ucontext_t on this platform\n"); + return 0; #endif } Modified: branches/Cog/platforms/unix/vm/sqUnixMain.c =================================================================== --- branches/Cog/platforms/unix/vm/sqUnixMain.c 2013-09-25 20:37:08 UTC (rev 2786) +++ branches/Cog/platforms/unix/vm/sqUnixMain.c 2013-09-25 21:15:45 UTC (rev 2787) @@ -786,13 +786,14 @@ /* Print an error message, possibly a stack trace, do /not/ exit. * Allows e.g. writing to a log file and stderr. */ -static void printRegisterState(ucontext_t *uap); +static void *printRegisterState(ucontext_t *uap); static void reportStackState(char *msg, char *date, int printAll, ucontext_t *uap) { #if !defined(NOEXECINFO) void *addrs[BACKTRACE_DEPTH]; + void *pc; int depth; #endif /* flag prevents recursive error when trying to print a broken stack */ @@ -802,11 +803,13 @@ printf("%s\n\n", getVersionInfo(1)); #if !defined(NOEXECINFO) - printRegisterState(uap); - printf("C stack backtrace:\n"); + printf("C stack backtrace & registers:\n"); + pc = printRegisterState(uap); + depth = backtrace(addrs + 1, BACKTRACE_DEPTH); + addrs[0] = pc; + putchar('*'); /* indicate where pc is */ fflush(stdout); /* backtrace_symbols_fd uses unbuffered i/o */ - depth = backtrace(addrs, BACKTRACE_DEPTH); - backtrace_symbols_fd(addrs, depth, fileno(stdout)); + backtrace_symbols_fd(addrs, depth + 1, fileno(stdout)); #endif if (ioOSThreadsEqual(ioCurrentOSThread(),getVMOSThread())) { @@ -870,23 +873,30 @@ } /* Attempt to dump the registers to stdout. Only do so if we know how. */ -static void +static void * printRegisterState(ucontext_t *uap) { #if __linux__ && __i386__ gregset_t *regs = &uap->uc_mcontext.gregs; - printf( "eax 0x%08x ebx 0x%08x ecx 0x%08x edx 0x%08x\n" - "edi 0x%08x esi 0x%08x ebp 0x%08x esp 0x%08x\n", + printf( "\teax 0x%08x ebx 0x%08x ecx 0x%08x edx 0x%08x\n" + "\tedi 0x%08x esi 0x%08x ebp 0x%08x esp 0x%08x\n" + "\teip 0x%08x\n", regs[REG_EAX], regs[REG_EBX], regs[REG_ECX], regs[REG_EDX], - regs[REG_EDI], regs[REG_EDI], regs[REG_EBP], regs[REG_ESP]); + regs[REG_EDI], regs[REG_EDI], regs[REG_EBP], regs[REG_ESP], + regs[REG_EIP]); + return regs[REG_EIP]; #elif __FreeBSD__ && __i386__ struct mcontext *regs = &uap->uc_mcontext; - printf( "eax 0x%08x ebx 0x%08x ecx 0x%08x edx 0x%08x\n" - "edi 0x%08x esi 0x%08x ebp 0x%08x esp 0x%08x\n", + printf( "\teax 0x%08x ebx 0x%08x ecx 0x%08x edx 0x%08x\n" + "\tedi 0x%08x esi 0x%08x ebp 0x%08x esp 0x%08x\n" + "\teip 0x%08x\n", regs->mc_eax, regs->mc_ebx, regs->mc_ecx, regs->mc_edx, - regs->mc_edi, regs->mc_edi, regs->mc_ebp, regs->mc_esp); + regs->mc_edi, regs->mc_edi, regs->mc_ebp, regs->mc_esp, + regs->mc_eip); + return regs->mc_eip; #else printf("don't know how to derive register state from a ucontext_t on this platform\n"); + return 0; #endif }

1 0

[commit][2786] Use zero-padding to print register state.
by commits＠squeakvm.org 25 Sep '13

25 Sep '13

Revision: 2786 Author: eliot Date: 2013-09-25 13:37:08 -0700 (Wed, 25 Sep 2013) Log Message: ----------- Use zero-padding to print register state. Modified Paths: -------------- branches/Cog/platforms/Mac OS/vm/sqMacMain.c branches/Cog/platforms/unix/vm/sqUnixMain.c Modified: branches/Cog/platforms/Mac OS/vm/sqMacMain.c =================================================================== --- branches/Cog/platforms/Mac OS/vm/sqMacMain.c 2013-09-25 18:55:57 UTC (rev 2785) +++ branches/Cog/platforms/Mac OS/vm/sqMacMain.c 2013-09-25 20:37:08 UTC (rev 2786) @@ -255,14 +255,14 @@ { #if __DARWIN_UNIX03 && __APPLE__ && __MACH__ && __i386__ _STRUCT_X86_THREAD_STATE32 *regs = &uap->uc_mcontext->__ss; - printf( "eax 0x%8x ebx 0x%8x ecx 0x%8x edx 0x%8x\n" - "edi 0x%8x esi 0x%8x ebp 0x%8x esp 0x%8x\n", + printf( "eax 0x%08x ebx 0x%08x ecx 0x%08x edx 0x%08x\n" + "edi 0x%08x esi 0x%08x ebp 0x%08x esp 0x%08x\n", regs->__eax, regs->__ebx, regs->__ecx, regs->__edx, regs->__edi, regs->__edi, regs->__ebp, regs->__esp); #elif __APPLE__ && __MACH__ && __i386__ _STRUCT_X86_THREAD_STATE32 *regs = &uap->uc_mcontext->ss; - printf( "eax 0x%8x ebx 0x%8x ecx 0x%8x edx 0x%8x\n" - "edi 0x%8x esi 0x%8x ebp 0x%8x esp 0x%8x\n", + printf( "eax 0x%08x ebx 0x%08x ecx 0x%08x edx 0x%08x\n" + "edi 0x%08x esi 0x%08x ebp 0x%08x esp 0x%08x\n", regs->eax, regs->ebx, regs->ecx, regs->edx, regs->edi, regs->edi, regs->ebp, regs->esp); #else Modified: branches/Cog/platforms/unix/vm/sqUnixMain.c =================================================================== --- branches/Cog/platforms/unix/vm/sqUnixMain.c 2013-09-25 18:55:57 UTC (rev 2785) +++ branches/Cog/platforms/unix/vm/sqUnixMain.c 2013-09-25 20:37:08 UTC (rev 2786) @@ -875,14 +875,14 @@ { #if __linux__ && __i386__ gregset_t *regs = &uap->uc_mcontext.gregs; - printf( "eax 0x%8x ebx 0x%8x ecx 0x%8x edx 0x%8x\n" - "edi 0x%8x esi 0x%8x ebp 0x%8x esp 0x%8x\n", + printf( "eax 0x%08x ebx 0x%08x ecx 0x%08x edx 0x%08x\n" + "edi 0x%08x esi 0x%08x ebp 0x%08x esp 0x%08x\n", regs[REG_EAX], regs[REG_EBX], regs[REG_ECX], regs[REG_EDX], regs[REG_EDI], regs[REG_EDI], regs[REG_EBP], regs[REG_ESP]); #elif __FreeBSD__ && __i386__ struct mcontext *regs = &uap->uc_mcontext; - printf( "eax 0x%8x ebx 0x%8x ecx 0x%8x edx 0x%8x\n" - "edi 0x%8x esi 0x%8x ebp 0x%8x esp 0x%8x\n", + printf( "eax 0x%08x ebx 0x%08x ecx 0x%08x edx 0x%08x\n" + "edi 0x%08x esi 0x%08x ebp 0x%08x esp 0x%08x\n", regs->mc_eax, regs->mc_ebx, regs->mc_ecx, regs->mc_edx, regs->mc_edi, regs->mc_edi, regs->mc_ebp, regs->mc_esp); #else

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev September 2013