Vm-dev November 2017

vm-dev@lists.squeakfoundation.org

16 participants
44 discussions

VM Maker: VMMaker.oscog-eem.2280.mcz
by commits＠source.squeak.org 16 Nov '17

16 Nov '17

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2280.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.2280 Author: eem Time: 15 November 2017, 3:02:00.246045 pm UUID: 7565e097-b4de-4e38-a0c4-576aaf2864f4 Ancestors: VMMaker.oscog-eem.2278 VMMaker: Fix bad misinitialization bug where running VMMaker generation routines in 64-bits would cause 64-but sources to be generated. This caused the recent misgeneration of src/vm/cogit.c. Do not inherit from VMMaker.oscog-eem.2279 which msidiagnosed the problem. Slang: Back out of the inlining optimization used in the logging methods that are now unused. Better format a method. =============== Diff against VMMaker.oscog-eem.2278 =============== Item was changed: ----- Method: BraceNode>>asTranslatorNodeIn: (in category '*VMMaker-C translation') ----- asTranslatorNodeIn: aTMethod "make a CCodeGenerator equivalent of me." "This is for case statements" + (elements allSatisfy: [:elem| elem isMessageNode and: [elem selector key = #->]]) ifTrue: - (elements allSatisfy: - [:elem| - elem isMessageNode - and: [elem selector key = #->]]) ifTrue: [self assert: (elements allSatisfy: [:elem| elem receiver isBlockNode and: [elem arguments first isBlockNode and: [elem receiver isPotentialCCaseLabelIn: aTMethod]]]). ^TBraceCaseNode new caseLabels: (elements collect: [:elem| elem receiver asTranslatorNodeIn: aTMethod]); cases: (elements collect: [:elem| elem arguments first asTranslatorNodeIn: aTMethod]); comment: comment]. "This is for varargs selectors (variants of printf:)" ^elements collect: [:elem| elem asTranslatorNodeIn: aTMethod]! Item was changed: ----- Method: TMethod>>inlineSend:directReturn:exitVar:in: (in category 'inlining') ----- inlineSend: aSendNode directReturn: directReturn exitVar: exitVar in: aCodeGen "Answer a collection of statements to replace the given send. directReturn indicates that the send is the expression in a return statement, so returns can be left in the body of the inlined method. If exitVar is nil, the value returned by the send is not used; thus, returns need not assign to the output variable. Types are propagated to as-yet-untyped variables when inlining a send that is assigned, otherwise the assignee variable type must match the return type of the inlinee. Return types are not propagated." + | sel meth methArgs exitLabel inlineStmts label exitType elidedArgs | - | sel meth methArgs exitLabel inlineStmts exitType elidedArgs argAssignments conditional | sel := aSendNode selector. meth := aCodeGen methodNamed: sel. methArgs := meth args. "convenient for debugging..." aCodeGen maybeBreakForInlineOf: aSendNode in: self. elidedArgs := #(). (methArgs notEmpty and: [methArgs first beginsWith: 'self_in_']) ifTrue: "If the first arg is not used we can and should elide it." [| varNode | varNode := TVariableNode new setName: methArgs first. (meth parseTree noneSatisfy: [:node| varNode isSameAs: node]) ifTrue: [elidedArgs := {methArgs first}]. methArgs := methArgs allButFirst]. methArgs size = aSendNode args size ifFalse: [^nil]. meth := meth copy. (meth statements size > 1 and: [meth statements first isSend and: [meth statements first selector == #flag:]]) ifTrue: [meth statements removeFirst]. "Propagate the return type of an inlined method" (directReturn or: [exitVar notNil]) ifTrue: [exitType := directReturn ifTrue: [returnType] ifFalse: [(self typeFor: exitVar in: aCodeGen) ifNil: [#sqInt]]. (exitType = #void or: [exitType = meth returnType]) ifFalse: [meth propagateReturnIn: aCodeGen]]. "Propagate any unusual argument types to untyped argument variables" methArgs with: aSendNode args do: [:formal :actual| (meth declarationAt: formal ifAbsent: nil) ifNil: [(self typeFor: actual in: aCodeGen) ifNotNil: [:type| type ~= #sqInt ifTrue: [meth declarationAt: formal put: (type last = $* ifTrue: [type, formal] ifFalse: [type, ' ', formal])]]]]. meth renameVarsForInliningInto: self except: elidedArgs in: aCodeGen. meth renameLabelsForInliningInto: self. self addVarsDeclarationsAndLabelsOf: meth except: elidedArgs. meth hasReturn ifTrue: [directReturn ifFalse: [exitLabel := self unusedLabelForInliningInto: self. (meth exitVar: exitVar label: exitLabel) "is label used?" ifTrue: [ labels add: exitLabel ] ifFalse: [ exitLabel := nil ]]]. (inlineStmts := OrderedCollection new: meth statements size + meth args size + 2) + add: (label := TLabeledCommentNode new setComment: 'begin ', sel); + addAll: (self argAssignmentsFor: meth send: aSendNode except: elidedArgs in: aCodeGen); + addAll: meth statements. "method body" - add: (TLabeledCommentNode new setComment: 'begin ', sel). - argAssignments := self argAssignmentsFor: meth send: aSendNode except: elidedArgs in: aCodeGen. - "If the method being inlined is of the form - guard ifTrue: [...] - and the argument assignments have no side effects, - and guard does not refer to the arguments, - push the argument assignments past the guard into the block so that they too are guarded." - (meth statements size = 1 - and: [(conditional := meth statements first) isConditionalSend - and: [conditional numArgs = 1 - and: [(argAssignments noneSatisfy: [:assign| assign expression hasSideEffect]) - and: [conditional receiver noneSatisfy: [:node| node isVariable and: [methArgs includes: node name]]]]]]) - ifTrue: [conditional args first statements addAllFirst: argAssignments] - ifFalse: [inlineStmts addAll: argAssignments]. - inlineStmts addAll: meth statements. "method body" directReturn ifTrue: [meth endsWithReturn ifTrue: [exitVar ifNotNil: "don't remove the returns if being invoked in the context of a return" [inlineStmts at: inlineStmts size put: inlineStmts last copyWithoutReturn]] ifFalse: [inlineStmts add: (TReturnNode new setExpression: (TVariableNode new setName: 'nil'))]]. exitLabel ifNotNil: [inlineStmts add: (TLabeledCommentNode new setLabel: exitLabel comment: 'end ', meth selector)]. inlineStmts size = 1 ifTrue: "Nuke empty methods; e.g. override of flushAtCache" [self assert: inlineStmts first isComment. inlineStmts removeFirst]. ^inlineStmts! Item was changed: ----- Method: VMMaker>>initialize (in category 'initialize') ----- initialize logger := Transcript. inline := true. forBrowser := false. internalPlugins := SortedCollection new. externalPlugins := SortedCollection new. platformName := self class machinesDirName. + is64BitVM := false. - is64BitVM := Smalltalk wordSize == 8. interpreterClassName := Interpreter name. + optionsDictionary := Dictionary newFromPairs: {#BytesPerWord. 4}. - optionsDictionary := Dictionary new. - optionsDictionary at: #BytesPerWord put: Smalltalk wordSize. VMStructType voidStructTypeCache!

1 0

[OpenSmalltalk/opensmalltalk-vm] 21ad7c: Cog V3 source as per VMMaker.oscog-eem.2279.
by GitHub 15 Nov '17

15 Nov '17

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: 21ad7cb6b78add15a65f27f0346652ee992daa57 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/21ad7cb6b78add15a6… Author: Eliot Miranda <eliot.miranda(a)gmail.com> Date: 2017-11-15 (Wed, 15 Nov 2017) Changed paths: M src/vm/cogit.c Log Message: ----------- Cog V3 source as per VMMaker.oscog-eem.2279. Undo the damage to src/vm/cogit.c done by VMMaker.oscog-eem.2278.

1 0

VM Maker: VMMaker.oscog-eem.2279.mcz
by commits＠source.squeak.org 15 Nov '17

15 Nov '17

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2279.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.2279 Author: eem Time: 15 November 2017, 2:26:14.871383 pm UUID: ec78abc1-3128-47b0-bff8-25bab14bae2d Ancestors: VMMaker.oscog-eem.2278 VMMaker: Fix bad misinitialization bug where options from a previous translation were not clearedf out. This caused the recent misgeneration of src/vm/cogit.c Slang: Back out of the inlining optimization used in the logging methods that are now unused. Better format a method. =============== Diff against VMMaker.oscog-eem.2278 =============== Item was changed: ----- Method: BraceNode>>asTranslatorNodeIn: (in category '*VMMaker-C translation') ----- asTranslatorNodeIn: aTMethod "make a CCodeGenerator equivalent of me." "This is for case statements" + (elements allSatisfy: [:elem| elem isMessageNode and: [elem selector key = #->]]) ifTrue: - (elements allSatisfy: - [:elem| - elem isMessageNode - and: [elem selector key = #->]]) ifTrue: [self assert: (elements allSatisfy: [:elem| elem receiver isBlockNode and: [elem arguments first isBlockNode and: [elem receiver isPotentialCCaseLabelIn: aTMethod]]]). ^TBraceCaseNode new caseLabels: (elements collect: [:elem| elem receiver asTranslatorNodeIn: aTMethod]); cases: (elements collect: [:elem| elem arguments first asTranslatorNodeIn: aTMethod]); comment: comment]. "This is for varargs selectors (variants of printf:)" ^elements collect: [:elem| elem asTranslatorNodeIn: aTMethod]! Item was changed: ----- Method: TMethod>>inlineSend:directReturn:exitVar:in: (in category 'inlining') ----- inlineSend: aSendNode directReturn: directReturn exitVar: exitVar in: aCodeGen "Answer a collection of statements to replace the given send. directReturn indicates that the send is the expression in a return statement, so returns can be left in the body of the inlined method. If exitVar is nil, the value returned by the send is not used; thus, returns need not assign to the output variable. Types are propagated to as-yet-untyped variables when inlining a send that is assigned, otherwise the assignee variable type must match the return type of the inlinee. Return types are not propagated." + | sel meth methArgs exitLabel inlineStmts label exitType elidedArgs | - | sel meth methArgs exitLabel inlineStmts exitType elidedArgs argAssignments conditional | sel := aSendNode selector. meth := aCodeGen methodNamed: sel. methArgs := meth args. "convenient for debugging..." aCodeGen maybeBreakForInlineOf: aSendNode in: self. elidedArgs := #(). (methArgs notEmpty and: [methArgs first beginsWith: 'self_in_']) ifTrue: "If the first arg is not used we can and should elide it." [| varNode | varNode := TVariableNode new setName: methArgs first. (meth parseTree noneSatisfy: [:node| varNode isSameAs: node]) ifTrue: [elidedArgs := {methArgs first}]. methArgs := methArgs allButFirst]. methArgs size = aSendNode args size ifFalse: [^nil]. meth := meth copy. (meth statements size > 1 and: [meth statements first isSend and: [meth statements first selector == #flag:]]) ifTrue: [meth statements removeFirst]. "Propagate the return type of an inlined method" (directReturn or: [exitVar notNil]) ifTrue: [exitType := directReturn ifTrue: [returnType] ifFalse: [(self typeFor: exitVar in: aCodeGen) ifNil: [#sqInt]]. (exitType = #void or: [exitType = meth returnType]) ifFalse: [meth propagateReturnIn: aCodeGen]]. "Propagate any unusual argument types to untyped argument variables" methArgs with: aSendNode args do: [:formal :actual| (meth declarationAt: formal ifAbsent: nil) ifNil: [(self typeFor: actual in: aCodeGen) ifNotNil: [:type| type ~= #sqInt ifTrue: [meth declarationAt: formal put: (type last = $* ifTrue: [type, formal] ifFalse: [type, ' ', formal])]]]]. meth renameVarsForInliningInto: self except: elidedArgs in: aCodeGen. meth renameLabelsForInliningInto: self. self addVarsDeclarationsAndLabelsOf: meth except: elidedArgs. meth hasReturn ifTrue: [directReturn ifFalse: [exitLabel := self unusedLabelForInliningInto: self. (meth exitVar: exitVar label: exitLabel) "is label used?" ifTrue: [ labels add: exitLabel ] ifFalse: [ exitLabel := nil ]]]. (inlineStmts := OrderedCollection new: meth statements size + meth args size + 2) + add: (label := TLabeledCommentNode new setComment: 'begin ', sel); + addAll: (self argAssignmentsFor: meth send: aSendNode except: elidedArgs in: aCodeGen); + addAll: meth statements. "method body" - add: (TLabeledCommentNode new setComment: 'begin ', sel). - argAssignments := self argAssignmentsFor: meth send: aSendNode except: elidedArgs in: aCodeGen. - "If the method being inlined is of the form - guard ifTrue: [...] - and the argument assignments have no side effects, - and guard does not refer to the arguments, - push the argument assignments past the guard into the block so that they too are guarded." - (meth statements size = 1 - and: [(conditional := meth statements first) isConditionalSend - and: [conditional numArgs = 1 - and: [(argAssignments noneSatisfy: [:assign| assign expression hasSideEffect]) - and: [conditional receiver noneSatisfy: [:node| node isVariable and: [methArgs includes: node name]]]]]]) - ifTrue: [conditional args first statements addAllFirst: argAssignments] - ifFalse: [inlineStmts addAll: argAssignments]. - inlineStmts addAll: meth statements. "method body" directReturn ifTrue: [meth endsWithReturn ifTrue: [exitVar ifNotNil: "don't remove the returns if being invoked in the context of a return" [inlineStmts at: inlineStmts size put: inlineStmts last copyWithoutReturn]] ifFalse: [inlineStmts add: (TReturnNode new setExpression: (TVariableNode new setName: 'nil'))]]. exitLabel ifNotNil: [inlineStmts add: (TLabeledCommentNode new setLabel: exitLabel comment: 'end ', meth selector)]. inlineStmts size = 1 ifTrue: "Nuke empty methods; e.g. override of flushAtCache" [self assert: inlineStmts first isComment. inlineStmts removeFirst]. ^inlineStmts! Item was changed: ----- Method: VMMaker>>options: (in category 'initialize') ----- options: anArrayOfPairs + optionsDictionary removeAll. self assert: anArrayOfPairs size even. 1 to: anArrayOfPairs size by: 2 do: [:i| | key | key := anArrayOfPairs at: i. self assert: key isSymbol. optionsDictionary at: key put: (anArrayOfPairs at: i + 1)]!

1 0

[OpenSmalltalk/opensmalltalk-vm] 1dd1a9: CogVM source as per VMMaker.oscog-eem.2278
by GitHub 14 Nov '17

14 Nov '17

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: 1dd1a9c0d61ed1c367fb226e59ce0386af3bf5ed https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/1dd1a9c0d61ed1c367… Author: Eliot Miranda <eliot.miranda(a)gmail.com> Date: 2017-11-14 (Tue, 14 Nov 2017) Changed paths: M nsspur64src/vm/cogit.h M nsspur64src/vm/cointerp.c M nsspur64src/vm/cointerp.h M nsspur64src/vm/gcc3x-cointerp.c M nsspursrc/vm/cogit.h M nsspursrc/vm/cointerp.c M nsspursrc/vm/cointerp.h M nsspursrc/vm/gcc3x-cointerp.c M nsspurstack64src/vm/gcc3x-interp.c M nsspurstack64src/vm/interp.c M nsspurstacksrc/vm/gcc3x-interp.c M nsspurstacksrc/vm/interp.c M spur64src/vm/cogit.h M spur64src/vm/cointerp.c M spur64src/vm/cointerp.h M spur64src/vm/gcc3x-cointerp.c M spur64src/vm/interp.h M spurlowcode64src/vm/cogit.h M spurlowcode64src/vm/cogitX64SysV.c M spurlowcode64src/vm/cogitX64WIN64.c M spurlowcode64src/vm/cointerp.c M spurlowcode64src/vm/cointerp.h M spurlowcode64src/vm/gcc3x-cointerp.c M spurlowcodesrc/vm/cogit.h M spurlowcodesrc/vm/cogitARMv5.c M spurlowcodesrc/vm/cogitIA32.c M spurlowcodesrc/vm/cogitMIPSEL.c M spurlowcodesrc/vm/cointerp.c M spurlowcodesrc/vm/cointerp.h M spurlowcodesrc/vm/gcc3x-cointerp.c M spurlowcodestack64src/vm/gcc3x-interp.c M spurlowcodestack64src/vm/interp.c M spurlowcodestacksrc/vm/gcc3x-interp.c M spurlowcodestacksrc/vm/interp.c M spursista64src/vm/cogit.h M spursista64src/vm/cointerp.c M spursista64src/vm/cointerp.h M spursista64src/vm/gcc3x-cointerp.c M spursista64src/vm/interp.h M spursistasrc/vm/cogit.h M spursistasrc/vm/cointerp.c M spursistasrc/vm/cointerp.h M spursistasrc/vm/gcc3x-cointerp.c M spursistasrc/vm/interp.h M spursrc/vm/cogit.h M spursrc/vm/cointerp.c M spursrc/vm/cointerp.h M spursrc/vm/gcc3x-cointerp.c M spursrc/vm/interp.h M spurstack64src/vm/gcc3x-interp.c M spurstack64src/vm/interp.c M spurstacksrc/vm/gcc3x-interp.c M spurstacksrc/vm/interp.c M src/plugins/B2DPlugin/B2DPlugin.c M src/plugins/B3DAcceleratorPlugin/B3DAcceleratorPlugin.c M src/plugins/BitBltPlugin/BitBltPlugin.c M src/plugins/CroquetPlugin/CroquetPlugin.c M src/plugins/DSAPrims/DSAPrims.c M src/plugins/FFTPlugin/FFTPlugin.c M src/plugins/FilePlugin/FilePlugin.c M src/plugins/GeniePlugin/GeniePlugin.c M src/plugins/IA32ABI/IA32ABI.c M src/plugins/JPEGReadWriter2Plugin/JPEGReadWriter2Plugin.c M src/plugins/JPEGReaderPlugin/JPEGReaderPlugin.c M src/plugins/LargeIntegers/LargeIntegers.c M src/plugins/MiscPrimitivePlugin/MiscPrimitivePlugin.c M src/plugins/ScratchPlugin/ScratchPlugin.c M src/plugins/Squeak3D/Squeak3D.c M src/plugins/SqueakFFIPrims/ARM32FFIPlugin.c M src/plugins/SqueakFFIPrims/IA32FFIPlugin.c M src/plugins/SqueakFFIPrims/X64SysVFFIPlugin.c M src/plugins/SqueakFFIPrims/X64Win64FFIPlugin.c M src/plugins/VMProfileLinuxSupportPlugin/VMProfileLinuxSupportPlugin.c M src/plugins/VMProfileMacSupportPlugin/VMProfileMacSupportPlugin.c M src/plugins/Win32OSProcessPlugin/Win32OSProcessPlugin.c M src/plugins/XDisplayControlPlugin/XDisplayControlPlugin.c M src/plugins/ZipPlugin/ZipPlugin.c M src/vm/cogit.c M src/vm/cogit.h M src/vm/cogmethod.h M src/vm/cointerp.c M src/vm/cointerp.h M src/vm/cointerpmt.c M src/vm/cointerpmt.h M src/vm/gcc3x-cointerp.c M src/vm/gcc3x-cointerpmt.c M src/vm/interp.h M stacksrc/vm/gcc3x-interp.c M stacksrc/vm/interp.c M stacksrc/vm/interp.h Log Message: ----------- CogVM source as per VMMaker.oscog-eem.2278 Spur: Revise scavenger logging to collect data during the scavenge and log it to the file after the scavenge, hence including an accurate elapsed time. Plugins: Regenerate plugins after type inference fixes from VMMaker.oscog-eem.2243 and VMMaker.oscog-eem.2266. Slang: Output the printf format string using the LP64/LLP64 convention in sqMemoryAccess.h

1 0

VM Maker: VMMaker.oscog-eem.2278.mcz
by commits＠source.squeak.org 14 Nov '17

14 Nov '17

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2278.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.2278 Author: eem Time: 13 November 2017, 4:18:45.220569 pm UUID: 4fd41cee-a3bd-48f4-b02e-cf367a331eec Ancestors: VMMaker.oscog-eem.2277 Spur: Revise scavenger logging to collect data during the scavenge and log it to the file after the scavenge, hence including an accurate elapsed time. Consequently nuke the log: methods in VMClass; they arent needed, which neatly avoids the issue of multiple classes needing to define logStream. Slang: Output the printf format string using the LP64/LLP64 convention in sqMemoryAccess.h =============== Diff against VMMaker.oscog-eem.2277 =============== Item was changed: ----- Method: BraceNode>>asTranslatorNodeIn: (in category '*VMMaker-C translation') ----- asTranslatorNodeIn: aTMethod "make a CCodeGenerator equivalent of me." - "This is for varargs selectors (variants of printf:)" - (elements notEmpty and: [elements allSatisfy: [:e| e isTemp]]) ifTrue: - [^elements collect: [:elem| elem asTranslatorNodeIn: aTMethod]]. - "This is for case statements" + (elements allSatisfy: - self assert: (elements allSatisfy: [:elem| elem isMessageNode + and: [elem selector key = #->]]) ifTrue: + [self assert: (elements allSatisfy: + [:elem| + elem receiver isBlockNode + and: [elem arguments first isBlockNode + and: [elem receiver isPotentialCCaseLabelIn: aTMethod]]]). + ^TBraceCaseNode new + caseLabels: (elements collect: [:elem| elem receiver asTranslatorNodeIn: aTMethod]); + cases: (elements collect: [:elem| elem arguments first asTranslatorNodeIn: aTMethod]); + comment: comment]. + + "This is for varargs selectors (variants of printf:)" + ^elements collect: [:elem| elem asTranslatorNodeIn: aTMethod]! - and: [elem selector key = #-> - and: [elem receiver isBlockNode - and: [elem arguments first isBlockNode - and: [elem receiver isPotentialCCaseLabelIn: aTMethod]]]]]). - ^TBraceCaseNode new - caseLabels: (elements collect: [:elem| elem receiver asTranslatorNodeIn: aTMethod]); - cases: (elements collect: [:elem| elem arguments first asTranslatorNodeIn: aTMethod]); - comment: comment! Item was added: + ----- Method: CCodeGenerator>>cLiteralForPrintfString: (in category 'C code generator') ----- + cLiteralForPrintfString: aString + ^(('"', (PrintfFormatString new setFormat: aString) transformForVMMaker, '"') + copyReplaceAll: (String with: Character cr) with: '\n') + copyReplaceAll: (String with: Character tab) with: '\t'! Item was added: + ----- Method: PrintfFormatDescriptor>>transformForVMMaker (in category '*VMMaker-C code generation') ----- + transformForVMMaker + ^self printString! Item was added: + ----- Method: PrintfFormatString>>transformForVMMaker (in category '*VMMaker-C code generation') ----- + transformForVMMaker + [outputStream nextPutAll: string next. + format atEnd] whileFalse: + [outputStream nextPutAll: format next transformForVMMaker]. + "any remainder is string, if so append to outputStream" + string atEnd ifFalse: [outputStream nextPutAll: string next]. + ^outputStream contents! Item was added: + ----- Method: PrintfNumberFormatDescriptor>>transformForVMMaker (in category '*VMMaker-C code generation') ----- + transformForVMMaker + ('duxX' includes: operator) ifFalse: + [^super transformForVMMaker]. + ^'%" PRI', (String with: operator), 'SQINT "'! Item was changed: CogClass subclass: #SpurGenerationScavenger + instanceVariableNames: 'coInterpreter manager eden futureSpace pastSpace futureSurvivorStart rememberedSet rememberedSetSize previousRememberedSetSize rememberedSetRedZone rememberedSetLimit refCountToShrinkRT weakList ephemeronList tenureCriterion tenureThreshold tenuringClassIndex tenuringProportion numRememberedEphemerons scavengeLog scavengeLogRecord statSurvivorCount statTenures' - instanceVariableNames: 'coInterpreter manager eden futureSpace pastSpace futureSurvivorStart rememberedSet rememberedSetSize previousRememberedSetSize rememberedSetRedZone rememberedSetLimit refCountToShrinkRT weakList ephemeronList tenureCriterion tenureThreshold tenuringClassIndex tenuringProportion numRememberedEphemerons scavengeLog statSurvivorCount statTenures' classVariableNames: '' poolDictionaries: 'SpurMemoryManagementConstants' category: 'VMMaker-SpurMemoryManager'! !SpurGenerationScavenger commentStamp: 'eem 11/7/2017 17:26' prior: 0! SpurGenerationScavenger is an implementation of David Ungar's Generation Scavenging garbage collection algorithm. See Generation Scavenging, A Non-disruptive, High-Performance Storage Reclamation Algorithm David Ungar Proceeding SDE 1 Proceedings of the first ACM SIGSOFT/SIGPLAN software engineering symposium on Practical software development environments Pages 157 - 167 ACM New York, NY, USA ©1984 Also relevant are An adaptive tenuring policy for generation scavengers David Ungar & Frank Jackson ACM Transactions on Programming Languages and Systems (TOPLAS) TOPLAS Homepage archive Volume 14 Issue 1, Jan. 1992 Pages 1 - 27 ACM New York, NY, USA ©1992 and Ephemerons: a new finalization mechanism Barry Hayes Proceedings of the 12th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications Pages 176-183 ACM New York, NY, USA ©1997 See text below the variable definitions and explanation below for a full explanation of weak and ephemeron processing. Instance Variables coInterpreter: <StackInterpreterSimulator|CogVMSimulator> eden: <SpurNewSpaceSpace> ephemeronList: <Integer|nil> futureSpace: <SpurNewSpaceSpace> futureSurvivorStart: <Integer address> manager: <SpurMemoryManager|Spur32BitMMLESimulator et al> numRememberedEphemerons: <Integer> pastSpace: <SpurNewSpaceSpace> previousRememberedSetSize: <Integer> rememberedSet: <CArrayAccessor on: Array> rememberedSetSize: <Integer> tenuringProportion: <Float> tenuringThreshold: <Integer address> weakList: <Integer|nil> coInterpreter - the interpreter/vm, in this context, the mutator manager - the Spur memory manager eden - the space containing newly created objects futureSpace - the space to which surviving objects are copied during a scavenge futureSurvivorStart - the allocation pointer into futureSpace pastSpace - the space surviving objects live in until the next scavenge rememberedSet - the root old space objects that refer to objects in new space; a scavenge starts form these roots and the interpreter's stack rememberedSetSize - the size of the remembered set, also the first unused index in the rememberedSet previousRememberedSetSize: - the size of the remembered set before scavenging objects in future space. numRememberedEphemerons - the number of unscavenged ephemerons at the front of the rememberedSet. ephemeronList - the head of the list of corpses of unscavenged ephemerons reached in the current phase weakList - the head of the list of corpses of weak arrays reached during the scavenge. tenuringProportion - the amount of pastSpace below which the system will not tenure unless futureSpace fills up, and above which it will eagerly tenure tenuringThreshold - the pointer into pastSpace below which objects will be tenured Weakness and Ephemerality in the Scavenger. Weak arrays should not hold onto their referents (except from their strong fileds, their named inst vars). Ephemerons are objects that implement instance-based finalization; attaching an ephemeron to an object keeps that object alive and causes the ephemeron to "fire" when the object is only reachable from the ephemeron (or other ephemerons & weak arrays). They are a special kind of Associations that detect when their keys are about to die, i.e. when an ephemeron's key is not reachable from the roots except from weak arrays and other ephemerons with about-to-die keys. Note that if an ephemeron's key is not about to die then references from the rest of the ephemeron can indeed prevent ephemeron keys from dying. The scavenger is concerned with collecting objects in new space, therefore it ony deals with weak arrays and ephemerons that are either in the remembered set or in new space. By deferring scanning these objects until other reachable objects have been scavenged, the scavenger can detect dead or dying references. Weak Array Processing In the case of weak arrays this is simple. The scavenger refuses to scavenge the referents of weak arrays in scavengeReferentsOf: until the entire scavenge is over. It then scans the weak arrays in the remembered set and in future space and nils all fields in them that are referring to unforwarded objects in eden and past space, because these objects have not survived the scavenge. The root weak arrays remaining to be scavenged are in the remembered table. Surviving weak arrays in future space are collected on a list. The list is threaded through the corpses of weak arrays in eden and/or past space. weakList holds the slot offset of the first weak array found in eden and/or past space. The next offset is stored in the weak array corpse's identityHash and format fields (22 bits & 5 bits of allocationUnits, for a max new space size of 2^28 bytes, 256Mb). The list is threaded throguh corpses, but the surviving arrays are pointed to by the corpses' forwarding pointers. Ephemeron Processing The case of ephemerons is a little more complicated because an ephemeron's key should survive. The scavenger is cyclical. It scavenges the remembered set, which may copy and forward surviving objects in past and/or eden spaces to future space. It then scavenges those promoted objects in future space until no more are promoted, which may in turn remember more objects. The cycles continue until no more objects get promoted to future space and no more objects get remembered. At this point all surviving objecta are in futureSpace. So if the scavenger does not scan ephemerons in the remembered set or in future space until the scavenger finishes cycling, it can detect ephemerons whose keys are about to die because these will be unforwarded objects in eden and/or past space. Ephemerons encountered in the remembered set are either processed like ordinary objects if their keys have been promoted to futureSpace, or are moved to the front of the rememberedSet (because, dear reader, it is a sequence) if their keys have not been promoted. Ephemerons encountered in scavengeReferentsOf: are either scanned like normal objects if their keys have been promoted, or added to the ephemeronList, organized identically to the weakList, if their keys are yet to be promoted. Since references from other ephemerons with surviving keys to ephemeron keys can and should prevent the ephemerons whose keys they are from firing the scavenger does not fire ephemerons unless all unscavenged ephemerons have unscavenged keys. So the unsca venged ephemerons (they will be at the beginning of the remembered set and on the ephemeronList) are scanned and any that have promoted keys are scavenged. But if no unscavenged ephemerons have surviving keys then all the unscavenged ephemerons are fired and then scavenged. This in turn may remember more objects and promote more objects to future space, and encounter more unscavenged ephemerons. So the scavenger continues until no more objects are remembered, no more objects are promoted to future space and no more unscavenged ephemerons exist.! Item was changed: ----- Method: SpurGenerationScavenger class>>declareCVarsIn: (in category 'translation') ----- declareCVarsIn: aCCodeGenerator #(eden futureSpace pastSpace) do: [:var| aCCodeGenerator var: var type: #SpurNewSpaceSpace]. aCCodeGenerator var: #rememberedSet type: #'sqInt *'; var: #tenuringProportion type: #double; + var: #scavengeLogRecord type: #SpurScavengeLogRecord; var: #scavengeLog type: #'FILE *'! Item was changed: ----- Method: SpurGenerationScavenger>>initialize (in category 'initialization') ----- initialize pastSpace := SpurNewSpaceSpace new. futureSpace := SpurNewSpaceSpace new. eden := SpurNewSpaceSpace new. rememberedSetSize := 0. tenureThreshold := 0. + statSurvivorCount := statTenures := 0. + scavengeLogRecord := SpurScavengeLogRecord new! - statSurvivorCount := statTenures := 0! Item was changed: ----- Method: SpurGenerationScavenger>>logEndScavenge (in category 'logging') ----- logEndScavenge <inline: #always> + scavengeLogRecord + eSurvivorBytes: futureSurvivorStart - pastSpace start; + eRememberedSetSize: rememberedSetSize; + eStatTenures: statTenures! - self log: ' survivor bytes: 0x%lx/%ld remembered objects: %ld tenured: %ld\n' - with: futureSurvivorStart - pastSpace start - with: futureSurvivorStart - pastSpace start - with: rememberedSetSize - with: statTenures; - flushLog! Item was added: + ----- Method: SpurGenerationScavenger>>logScavenge (in category 'logging') ----- + logScavenge + <inline: #always> + scavengeLog ifNotNil: + [self writeScavengeLog]! Item was changed: ----- Method: SpurGenerationScavenger>>logStartScavenge (in category 'logging') ----- logStartScavenge <inline: #always> + scavengeLogRecord + sEdenBytes: manager freeStart - eden start; + sPastBytes: manager pastSpaceStart - pastSpace start; + sRememberedSetSize: rememberedSetSize; + sRememberedSetRedZone: rememberedSetRedZone; + sRememberedSetLimit: rememberedSetLimit; + sStatTenures: statTenures! - self log: 'scavenge %ld eden bytes: 0x%lx/%ld past bytes: 0x%lx/%ld\n\tremembered set: %ld redzone: %ld size: %ld\n' - with: manager statScavenges - with: manager freeStart - eden start - with: manager freeStart - eden start - with: manager pastSpaceStart - pastSpace start - with: manager pastSpaceStart - pastSpace start - with: rememberedSetSize - with: rememberedSetRedZone - with: rememberedSetLimit! Item was changed: ----- Method: SpurGenerationScavenger>>logTenuringPolicy (in category 'logging') ----- logTenuringPolicy <inline: #always> + scavengeLogRecord + tTenureCriterion: tenureCriterion; + tTenureThreshold: ((tenureCriterion = TenureByAge and: [tenureThreshold > pastSpace start]) + ifTrue: [tenureThreshold - pastSpace start] + ifFalse: [0]); + tRefCountToShrinkRT: refCountToShrinkRT! - | policyNames | - <var: 'policyNames' declareC: 'static char *policyNames[] = {"", "by age", "by class", "to shrink rt", "don''t tenure", "mark on tenure"}'> - self cCode: [] - inSmalltalk: [policyNames := CLiteralArray on: #('' 'by age' 'by class' 'to shrink rt' 'don''t tenure' 'mark on tenure')]. - self log: (tenureCriterion = TenureToShrinkRT ifFalse: [' tenure below 0x%lx/%ld %s\n'] ifTrue: [' tenure below 0x%lx/%ld %s refct %ld\n']) - with: ((tenureCriterion = TenureByAge and: [tenureThreshold > pastSpace start]) ifTrue: [tenureThreshold - pastSpace start] ifFalse: [0]) - with: ((tenureCriterion = TenureByAge and: [tenureThreshold > pastSpace start]) ifTrue: [tenureThreshold - pastSpace start] ifFalse: [0]) - with: (policyNames at: tenureCriterion) - with: refCountToShrinkRT! Item was added: + ----- Method: SpurGenerationScavenger>>writeScavengeLog (in category 'logging') ----- + writeScavengeLog + "Output the entire record." + <inline: #never> + | policyNames | + <var: 'policyNames' declareC: 'static char *policyNames[] = {"", "by age", "by class", "to shrink rt", "don''t tenure", "mark on tenure"}'> + self cCode: [] + inSmalltalk: [policyNames := CLiteralArray on: #('' 'by age' 'by class' 'to shrink rt' 'don''t tenure' 'mark on tenure')]. + scavengeLog "log data collected by logStartScavenge" + f: 'scavenge %ld eden bytes: 0x%lx/%ld past bytes: 0x%lx/%ld\n\trem set: %ld redzone: %ld size: %ld\n' + printf:{ manager statScavenges. + scavengeLogRecord sEdenBytes. + scavengeLogRecord sEdenBytes. + scavengeLogRecord sPastBytes. + scavengeLogRecord sPastBytes. + scavengeLogRecord sRememberedSetSize. + scavengeLogRecord sRememberedSetRedZone. + scavengeLogRecord sRememberedSetLimit }. + scavengeLog "log data collected by logTenuringPolicy" + f: (scavengeLogRecord tTenureCriterion = TenureToShrinkRT + ifFalse: [' tenure below 0x%lx/%ld %s\n'] + ifTrue: [' tenure below 0x%lx/%ld %s refct %ld\n']) + printf:{ scavengeLogRecord tTenureThreshold. + scavengeLogRecord tTenureThreshold. + policyNames at: scavengeLogRecord tTenureCriterion. + scavengeLogRecord tRefCountToShrinkRT }. + scavengeLog "log data collected by logEndScavenge" + f: ' survivor bytes: 0x%lx/%ld rem set: %ld tenured: %ld usecs: %ld\n' + printf:{ scavengeLogRecord eSurvivorBytes. + scavengeLogRecord eSurvivorBytes. + scavengeLogRecord eRememberedSetSize. + scavengeLogRecord eStatTenures - scavengeLogRecord sStatTenures. + manager statSGCDeltaUsecs asUnsignedLong }. + scavengeLog fflush! Item was changed: ----- Method: SpurMemoryManager class>>ancilliaryClasses: (in category 'translation') ----- ancilliaryClasses: options initializationOptions ifNil: [initializationOptions := options]. + ^{ SpurGenerationScavenger. SpurScavengeLogRecord. SpurSegmentManager. SpurSegmentInfo. self compactorClass }, - ^{ SpurGenerationScavenger. SpurSegmentManager. SpurSegmentInfo. self compactorClass }, SpurNewSpaceSpace withAllSubclasses! Item was changed: ----- Method: SpurMemoryManager>>scavengingGCTenuringIf: (in category 'gc - scavenging') ----- scavengingGCTenuringIf: tenuringCriterion "Run the scavenger." <inline: false> self assert: remapBufferCount = 0. (self asserta: scavenger eden limit - freeStart > coInterpreter interpreterAllocationReserveBytes) ifFalse: [coInterpreter tab; printNum: scavenger eden limit - freeStart; space; printNum: coInterpreter interpreterAllocationReserveBytes; space; printNum: coInterpreter interpreterAllocationReserveBytes - (scavenger eden limit - freeStart); cr]. self checkMemoryMap. self checkFreeSpace: GCModeNewSpace. self runLeakCheckerFor: GCModeNewSpace. coInterpreter preGCAction: GCModeNewSpace; "would prefer this to be in mapInterpreterOops, but compatibility with ObjectMemory dictates it goes here." flushMethodCacheFrom: newSpaceStart to: newSpaceLimit. needGCFlag := false. gcStartUsecs := coInterpreter ioUTCMicrosecondsNow. self doScavenge: tenuringCriterion. statScavenges := statScavenges + 1. statGCEndUsecs := coInterpreter ioUTCMicrosecondsNow. statSGCDeltaUsecs := statGCEndUsecs - gcStartUsecs. statScavengeGCUsecs := statScavengeGCUsecs + statSGCDeltaUsecs. statRootTableCount := scavenger rememberedSetSize. + scavenger logScavenge. + coInterpreter postGCAction: GCModeNewSpace. self runLeakCheckerFor: GCModeNewSpace. self checkFreeSpace: GCModeNewSpace! Item was added: + VMStructType subclass: #SpurScavengeLogRecord + instanceVariableNames: 'sEdenBytes sPastBytes sRememberedSetSize sRememberedSetRedZone sRememberedSetLimit sStatTenures tTenureCriterion tTenureThreshold tRefCountToShrinkRT eSurvivorBytes eRememberedSetSize eStatTenures' + classVariableNames: '' + poolDictionaries: '' + category: 'VMMaker-SpurMemoryManager'! Item was added: + ----- Method: SpurScavengeLogRecord class>>instVarNamesAndTypesForTranslationDo: (in category 'translation') ----- + instVarNamesAndTypesForTranslationDo: aBinaryBlock + self allInstVarNames do: + [:ivn| + (SpurMemoryManager isNonArgumentImplicitReceiverVariableName: ivn) ifFalse: + [aBinaryBlock value: ivn value: #usqInt]]! Item was added: + ----- Method: SpurScavengeLogRecord>>eRememberedSetSize (in category 'accessing') ----- + eRememberedSetSize + + ^ eRememberedSetSize + ! Item was added: + ----- Method: SpurScavengeLogRecord>>eRememberedSetSize: (in category 'accessing') ----- + eRememberedSetSize: anObject + + ^ eRememberedSetSize := anObject. + ! Item was added: + ----- Method: SpurScavengeLogRecord>>eStatTenures (in category 'accessing') ----- + eStatTenures + + ^ eStatTenures + ! Item was added: + ----- Method: SpurScavengeLogRecord>>eStatTenures: (in category 'accessing') ----- + eStatTenures: anObject + + ^ eStatTenures := anObject. + ! Item was added: + ----- Method: SpurScavengeLogRecord>>eSurvivorBytes (in category 'accessing') ----- + eSurvivorBytes + + ^ eSurvivorBytes + ! Item was added: + ----- Method: SpurScavengeLogRecord>>eSurvivorBytes: (in category 'accessing') ----- + eSurvivorBytes: anObject + + ^ eSurvivorBytes := anObject. + ! Item was added: + ----- Method: SpurScavengeLogRecord>>sEdenBytes (in category 'accessing') ----- + sEdenBytes + + ^ sEdenBytes + ! Item was added: + ----- Method: SpurScavengeLogRecord>>sEdenBytes: (in category 'accessing') ----- + sEdenBytes: anObject + + ^ sEdenBytes := anObject. + ! Item was added: + ----- Method: SpurScavengeLogRecord>>sPastBytes (in category 'accessing') ----- + sPastBytes + + ^ sPastBytes + ! Item was added: + ----- Method: SpurScavengeLogRecord>>sPastBytes: (in category 'accessing') ----- + sPastBytes: anObject + + ^ sPastBytes := anObject. + ! Item was added: + ----- Method: SpurScavengeLogRecord>>sRememberedSetLimit (in category 'accessing') ----- + sRememberedSetLimit + + ^ sRememberedSetLimit + ! Item was added: + ----- Method: SpurScavengeLogRecord>>sRememberedSetLimit: (in category 'accessing') ----- + sRememberedSetLimit: anObject + + ^ sRememberedSetLimit := anObject. + ! Item was added: + ----- Method: SpurScavengeLogRecord>>sRememberedSetRedZone (in category 'accessing') ----- + sRememberedSetRedZone + + ^ sRememberedSetRedZone + ! Item was added: + ----- Method: SpurScavengeLogRecord>>sRememberedSetRedZone: (in category 'accessing') ----- + sRememberedSetRedZone: anObject + + ^ sRememberedSetRedZone := anObject. + ! Item was added: + ----- Method: SpurScavengeLogRecord>>sRememberedSetSize (in category 'accessing') ----- + sRememberedSetSize + + ^ sRememberedSetSize + ! Item was added: + ----- Method: SpurScavengeLogRecord>>sRememberedSetSize: (in category 'accessing') ----- + sRememberedSetSize: anObject + + ^ sRememberedSetSize := anObject. + ! Item was added: + ----- Method: SpurScavengeLogRecord>>sStatTenures (in category 'accessing') ----- + sStatTenures + + ^ sStatTenures + ! Item was added: + ----- Method: SpurScavengeLogRecord>>sStatTenures: (in category 'accessing') ----- + sStatTenures: anObject + + ^ sStatTenures := anObject. + ! Item was added: + ----- Method: SpurScavengeLogRecord>>tRefCountToShrinkRT (in category 'accessing') ----- + tRefCountToShrinkRT + + ^ tRefCountToShrinkRT + ! Item was added: + ----- Method: SpurScavengeLogRecord>>tRefCountToShrinkRT: (in category 'accessing') ----- + tRefCountToShrinkRT: anObject + + ^ tRefCountToShrinkRT := anObject. + ! Item was added: + ----- Method: SpurScavengeLogRecord>>tTenureCriterion (in category 'accessing') ----- + tTenureCriterion + + ^ tTenureCriterion + ! Item was added: + ----- Method: SpurScavengeLogRecord>>tTenureCriterion: (in category 'accessing') ----- + tTenureCriterion: anObject + + ^ tTenureCriterion := anObject. + ! Item was added: + ----- Method: SpurScavengeLogRecord>>tTenureThreshold (in category 'accessing') ----- + tTenureThreshold + + ^ tTenureThreshold + ! Item was added: + ----- Method: SpurScavengeLogRecord>>tTenureThreshold: (in category 'accessing') ----- + tTenureThreshold: anObject + + ^ tTenureThreshold := anObject. + ! Item was added: + ----- Method: TConstantNode>>asPrintfFormatStringNode (in category 'transformations') ----- + asPrintfFormatStringNode + ^TPrintfFormatStringNode new + setValue: value; + comment: comment; + yourself! Item was changed: ----- Method: TMethod>>prepareMethodIn: (in category 'transformations') ----- prepareMethodIn: aCodeGen "Record sends of builtin operators, map sends of the special selector dispatchOn:in: with case statement nodes, and map sends of caseOf:[otherwise:] to switch statements. Declare limit variables for to:[by:]do: loops with limits that potentially have side-effects. As a hack also update the types of variables introduced to implement cascades correctly. This has to be done at the same time as this is done, so why not piggy back here?" extraVariableNumber ifNotNil: [declarations keysAndValuesDo: [:varName :decl| decl isBlock ifTrue: [self assert: ((varName beginsWith: 'cascade') and: [varName last isDigit]). locals add: varName. self declarationAt: varName put: (decl value: self value: aCodeGen), ' ', varName]]]. aCodeGen pushScope: declarations while:"N.B. nodesWithParentsDo: is bottom-up, hence replacement is destructive and conserved." [parseTree nodesWithParentsDo: [:node :parent| node isSend ifTrue: [(aCodeGen isBuiltinSelector: node selector) ifTrue: [node isBuiltinOperator: true. "If a to:by:do:'s limit has side-effects, declare the limit variable, otherwise delete it from the args" node selector = #to:by:do: ifTrue: [self ensureToByDoLoopLimitIsSafeAndEfficient: node in: aCodeGen]] ifFalse: [(CaseStatements includes: node selector) ifTrue: [parent replaceNodesIn: (Dictionary newFromPairs: { node. self buildCaseStmt: node in: aCodeGen})]. (#(caseOf: #caseOf:otherwise:) includes: node selector) ifTrue: + [parent replaceNodesIn: (Dictionary newFromPairs: { node. self buildSwitchStmt: node parent: parent })]. + (#(printf: f:printf:) includes: node selector) ifTrue: + [| map | + map := Dictionary new. + node nodesDo: + [:subNode| + (subNode isConstant and: [subNode value isString and: [subNode value includes: $%]]) ifTrue: + [map at: subNode put: subNode asPrintfFormatStringNode]. + node replaceNodesIn: map]]]]]]! - [parent replaceNodesIn: (Dictionary newFromPairs: { node. self buildSwitchStmt: node parent: parent })]]]]]! Item was added: + TConstantNode subclass: #TPrintfFormatStringNode + instanceVariableNames: '' + classVariableNames: '' + poolDictionaries: '' + category: 'VMMaker-Translation to C'! Item was added: + ----- Method: TPrintfFormatStringNode>>emitCCodeOn:level:generator: (in category 'C code generation') ----- + emitCCodeOn: aStream level: level generator: aCodeGen + "Transform the printf string according to the LP64/LLP64 convention in sqMemoryAccess.h." + aStream nextPutAll: (aCodeGen cLiteralForPrintfString: value). + comment ifNotNil: + [aStream + nextPutAll: ' /* '; + nextPutAll: comment; + nextPutAll: ' */']! Item was changed: ----- Method: VMClass>>f:open: (in category 'printf logging') ----- f: logFilename open: mode "Simulate a FILE stream by answering a transcript. + Can be used with the f:printf: method." - Can be used with the log:* methods." <doNotGenerate> ^true ifTrue: [(TranscriptStream on: (String new: 100000)) openLabel: logFilename; yourself] ifFalse: [self coInterpreter transcript]! Item was removed: - ----- Method: VMClass>>flushLog (in category 'printf logging') ----- - flushLog - "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." - <inline: #always> - self logStream ifNotNil: - [self logStream fflush]! Item was removed: - ----- Method: VMClass>>log: (in category 'printf logging') ----- - log: printfString - "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." - <inline: #always> - self logStream ifNotNil: - [printfString fprintf: self logStream]! Item was removed: - ----- Method: VMClass>>log:with: (in category 'printf logging') ----- - log: printfString with: anArgument - "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." - <inline: #always> - self logStream ifNotNil: - [self logStream f: printfString printf: {anArgument}]! Item was removed: - ----- Method: VMClass>>log:with:with: (in category 'printf logging') ----- - log: printfString with: aArgument with: bArgument - "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." - <inline: #always> - self logStream ifNotNil: - [self logStream f: printfString printf: {aArgument. bArgument}]! Item was removed: - ----- Method: VMClass>>log:with:with:with: (in category 'printf logging') ----- - log: printfString with: aArgument with: bArgument with: cArgument - "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." - <inline: #always> - self logStream ifNotNil: - [self logStream f: printfString printf: {aArgument. bArgument. cArgument}]! Item was removed: - ----- Method: VMClass>>log:with:with:with:with: (in category 'printf logging') ----- - log: printfString with: aArgument with: bArgument with: cArgument with: dArgument - "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." - <inline: #always> - self logStream ifNotNil: - [self logStream f: printfString printf: {aArgument. bArgument. cArgument. dArgument}]! Item was removed: - ----- Method: VMClass>>log:with:with:with:with:with: (in category 'printf logging') ----- - log: printfString with: aArgument with: bArgument with: cArgument with: dArgument with: eArgument - "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." - <inline: #always> - self logStream ifNotNil: - [self logStream f: printfString printf: {aArgument. bArgument. cArgument. dArgument. eArgument}]! Item was removed: - ----- Method: VMClass>>log:with:with:with:with:with:with: (in category 'printf logging') ----- - log: printfString with: aArgument with: bArgument with: cArgument with: dArgument with: eArgument with: fArgument - "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." - <inline: #always> - self logStream ifNotNil: - [self logStream f: printfString printf: {aArgument. bArgument. cArgument. dArgument. eArgument. fArgument}]! Item was removed: - ----- Method: VMClass>>log:with:with:with:with:with:with:with: (in category 'printf logging') ----- - log: printfString with: aArgument with: bArgument with: cArgument with: dArgument with: eArgument with: fArgument with: gArgument - "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." - <inline: #always> - self logStream ifNotNil: - [self logStream f: printfString printf: {aArgument. bArgument. cArgument. dArgument. eArgument. fArgument. gArgument}]! Item was removed: - ----- Method: VMClass>>log:with:with:with:with:with:with:with:with: (in category 'printf logging') ----- - log: printfString with: aArgument with: bArgument with: cArgument with: dArgument with: eArgument with: fArgument with: gArgument with: hArgument - "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." - <inline: #always> - self logStream ifNotNil: - [self logStream f: printfString printf: {aArgument. bArgument. cArgument. dArgument. eArgument. fArgument. gArgument. hArgument}]! Item was removed: - ----- Method: VMClass>>log:withArgs: (in category 'printf logging') ----- - log: printfString withArgs: arguments - "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." - <inline: #always> - self logStream ifNotNil: - [self logStream f: printfString printf: arguments]!

1 0

[OpenSmalltalk/opensmalltalk-vm] 4ff342: CogVM source as per VMMaker.oscog-eem.2277
by GitHub 12 Nov '17

12 Nov '17

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: 4ff34235c24ea2fd690287f96a73834e744ad6c9 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/4ff34235c24ea2fd69… Author: Eliot Miranda <eliot.miranda(a)gmail.com> Date: 2017-11-12 (Sun, 12 Nov 2017) Changed paths: M image/BuildSqueakSpurTrunkVMMakerImage.st M image/Slang Test Workspace.text M nsspur64src/vm/cogit.h M nsspur64src/vm/cogitX64SysV.c M nsspur64src/vm/cogitX64WIN64.c M nsspur64src/vm/cointerp.c M nsspur64src/vm/cointerp.h M nsspur64src/vm/gcc3x-cointerp.c M nsspur64src/vm/interp.h M nsspursrc/vm/cogit.h M nsspursrc/vm/cogitARMv5.c M nsspursrc/vm/cogitIA32.c M nsspursrc/vm/cogitMIPSEL.c M nsspursrc/vm/cointerp.c M nsspursrc/vm/cointerp.h M nsspursrc/vm/gcc3x-cointerp.c M nsspursrc/vm/interp.h M nsspurstacksrc/vm/gcc3x-interp.c M nsspurstacksrc/vm/interp.c M platforms/iOS/vm/OSX/sqSqueakOSXApplication.m M platforms/unix/vm/sqUnixMain.c M platforms/win32/vm/sqWin32Main.c M platforms/win32/vm/sqWin32Window.c M spur64src/vm/cogit.h M spur64src/vm/cogitX64SysV.c M spur64src/vm/cogitX64WIN64.c M spur64src/vm/cointerp.c M spur64src/vm/cointerp.h M spur64src/vm/gcc3x-cointerp.c M spur64src/vm/interp.h M spursista64src/vm/cogit.h M spursista64src/vm/cogitX64SysV.c M spursista64src/vm/cogitX64WIN64.c M spursista64src/vm/cointerp.c M spursista64src/vm/cointerp.h M spursista64src/vm/gcc3x-cointerp.c M spursista64src/vm/interp.h M spursistasrc/vm/cogit.h M spursistasrc/vm/cogitARMv5.c M spursistasrc/vm/cogitIA32.c M spursistasrc/vm/cogitMIPSEL.c M spursistasrc/vm/cointerp.c M spursistasrc/vm/cointerp.h M spursistasrc/vm/gcc3x-cointerp.c M spursistasrc/vm/interp.h M spursrc/vm/cogit.h M spursrc/vm/cogitARMv5.c M spursrc/vm/cogitIA32.c M spursrc/vm/cogitMIPSEL.c M spursrc/vm/cointerp.c M spursrc/vm/cointerp.h M spursrc/vm/gcc3x-cointerp.c M spursrc/vm/interp.h M spurstack64src/vm/gcc3x-interp.c M spurstack64src/vm/interp.c M spurstacksrc/vm/gcc3x-interp.c M spurstacksrc/vm/interp.c Log Message: ----------- CogVM source as per VMMaker.oscog-eem.2277 Spur: Fix the growth of the remembered table during GC when there is not enough memory in old space. Allocates an extra memory segment in that case. This is very uncommon. Please check the thread: "[Vm-dev] VM crash with message 'could not grow remembered set' " to reproduce the problem. Provide a logging facility that uses the Printf package. See VMClass>>log:with:* for API. See Slang changes below for support. Use the logging facility to log scavenges. Each scavenge produces output such as scavenge 31 eden bytes: 0x2cfaf00/47165184 past bytes: 0x17c618/1558040 remembered set: 73 redzone: 98304 size: 131072 tenure below 0x0/0 by age survivor bytes: 0x17dd90/1564048 remembered objects: 73 tenured: 0 Use VM command line arg -/-logscavenger to enable. This facility is provisional. I (eliot) plan to reorganize so that accurate timing can be included. Cogit: Added primitiveStringReplace in the JIT with Spur, implementing only the quick paths for byte objects and array-like objects. FFI: Fixed bug with large struct in FFI with SysV x86-64. A typo in the Smalltalk source meant structs were processed incorrectly, confusing pass-in-register and pass-on-stack cases. Slang: Add the notion of a varargs selector (currently anything ending in printf:, such as f:printf:). Transform sends of such selectors with a brace of arguments into a varargs call (i.e. include all the arguments in the brace). When inlining, if the method being inlined is of the form guard ifTrue: [...] and the argument assignments are side-effect free, and the guard does not refer to the arguments, push the argument assignments past the guard into the block so that they too are guarded. Commit: d478d6a686bff3734028f4a3c9897f877a513fff https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/d478d6a686bff37340… Author: Eliot Miranda <eliot.miranda(a)gmail.com> Date: 2017-11-12 (Sun, 12 Nov 2017) Changed paths: M .appveyor.yml M platforms/unix/vm/sqUnixExternalPrims.c M platforms/unix/vm/sqUnixMain.c M platforms/unix/vm/sqUnixVMProfile.c A platforms/win32/plugins/FileAttributesPlugin/Makefile.plugin Log Message: ----------- Merge branch 'Cog' of https://github.com/OpenSmalltalk/opensmalltalk-vm into Cog Commit: 496050b1ce25e957786aa9bfbbb94d2fd0796aa6 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/496050b1ce25e95778… Author: Eliot Miranda <eliot.miranda(a)gmail.com> Date: 2017-11-12 (Sun, 12 Nov 2017) Changed paths: M platforms/iOS/vm/OSX/sqSqueakOSXApplication.m M platforms/unix/vm/sqUnixMain.c M platforms/win32/vm/sqWin32Main.c M platforms/win32/vm/sqWin32Window.c Log Message: ----------- --logscavenger => --logscavenge Compare: https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/b351fe0edeac...49…

1 0

VM Maker: VMMaker.oscog-eem.2277.mcz
by commits＠source.squeak.org 11 Nov '17

11 Nov '17

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2277.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.2277 Author: eem Time: 10 November 2017, 6:37:45.984021 pm UUID: e6b348b5-2875-4643-99ef-316ababea0f2 Ancestors: VMMaker.oscog-eem.2276 Oops. Revert an inadvertent change to flushLog. =============== Diff against VMMaker.oscog-eem.2276 =============== Item was changed: ----- Method: VMClass>>flushLog (in category 'printf logging') ----- flushLog "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." <inline: #always> self logStream ifNotNil: + [self logStream fflush]! - [:s| s fflush]!

1 0

VM Maker: VMMaker.oscog-eem.2276.mcz
by commits＠source.squeak.org 11 Nov '17

11 Nov '17

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2276.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.2276 Author: eem Time: 10 November 2017, 6:33:10.564346 pm UUID: 4ada0ebf-5c79-45d6-a0d4-80660c1b5575 Ancestors: VMMaker.oscog-eem.2275 Simplify flushLog in the logging framework by providing fflush on WriteStream. Slang: Now that scavengeLog is typed isFunctionalIn: must accept FILE * as an acceptable return type of a functional method. =============== Diff against VMMaker.oscog-eem.2275 =============== Item was changed: ----- Method: TMethod>>isFunctionalIn: (in category 'inlining') ----- isFunctionalIn: aCodeGen "Answer if the receiver is a functional method. That is, if it consists of a single return statement of an expression that contains no other returns, or an assert or flag followed by such a statement. Answer false for methods with return types other than the simple integer types to work around bugs in the inliner." parseTree statements size = 1 ifFalse: [(parseTree statements size = 2 and: [parseTree statements first isSend and: [parseTree statements first selector == #flag: or: [(aCodeGen isAssertSelector: parseTree statements first selector) and: [parseTree statements first selector ~~ #asserta:]]]]) ifFalse: [^false]]. parseTree statements last isReturn ifFalse: [^false]. parseTree statements last expression nodesDo: [ :n | n isReturn ifTrue: [^false]]. ^#(int #'unsigned int' #long #'unsigned long' #'long long' #'unsigned long long' sqInt usqInt #'sqIntptr_t' #'usqIntptr_t' sqLong usqLong #'int *' #'unsigned int *' #'sqInt *' #'usqInt *' #'sqLong *' #'usqLong *' #'char *' + #'CogMethod *' #'AbstractInstruction *' + #'FILE *') includes: returnType! - #'CogMethod *' #'AbstractInstruction *') includes: returnType! Item was changed: ----- Method: VMClass>>flushLog (in category 'printf logging') ----- flushLog "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." <inline: #always> self logStream ifNotNil: + [:s| s fflush]! - [self cCode: [self logStream fflush] inSmalltalk: [self logStream flush]]! Item was added: + ----- Method: WriteStream>>fflush (in category '*VMMaker-logging') ----- + fflush + "compatibility to map the stdio fflush(FILE *) routine onto flush" + self flush!

1 0

VM Maker: VMMaker.oscog-eem.2275.mcz
by commits＠source.squeak.org 11 Nov '17

11 Nov '17

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2275.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.2275 Author: eem Time: 10 November 2017, 3:07:20.900806 pm UUID: 3eae42a8-dc29-4b23-b794-b910c178a4dc Ancestors: VMMaker.oscog-cb.2274 Provide a logging facility that uses the Printf package. See VMClass>>log:with:* for API. See Slang changes below for support. Use the logging facility to log scavenges. Each scavenge produces output such as scavenge 31 eden bytes: 0x2cfaf00/47165184 past bytes: 0x17c618/1558040 remembered set: 73 redzone: 98304 size: 131072 tenure below 0x0/0 by age survivor bytes: 0x17dd90/1564048 remembered objects: 73 tenured: 0 Slang: Add the notion of a varargs selector (currently anything ending in printf:, such as f:printf:). Transform sends of such selectors with a brace of arguments into a varargs call (i.e. include all the arguments in the brace). When inlining, if the method being inlined is of the form guard ifTrue: [...] and the argument assignments are side-effect free, and the guard does not refer to the arguments, push the argument assignments past the guard into the block so that they too are guarded. =============== Diff against VMMaker.oscog-cb.2274 =============== Item was changed: ----- Method: BraceNode>>asTranslatorNodeIn: (in category '*VMMaker-C translation') ----- asTranslatorNodeIn: aTMethod + "make a CCodeGenerator equivalent of me." + + "This is for varargs selectors (variants of printf:)" + (elements notEmpty and: [elements allSatisfy: [:e| e isTemp]]) ifTrue: + [^elements collect: [:elem| elem asTranslatorNodeIn: aTMethod]]. + + "This is for case statements" - "make a CCodeGenerator equivalent of me" self assert: (elements allSatisfy: [:elem| elem isMessageNode and: [elem selector key = #-> and: [elem receiver isBlockNode and: [elem arguments first isBlockNode and: [elem receiver isPotentialCCaseLabelIn: aTMethod]]]]]). ^TBraceCaseNode new caseLabels: (elements collect: [:elem| elem receiver asTranslatorNodeIn: aTMethod]); cases: (elements collect: [:elem| elem arguments first asTranslatorNodeIn: aTMethod]); comment: comment! Item was added: + ----- Method: CCodeGenerator class>>isVarargsSelector: (in category 'testing') ----- + isVarargsSelector: aSymbol + ^aSymbol endsWith: 'printf:'! Item was changed: ----- Method: MessageNode>>asTranslatorNodeIn: (in category '*VMMaker-C translation') ----- asTranslatorNodeIn: aTMethod "make a CCodeGenerator equivalent of me" "selector is sometimes a Symbol, sometimes a SelectorNode!! On top of this, numArgs is needed due to the (truly grody) use of arguments as a place to store the extra expressions needed to generate code for in-line to:by:do:, etc. see below, where it is used. Expand super nodes in place. Elide sends of halt so that halts can be sprinkled through the simulator but will be eliminated from the generated C." | rcvrOrNil sel args ifNotNilBlock | rcvrOrNil := receiver ifNotNil: [receiver asTranslatorNodeIn: aTMethod]. (rcvrOrNil notNil and: [rcvrOrNil isVariable and: [rcvrOrNil name = 'super']]) ifTrue: [^aTMethod superExpansionNodeFor: selector key args: arguments]. sel := selector isSymbol ifTrue: [selector] ifFalse: [selector key]. sel == #halt ifTrue: [^rcvrOrNil]. (sel == #cCode:inSmalltalk: "extracting here rather than in translation allows inlining in the block." or: [sel == #cCode:]) ifTrue: [arguments first isBlockNode ifTrue: [| block | ^(block := arguments first asTranslatorNodeIn: aTMethod) statements size = 1 ifTrue: [block statements first] ifFalse: [block]]. (arguments first isLiteralNode and: [arguments first key isString and: [arguments first key isEmpty]]) ifTrue: [^arguments first asTranslatorNodeIn: aTMethod]]. args := arguments select: [:arg| arg notNil] thenCollect: [:arg| arg asTranslatorNodeIn: aTMethod]. (sel = #to:by:do: and: [arguments size = 7 and: [(arguments at: 7) notNil]]) ifTrue: ["Restore limit expr that got moved by transformToDo:" args := {(arguments at: 7) value asTranslatorNodeIn: aTMethod. args second. args third. "add the limit var as a hidden extra argument; we may need it later" TVariableNode new setName: arguments first key}]. (sel == #ifTrue:ifFalse: and: [arguments first isJust: NodeNil]) ifTrue: [sel := #ifFalse:. args := {args last}]. (sel == #ifTrue:ifFalse: and: [arguments last isJust: NodeNil]) ifTrue: [sel := #ifTrue:. args := {args first}]. (sel == #ifFalse:ifTrue: and: [arguments first isJust: NodeNil]) ifTrue: [sel := #ifTrue:. args := {args last}]. (sel == #ifFalse:ifTrue: and: [arguments last isJust: NodeNil]) ifTrue: [sel := #ifTrue:. args := {args first}]. ((sel == #ifFalse: or: [sel == #or:]) and: [arguments size = 2 and: [(arguments at: 2) notNil]]) ifTrue: ["Restore argument block that got moved by transformOr: or transformIfFalse:" args := {(arguments at: 2) asTranslatorNodeIn: aTMethod}]. (args size > sel numArgs and: [sel ~~ #to:by:do:]) ifTrue: "to:by:do: has iLimiT hidden in last arg" ["prune the extra blocks left by ifTrue:, ifFalse:, and: & or:" self assert: args size - sel numArgs = 1. self assert: (args last isStmtList and: [args last statements size = 1 and: [(args last statements first isVariable or: [args last statements first isConstant]) and: [#('nil' true false) includes: args last statements first nameOrValue]]]). args := args first: sel numArgs]. "For the benefit of later passes, e.g. value: inlining, transform e ifNotNil: [:v| ...] into v := e. v ifNotNil: [...], which in fact means transforming (v := e) ifTrue: [:v|...] into v := e. v ifTrue: [...]." ((sel == #ifTrue: or: [sel == #ifFalse: or: [sel == #ifTrue:ifFalse: or: [sel == #ifFalse:ifTrue:]]]) and: [receiver notNil and: [receiver isAssignmentEqualsEqualsNil and: [(ifNotNilBlock := args detect: [:arg| arg isStmtList and: [arg args size = 1]] ifNone: []) notNil]]]) ifTrue: [ifNotNilBlock setArguments: #(). ^TStmtListNode new setArguments: #() statements: { receiver receiver asTranslatorNodeIn: aTMethod. TSendNode new setSelector: sel receiver: (TSendNode new setSelector: #== receiver: (receiver receiver variable asTranslatorNodeIn: aTMethod) arguments: {receiver arguments first asTranslatorNodeIn: aTMethod}) arguments: args }]. + ((CCodeGenerator isVarargsSelector: sel) + and: [args last isCollection + and: [args last isSequenceable]]) ifTrue: + [args := args allButLast, args last]. ^TSendNode new setSelector: sel receiver: rcvrOrNil arguments: args! Item was changed: CogClass subclass: #SpurGenerationScavenger + instanceVariableNames: 'coInterpreter manager eden futureSpace pastSpace futureSurvivorStart rememberedSet rememberedSetSize previousRememberedSetSize rememberedSetRedZone rememberedSetLimit refCountToShrinkRT weakList ephemeronList tenureCriterion tenureThreshold tenuringClassIndex tenuringProportion numRememberedEphemerons scavengeLog statSurvivorCount statTenures' - instanceVariableNames: 'coInterpreter manager eden futureSpace pastSpace futureSurvivorStart rememberedSet rememberedSetSize previousRememberedSetSize rememberedSetRedZone rememberedSetLimit refCountToShrinkRT weakList ephemeronList tenureCriterion tenureThreshold tenuringClassIndex tenuringProportion numRememberedEphemerons statSurvivorCount statTenures' classVariableNames: '' poolDictionaries: 'SpurMemoryManagementConstants' category: 'VMMaker-SpurMemoryManager'! + !SpurGenerationScavenger commentStamp: 'eem 11/7/2017 17:26' prior: 0! - !SpurGenerationScavenger commentStamp: 'eem 9/30/2013 11:05' prior: 0! SpurGenerationScavenger is an implementation of David Ungar's Generation Scavenging garbage collection algorithm. See Generation Scavenging, A Non-disruptive, High-Performance Storage Reclamation Algorithm David Ungar Proceeding SDE 1 Proceedings of the first ACM SIGSOFT/SIGPLAN software engineering symposium on Practical software development environments Pages 157 - 167 ACM New York, NY, USA ©1984 Also relevant are An adaptive tenuring policy for generation scavengers David Ungar & Frank Jackson ACM Transactions on Programming Languages and Systems (TOPLAS) TOPLAS Homepage archive Volume 14 Issue 1, Jan. 1992 Pages 1 - 27 ACM New York, NY, USA ©1992 and Ephemerons: a new finalization mechanism Barry Hayes Proceedings of the 12th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications Pages 176-183 ACM New York, NY, USA ©1997 See text below the variable definitions and explanation below for a full explanation of weak and ephemeron processing. Instance Variables coInterpreter: <StackInterpreterSimulator|CogVMSimulator> eden: <SpurNewSpaceSpace> ephemeronList: <Integer|nil> futureSpace: <SpurNewSpaceSpace> futureSurvivorStart: <Integer address> manager: <SpurMemoryManager|Spur32BitMMLESimulator et al> numRememberedEphemerons: <Integer> pastSpace: <SpurNewSpaceSpace> previousRememberedSetSize: <Integer> rememberedSet: <CArrayAccessor on: Array> rememberedSetSize: <Integer> tenuringProportion: <Float> tenuringThreshold: <Integer address> weakList: <Integer|nil> coInterpreter - the interpreter/vm, in this context, the mutator manager - the Spur memory manager eden - the space containing newly created objects futureSpace - the space to which surviving objects are copied during a scavenge futureSurvivorStart - the allocation pointer into futureSpace pastSpace - the space surviving objects live in until the next scavenge rememberedSet - the root old space objects that refer to objects in new space; a scavenge starts form these roots and the interpreter's stack rememberedSetSize - the size of the remembered set, also the first unused index in the rememberedSet previousRememberedSetSize: - the size of the remembered set before scavenging objects in future space. numRememberedEphemerons - the number of unscavenged ephemerons at the front of the rememberedSet. ephemeronList - the head of the list of corpses of unscavenged ephemerons reached in the current phase weakList - the head of the list of corpses of weak arrays reached during the scavenge. tenuringProportion - the amount of pastSpace below which the system will not tenure unless futureSpace fills up, and above which it will eagerly tenure tenuringThreshold - the pointer into pastSpace below which objects will be tenured Weakness and Ephemerality in the Scavenger. Weak arrays should not hold onto their referents (except from their strong fileds, their named inst vars). Ephemerons are objects that implement instance-based finalization; attaching an ephemeron to an object keeps that object alive and causes the ephemeron to "fire" when the object is only reachable from the ephemeron (or other ephemerons & weak arrays). They are a special kind of Associations that detect when their keys are about to die, i.e. when an ephemeron's key is not reachable from the roots except from weak arrays and other ephemerons with about-to-die keys. Note that if an ephemeron's key is not about to die then references from the rest of the ephemeron can indeed prevent ephemeron keys from dying. The scavenger is concerned with collecting objects in new space, therefore it ony deals with weak arrays and ephemerons that are either in the remembered set or in new space. By deferring scanning these objects until other reachable objects have been scavenged, the scavenger can detect dead or dying references. Weak Array Processing In the case of weak arrays this is simple. The scavenger refuses to scavenge the referents of weak arrays in scavengeReferentsOf: until the entire scavenge is over. It then scans the weak arrays in the remembered set and in future space and nils all fields in them that are referring to unforwarded objects in eden and past space, because these objects have not survived the scavenge. The root weak arrays remaining to be scavenged are in the remembered table. Surviving weak arrays in future space are collected on a list. The list is threaded through the corpses of weak arrays in eden and/or past space. weakList holds the slot offset of the first weak array found in eden and/or past space. The next offset is stored in the weak array corpse's identityHash and format fields (22 bits & 5 bits of allocationUnits, for a max new space size of 2^28 bytes, 256Mb). The list is threaded throguh corpses, but the surviving arrays are pointed to by the corpses' forwarding pointers. Ephemeron Processing The case of ephemerons is a little more complicated because an ephemeron's key should survive. The scavenger is cyclical. It scavenges the remembered set, which may copy and forward surviving objects in past and/or eden spaces to future space. It then scavenges those promoted objects in future space until no more are promoted, which may in turn remember more objects. The cycles continue until no more objects get promoted to future space and no more objects get remembered. At this point all surviving objecta are in futureSpace. + So if the scavenger does not scan ephemerons in the remembered set or in future space until the scavenger finishes cycling, it can detect ephemerons whose keys are about to die because these will be unforwarded objects in eden and/or past space. Ephemerons encountered in the remembered set are either processed like ordinary objects if their keys have been promoted to futureSpace, or are moved to the front of the rememberedSet (because, dear reader, it is a sequence) if their keys have not been promoted. Ephemerons encountered in scavengeReferentsOf: are either scanned like normal objects if their keys have been promoted, or added to the ephemeronList, organized identically to the weakList, if their keys are yet to be promoted. Since references from other ephemerons with surviving keys to ephemeron keys can and should prevent the ephemerons whose keys they are from firing the scavenger does not fire ephemerons unless all unscavenged ephemerons have unscavenged keys. So the unsca venged ephemerons (they will be at the beginning of the remembered set and on the ephemeronList) are scanned and any that have promoted keys are scavenged. But if no unscavenged ephemerons have surviving keys then all the unscavenged ephemerons are fired and then scavenged. This in turn may remember more objects and promote more objects to future space, and encounter more unscavenged ephemerons. So the scavenger continues until no more objects are remembered, no more objects are promoted to future space and no more unscavenged ephemerons exist.! - So if the scavenger does not scan ephemerons in the remembered set or in future space until the scavenger finishes cycling, it can detect ephemerons whose keys are about to die because these will be unforwarded objects in eden and/or past space. Ephemerons encountered in the remembered set are either processed like ordinary objects if their keys have been promoted to futureSpace, or are moved to the front of the rememberedSet (because, dear reader, it is a sequence) if their keys have not been promoted. Ephemerons encountered in scavengeReferentsOf: are either scanned like normal objects if their keys have been promoted, or added to the ephemeronList, organized identically to the weakList, if their keys are yet to be promoted. Since references from other ephemerons with surviving keys to ephemeron keys can and should prevent the ephemerons whose keys they are from firing the scavenger does not fire ephemerons unless all unscavenged ephemerons have unscavenged keys. So the unsca venged ephemerons (the will be at the beginning of the remembered set and on the ephemeronList) are scanned and any that have promoted keys are scavenged. But if no unscavenged ephemerons have surviving keys then all the unscavenged ephemerons are fired and then scavenged. This in turn may remember more objects and promote more objects to future space, and encounter more unscavenged ephemerons. So the scavenger continues until no more objects are remembered, no more objects are promoted to future space and no more unscavenged ephemerons exist.! Item was changed: ----- Method: SpurGenerationScavenger class>>declareCVarsIn: (in category 'translation') ----- declareCVarsIn: aCCodeGenerator #(eden futureSpace pastSpace) do: [:var| aCCodeGenerator var: var type: #SpurNewSpaceSpace]. + aCCodeGenerator + var: #rememberedSet type: #'sqInt *'; + var: #tenuringProportion type: #double; + var: #scavengeLog type: #'FILE *'! - aCCodeGenerator var: #rememberedSet type: #'sqInt *'. - aCCodeGenerator var: #tenuringProportion type: #double! Item was added: + ----- Method: SpurGenerationScavenger>>logEndScavenge (in category 'logging') ----- + logEndScavenge + <inline: #always> + self log: ' survivor bytes: 0x%lx/%ld remembered objects: %ld tenured: %ld\n' + with: futureSurvivorStart - pastSpace start + with: futureSurvivorStart - pastSpace start + with: rememberedSetSize + with: statTenures; + flushLog! Item was added: + ----- Method: SpurGenerationScavenger>>logStartScavenge (in category 'logging') ----- + logStartScavenge + <inline: #always> + self log: 'scavenge %ld eden bytes: 0x%lx/%ld past bytes: 0x%lx/%ld\n\tremembered set: %ld redzone: %ld size: %ld\n' + with: manager statScavenges + with: manager freeStart - eden start + with: manager freeStart - eden start + with: manager pastSpaceStart - pastSpace start + with: manager pastSpaceStart - pastSpace start + with: rememberedSetSize + with: rememberedSetRedZone + with: rememberedSetLimit! Item was added: + ----- Method: SpurGenerationScavenger>>logStream (in category 'logging') ----- + logStream + <inline: #always> + ^scavengeLog! Item was added: + ----- Method: SpurGenerationScavenger>>logTenuringPolicy (in category 'logging') ----- + logTenuringPolicy + <inline: #always> + | policyNames | + <var: 'policyNames' declareC: 'static char *policyNames[] = {"", "by age", "by class", "to shrink rt", "don''t tenure", "mark on tenure"}'> + self cCode: [] + inSmalltalk: [policyNames := CLiteralArray on: #('' 'by age' 'by class' 'to shrink rt' 'don''t tenure' 'mark on tenure')]. + self log: (tenureCriterion = TenureToShrinkRT ifFalse: [' tenure below 0x%lx/%ld %s\n'] ifTrue: [' tenure below 0x%lx/%ld %s refct %ld\n']) + with: ((tenureCriterion = TenureByAge and: [tenureThreshold > pastSpace start]) ifTrue: [tenureThreshold - pastSpace start] ifFalse: [0]) + with: ((tenureCriterion = TenureByAge and: [tenureThreshold > pastSpace start]) ifTrue: [tenureThreshold - pastSpace start] ifFalse: [0]) + with: (policyNames at: tenureCriterion) + with: refCountToShrinkRT! Item was added: + ----- Method: SpurGenerationScavenger>>openScavengeLog (in category 'logging') ----- + openScavengeLog + <api> + scavengeLog := self f: 'scavenge.log' open: 'a+'! Item was changed: ----- Method: SpurGenerationScavenger>>scavenge: (in category 'scavenger') ----- scavenge: tenuringCriterion "The main routine, scavenge, scavenges young objects reachable from the roots (the stack zone and the rememberedTable). It first scavenges the new objects immediately reachable from old ones (all in the remembered table), then the stack zone. Then it scavenges those that are transitively reachable. If this results in a promotion, the promotee gets remembered, and it first scavenges objects adjacent to the promotee, then scavenges the ones reachable from the promoted. This loop continues until no more reachable objects are left. At that point, pastSurvivorSpace is exchanged with futureSurvivorSpace. Then any surviving weakArrays and weakArrays in the remembered set can be processed and their dead elements nilled. By default promotion (tenuring) is based on age and ammount of objects scavenged. But tenuring can be based on e.g. a particular class. The argument selects the tenuring criterion. Answer the limit of pastSpace, to allow the memory manager to bounds check survivors." statSurvivorCount := 0. tenureCriterion := tenuringCriterion. + self logStartScavenge. self strategizeToLimitRememberedTable. + self logTenuringPolicy. self scavengeLoop. self processWeaklings. self computeTenuringThreshold. self exchangeSurvivorSpaces. + self logEndScavenge. ^self initFutureSpaceStart! Item was added: + ----- Method: SpurMemoryManager>>pastSpaceStart (in category 'accessing') ----- + pastSpaceStart + <cmacro: '() GIV(pastSpaceStart)'> + ^pastSpaceStart! Item was changed: ----- Method: TMethod>>inferReturnTypeFromReturnsIn: (in category 'type inference') ----- inferReturnTypeFromReturnsIn: aCodeGen "Attempt to infer the return type of the receiver from returns in the parse tree." "this for determining which returns have which return types:" "aCodeGen pushScope: declarations while: [parseTree nodesSelect: [:n| n isReturn] thenCollect: [:n| | s | s := Set new. self addTypesFor: n expression to: s in: aCodeGen. {n. s}]]" aCodeGen maybeBreakForTestToInline: selector in: self. returnType ifNotNil: [^self]. aCodeGen pushScope: declarations while: [| hasReturn returnTypes | hasReturn := false. returnTypes := Set new. "Debug: (| rettypes | rettypes := Dictionary new. parseTree nodesDo: [:node| node isReturn ifTrue: [| types | self addTypesFor: node expression to: (types := Set new) in: aCodeGen. rettypes at: node expression put: types]]. rettypes)" parseTree nodesDo: [:node| node isReturn ifTrue: [hasReturn := true. "If we encounter a send of an as-yet-untyped method then abort, retrying and computing the type when that method is fully typed." (self addTypesFor: node expression to: returnTypes in: aCodeGen) ifTrue: [^self]]]. returnTypes remove: #implicit ifAbsent: []. returnTypes := aCodeGen harmonizeReturnTypesIn: returnTypes. hasReturn ifTrue: [returnTypes size > 1 ifTrue: [| message | message := String streamContents: [:s| s nextPutAll: 'conflicting return types '. returnTypes do: [:t| s nextPutAll: t] separatedBy: [s nextPutAll: ', ']. s nextPutAll: ' in '; nextPutAll: selector; cr]. Notification signal: message. + aCodeGen logger ensureCr; show: message]. - aCodeGen logger show: message]. returnTypes size = 1 ifTrue: [self returnType: returnTypes anyOne]] ifFalse: [self returnType: (aCodeGen implicitReturnTypeFor: selector)]]! Item was changed: ----- Method: TMethod>>inlineSend:directReturn:exitVar:in: (in category 'inlining') ----- inlineSend: aSendNode directReturn: directReturn exitVar: exitVar in: aCodeGen "Answer a collection of statements to replace the given send. directReturn indicates that the send is the expression in a return statement, so returns can be left in the body of the inlined method. If exitVar is nil, the value returned by the send is not used; thus, returns need not assign to the output variable. Types are propagated to as-yet-untyped variables when inlining a send that is assigned, otherwise the assignee variable type must match the return type of the inlinee. Return types are not propagated." + | sel meth methArgs exitLabel inlineStmts exitType elidedArgs argAssignments conditional | - | sel meth methArgs exitLabel inlineStmts label exitType elidedArgs | sel := aSendNode selector. meth := aCodeGen methodNamed: sel. methArgs := meth args. "convenient for debugging..." aCodeGen maybeBreakForInlineOf: aSendNode in: self. elidedArgs := #(). (methArgs notEmpty and: [methArgs first beginsWith: 'self_in_']) ifTrue: "If the first arg is not used we can and should elide it." [| varNode | varNode := TVariableNode new setName: methArgs first. (meth parseTree noneSatisfy: [:node| varNode isSameAs: node]) ifTrue: [elidedArgs := {methArgs first}]. methArgs := methArgs allButFirst]. methArgs size = aSendNode args size ifFalse: [^nil]. meth := meth copy. (meth statements size > 1 and: [meth statements first isSend and: [meth statements first selector == #flag:]]) ifTrue: [meth statements removeFirst]. "Propagate the return type of an inlined method" (directReturn or: [exitVar notNil]) ifTrue: [exitType := directReturn ifTrue: [returnType] ifFalse: [(self typeFor: exitVar in: aCodeGen) ifNil: [#sqInt]]. (exitType = #void or: [exitType = meth returnType]) ifFalse: [meth propagateReturnIn: aCodeGen]]. "Propagate any unusual argument types to untyped argument variables" methArgs with: aSendNode args do: [:formal :actual| (meth declarationAt: formal ifAbsent: nil) ifNil: [(self typeFor: actual in: aCodeGen) ifNotNil: [:type| type ~= #sqInt ifTrue: [meth declarationAt: formal put: (type last = $* ifTrue: [type, formal] ifFalse: [type, ' ', formal])]]]]. meth renameVarsForInliningInto: self except: elidedArgs in: aCodeGen. meth renameLabelsForInliningInto: self. self addVarsDeclarationsAndLabelsOf: meth except: elidedArgs. meth hasReturn ifTrue: [directReturn ifFalse: [exitLabel := self unusedLabelForInliningInto: self. (meth exitVar: exitVar label: exitLabel) "is label used?" ifTrue: [ labels add: exitLabel ] ifFalse: [ exitLabel := nil ]]]. (inlineStmts := OrderedCollection new: meth statements size + meth args size + 2) + add: (TLabeledCommentNode new setComment: 'begin ', sel). + argAssignments := self argAssignmentsFor: meth send: aSendNode except: elidedArgs in: aCodeGen. + "If the method being inlined is of the form + guard ifTrue: [...] + and the argument assignments have no side effects, + and guard does not refer to the arguments, + push the argument assignments past the guard into the block so that they too are guarded." + (meth statements size = 1 + and: [(conditional := meth statements first) isConditionalSend + and: [conditional numArgs = 1 + and: [(argAssignments noneSatisfy: [:assign| assign expression hasSideEffect]) + and: [conditional receiver noneSatisfy: [:node| node isVariable and: [methArgs includes: node name]]]]]]) + ifTrue: [conditional args first statements addAllFirst: argAssignments] + ifFalse: [inlineStmts addAll: argAssignments]. + inlineStmts addAll: meth statements. "method body" - add: (label := TLabeledCommentNode new setComment: 'begin ', sel); - addAll: (self argAssignmentsFor: meth send: aSendNode except: elidedArgs in: aCodeGen); - addAll: meth statements. "method body" directReturn ifTrue: [meth endsWithReturn ifTrue: [exitVar ifNotNil: "don't remove the returns if being invoked in the context of a return" [inlineStmts at: inlineStmts size put: inlineStmts last copyWithoutReturn]] ifFalse: [inlineStmts add: (TReturnNode new setExpression: (TVariableNode new setName: 'nil'))]]. exitLabel ifNotNil: [inlineStmts add: (TLabeledCommentNode new setLabel: exitLabel comment: 'end ', meth selector)]. inlineStmts size = 1 ifTrue: "Nuke empty methods; e.g. override of flushAtCache" [self assert: inlineStmts first isComment. inlineStmts removeFirst]. ^inlineStmts! Item was added: + ----- Method: VMClass>>f:open: (in category 'printf logging') ----- + f: logFilename open: mode + "Simulate a FILE stream by answering a transcript. + Can be used with the log:* methods." + <doNotGenerate> + ^true + ifTrue: [(TranscriptStream on: (String new: 100000)) + openLabel: logFilename; + yourself] + ifFalse: [self coInterpreter transcript]! Item was added: + ----- Method: VMClass>>flushLog (in category 'printf logging') ----- + flushLog + "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." + <inline: #always> + self logStream ifNotNil: + [self cCode: [self logStream fflush] inSmalltalk: [self logStream flush]]! Item was added: + ----- Method: VMClass>>log: (in category 'printf logging') ----- + log: printfString + "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." + <inline: #always> + self logStream ifNotNil: + [printfString fprintf: self logStream]! Item was added: + ----- Method: VMClass>>log:with: (in category 'printf logging') ----- + log: printfString with: anArgument + "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." + <inline: #always> + self logStream ifNotNil: + [self logStream f: printfString printf: {anArgument}]! Item was added: + ----- Method: VMClass>>log:with:with: (in category 'printf logging') ----- + log: printfString with: aArgument with: bArgument + "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." + <inline: #always> + self logStream ifNotNil: + [self logStream f: printfString printf: {aArgument. bArgument}]! Item was added: + ----- Method: VMClass>>log:with:with:with: (in category 'printf logging') ----- + log: printfString with: aArgument with: bArgument with: cArgument + "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." + <inline: #always> + self logStream ifNotNil: + [self logStream f: printfString printf: {aArgument. bArgument. cArgument}]! Item was added: + ----- Method: VMClass>>log:with:with:with:with: (in category 'printf logging') ----- + log: printfString with: aArgument with: bArgument with: cArgument with: dArgument + "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." + <inline: #always> + self logStream ifNotNil: + [self logStream f: printfString printf: {aArgument. bArgument. cArgument. dArgument}]! Item was added: + ----- Method: VMClass>>log:with:with:with:with:with: (in category 'printf logging') ----- + log: printfString with: aArgument with: bArgument with: cArgument with: dArgument with: eArgument + "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." + <inline: #always> + self logStream ifNotNil: + [self logStream f: printfString printf: {aArgument. bArgument. cArgument. dArgument. eArgument}]! Item was added: + ----- Method: VMClass>>log:with:with:with:with:with:with: (in category 'printf logging') ----- + log: printfString with: aArgument with: bArgument with: cArgument with: dArgument with: eArgument with: fArgument + "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." + <inline: #always> + self logStream ifNotNil: + [self logStream f: printfString printf: {aArgument. bArgument. cArgument. dArgument. eArgument. fArgument}]! Item was added: + ----- Method: VMClass>>log:with:with:with:with:with:with:with: (in category 'printf logging') ----- + log: printfString with: aArgument with: bArgument with: cArgument with: dArgument with: eArgument with: fArgument with: gArgument + "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." + <inline: #always> + self logStream ifNotNil: + [self logStream f: printfString printf: {aArgument. bArgument. cArgument. dArgument. eArgument. fArgument. gArgument}]! Item was added: + ----- Method: VMClass>>log:with:with:with:with:with:with:with:with: (in category 'printf logging') ----- + log: printfString with: aArgument with: bArgument with: cArgument with: dArgument with: eArgument with: fArgument with: gArgument with: hArgument + "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." + <inline: #always> + self logStream ifNotNil: + [self logStream f: printfString printf: {aArgument. bArgument. cArgument. dArgument. eArgument. fArgument. gArgument. hArgument}]! Item was added: + ----- Method: VMClass>>log:withArgs: (in category 'printf logging') ----- + log: printfString withArgs: arguments + "Log via printf-style format strings. Subclasses must implement logStream to answer their log stream/file." + <inline: #always> + self logStream ifNotNil: + [self logStream f: printfString printf: arguments]!

1 0

VM Semaphores and external native threads
by Ronie Salgado 10 Nov '17

10 Nov '17

Hi, I started to make an IDE like tool in Pharo, and I have an external library for handling some asynchronous events generated in a separate thread, which I am putting in a queue that I am polling each 200 ms from a Pharo image. Currently I am using this library for handling pipes of external process (GDB with the machine interface), and next I am going to add support for inotify. I know that I can remove this polling each 200 ms by implementing a mechanism for registering multiples event handlers that are called by ioProcessEvents. The SDL2DisplayPlugin registers an event handler in this places, which asks SDL2 for the presence of events and it signals VM Semaphore. However, since in this case I have control of the code in the external thread, I would like to just signal the VM Semaphore in this external thread to avoid this polling. So here is my question: Is signaling a VM Semaphore thread safe? In case it is not, how much work would require to make it thread safe. Best regards, Ronie

5 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev November 2017