commits@source.squeak.org wrote:
Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.399.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.399 Author: eem Time: 20 September 2013, 6:28:56.308 pm UUID: 89f8fefe-b59d-42d7-9c11-7f848d0e5131 Ancestors: VMMaker.oscog-eem.398
A few isIntegerObject:'s => isImmediate:'s in primitives.
More protocol.
The Spur VM now draws its first window!!
A cheer goes up in the crowd of interested spectators. Probably lots still to do, but its a nice concrete milestone. Contributing is beyond me at this time, so I especially like to thank you for this important initiative.
cheers -ben
Great progress. I've got a few questions about Spur and the bootstrap: - How will HashedCollections other than MethodDictionaries get rehashed? How #hash be calculated from #identityHash? - What will happen to primitive 138 and 139? Will we still be able to iterate over the objects in the order of their creation time? - IIUC the memory is more segmented. Will there be new GC primitives for various levels of garbage collection? What about the GC parameters? - Are #become: and #becomeForward: optimized for the case where both objects use the same amount of memory?
Levente
On Sat, 21 Sep 2013, btc@openinworld.com wrote:
commits@source.squeak.org wrote:
Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.399.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.399 Author: eem Time: 20 September 2013, 6:28:56.308 pm UUID: 89f8fefe-b59d-42d7-9c11-7f848d0e5131 Ancestors: VMMaker.oscog-eem.398
A few isIntegerObject:'s => isImmediate:'s in primitives.
More protocol.
The Spur VM now draws its first window!!
A cheer goes up in the crowd of interested spectators. Probably lots still to do, but its a nice concrete milestone. Contributing is beyond me at this time, so I especially like to thank you for this important initiative.
cheers -ben
Hi Levente,
On Sat, Sep 21, 2013 at 11:39 AM, Levente Uzonyi leves@elte.hu wrote:
Great progress. I've got a few questions about Spur and the bootstrap:
- How will HashedCollections other than MethodDictionaries get rehashed?
How #hash be calculated from #identityHash?
The bootstrap first builds an image in the new format and then finds all classes in it that implement rehash and then uses the simulator to send rehash to all objects to use their own rehash method to rehash them, instead of assuming their format. In fact, the bootstrap does not rehash all objects that understand rehash since Symbol is one of them, and its Symbol table is rehashed anyway.
hash is computed from identityHash without change, but scaledIdentityHash is changed to avoid creating LargeIntegers (the larger identityHash needs less shift also): ProtoObjectPROTOTYPEscaledIdentityHash "For identityHash values returned by primitive 75, answer such values times 2^8. Otherwise, match the existing identityHash implementation"
^self identityHash * 256 "bitShift: 8"
- What will happen to primitive 138 and 139? Will we still be able to
iterate over the objects in the order of their creation time?
That's much more difficult. I'll add allInstances and allObjects primitives which can answer the objects instantianeously without needing to enumerate. One issue is that the scavenger could fire at any time during the enumeration and could tenure objects to old space so its in general not possible to reliably enumerate via firstInstance/nextInstance firstObject/nextObject.
Being able to enumerate in order of creation time is something the VM could maintain in old space, keeping objects in order there-in. But I'd rather drop this property and allow the Spur GC to compact using best-fit and lazy become, which will change the order of objects. How important is being able to enumerate in order of creation? I think it's nice to have but hardly essential. What do y'all think?
- IIUC the memory is more segmented. Will there be new GC primitives for
various levels of garbage collection? What about the GC parameters?
That remains to be seen. I take requests. I take design suggestions. But I also like to keep things simple.
My current approach is to keep things as compatible as possible to ease adoption. Once it is adopted we can start to evolve to provide appropriate and.or improved management facilities.
- Are #become: and #becomeForward: optimized for the case where both
objects use the same amount of memory?
Yes. This case simply swaps contents and adjusts the remembered table accordingly. Tim also suggested optimizing the other case, copying into the smaller object and only allocating one extra clone, which I'll implement soon.
Levente
On Sat, 21 Sep 2013, btc@openinworld.com wrote:
commits@source.squeak.org wrote:
Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/**VMMaker/VMMaker.oscog-eem.399.**mczhttp://source.squeak.org/VMMaker/VMMaker.oscog-eem.399.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.399 Author: eem Time: 20 September 2013, 6:28:56.308 pm UUID: 89f8fefe-b59d-42d7-9c11-**7f848d0e5131 Ancestors: VMMaker.oscog-eem.398
A few isIntegerObject:'s => isImmediate:'s in primitives.
More protocol.
The Spur VM now draws its first window!!
A cheer goes up in the crowd of interested spectators.
Probably lots still to do, but its a nice concrete milestone. Contributing is beyond me at this time, so I especially like to thank you for this important initiative.
cheers -ben
Hi eliiot
I'm curious to know the scenario of levente (for the creation order garanty enumeration) but to me it looks like that code based on such implicit semantics is fishy because really bound to the vm behavior - especially in presence of GC. I would not write code with such invariant in mind. It looks also like a complex constraint.
Stef
Hi Levente,
On Sat, Sep 21, 2013 at 11:39 AM, Levente Uzonyi leves@elte.hu wrote:
Great progress. I've got a few questions about Spur and the bootstrap:
- How will HashedCollections other than MethodDictionaries get rehashed? How #hash be calculated from #identityHash?
The bootstrap first builds an image in the new format and then finds all classes in it that implement rehash and then uses the simulator to send rehash to all objects to use their own rehash method to rehash them, instead of assuming their format. In fact, the bootstrap does not rehash all objects that understand rehash since Symbol is one of them, and its Symbol table is rehashed anyway.
hash is computed from identityHash without change, but scaledIdentityHash is changed to avoid creating LargeIntegers (the larger identityHash needs less shift also): ProtoObjectPROTOTYPEscaledIdentityHash "For identityHash values returned by primitive 75, answer such values times 2^8. Otherwise, match the existing identityHash implementation"
^self identityHash * 256 "bitShift: 8"
- What will happen to primitive 138 and 139? Will we still be able to iterate over the objects in the order of their creation time?
That's much more difficult. I'll add allInstances and allObjects primitives which can answer the objects instantianeously without needing to enumerate. One issue is that the scavenger could fire at any time during the enumeration and could tenure objects to old space so its in general not possible to reliably enumerate via firstInstance/nextInstance firstObject/nextObject.
Being able to enumerate in order of creation time is something the VM could maintain in old space, keeping objects in order there-in. But I'd rather drop this property and allow the Spur GC to compact using best-fit and lazy become, which will change the order of objects. How important is being able to enumerate in order of creation? I think it's nice to have but hardly essential. What do y'all think?
- IIUC the memory is more segmented. Will there be new GC primitives for various levels of garbage collection? What about the GC parameters?
That remains to be seen. I take requests. I take design suggestions. But I also like to keep things simple.
My current approach is to keep things as compatible as possible to ease adoption. Once it is adopted we can start to evolve to provide appropriate and.or improved management facilities.
- Are #become: and #becomeForward: optimized for the case where both objects use the same amount of memory?
Yes. This case simply swaps contents and adjusts the remembered table accordingly. Tim also suggested optimizing the other case, copying into the smaller object and only allocating one extra clone, which I'll implement soon.
Levente
On Sat, 21 Sep 2013, btc@openinworld.com wrote:
commits@source.squeak.org wrote: Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.399.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.399 Author: eem Time: 20 September 2013, 6:28:56.308 pm UUID: 89f8fefe-b59d-42d7-9c11-7f848d0e5131 Ancestors: VMMaker.oscog-eem.398
A few isIntegerObject:'s => isImmediate:'s in primitives.
More protocol.
The Spur VM now draws its first window!!
A cheer goes up in the crowd of interested spectators. Probably lots still to do, but its a nice concrete milestone. Contributing is beyond me at this time, so I especially like to thank you for this important initiative.
cheers -ben
-- best, Eliot
On 2013-09-22, at 20:29, Eliot Miranda eliot.miranda@gmail.com wrote:
Being able to enumerate in order of creation time is something the VM could maintain in old space, keeping objects in order there-in. But I'd rather drop this property and allow the Spur GC to compact using best-fit and lazy become, which will change the order of objects. How important is being able to enumerate in order of creation? I think it's nice to have but hardly essential. What do y'all think?
The only use case I know of is being able to know when an object got created, enabling statistics on object age and fulfilling curiosity. Like, wouldn't it be fun to know the date that "nil" was created? ;)
E.g. in one of my images I have:
1: 23 September 2013->93207 2: 17 September 2013->568 3: 13 September 2013->1384 4: 12 September 2013->6067 5: 11 September 2013->21392 6: 9 September 2013->8 7: 6 September 2013->29 8: 5 September 2013->2193 9: 4 September 2013->14 10: 2 September 2013->13 11: 31 August 2013->14 12: 22 August 2013->1627 13: 20 August 2013->1151 14: 16 August 2013->2109 15: 14 August 2013->5046 16: 7 August 2013->25493 17: 26 May 2013->2452 18: 24 May 2013->1389 19: 8 May 2013->137 20: 7 May 2013->498 21: 6 May 2013->3976 22: 4 May 2013->1130 23: 3 May 2013->7627 24: 25 March 2013->3735 25: 4 March 2013->1210 26: 27 February 2013->428 27: 26 February 2013->2
(attaching a breakdown with classes below). See class ObjectHistory in Squeak trunk for implementation details.
I would not see that as an essential feature, but just as a nice exploit of the old GC behavior. If it is simple to preserve it would be nice to have, but it should not stand in the way of better performance.
- Bert -
1: 23 September 2013->{25457->ByteString . 14315->Array . 6146->MCVersionName . 6129->MCVersionInfo . 6129->UUID . 6066->DateAndTime . 6059->Time . 6059->Date . 5714->Association . 4077->MCMethodDefinition . 1065->Point . 526->OrderedCollection . 466->Rectangle . 347->MCInstanceVariableDefinition . 346->Interval . 227->MCModification . 185->Color . 164->MorphExtension . 162->IdentityDictionary . 160->MessageNode . 155->MCClassVariableDefinition . 152->LargePositiveInteger . 142->MCClassDefinition . 140->Dictionary . 124->ByteArray . 116->TextLine . 114->SHRange . 98->SelectorNode . 87->MethodContext . 85->Bitmap . 82->MCPackage . 82->BlockNode . 77->MCAddition . 77->WeakArray . 76->DependentsArray . 70->MCVersionDependency . 69->BlockClosure . 67->InstanceVariableNode . 62->RectangleMorph . 61->SimpleBorder . 60->CompiledMethod . 58->TempVariableNode . 57->Float . 53->LayoutProperties . 49->EventHandler . 48->ZipFileMember . 46->LeafNode . 44->MethodChangeRecord . 41->RunArray . 41->Text . 37->WriteStream . 34->WeakKeyAssociation . 31->ReturnNode . 30->TableLayoutProperties . 29->MenuItemMorph . 28->TableLayout . 28->GradientFillStyle . 26->LayoutFrame . 26->AssignmentNode . 26->Form . 24->WeakActionSequenceTrappingErrors . 24->ImageMorph . 20->WeakMessageSend . 19->AdditionalMethodState . 19->Set . 19->ClassChangeRecord . 19->IdentitySet . 18->CompilationCue . 18->ReadStream . 18->Parser . 18->PluggableDictionary . 18->EncoderForV3PlusClosures . 18->MethodNode . 18->DebuggerMethodMapForClosureCompiledMethods . 17->WeakFinalizerItem . 16->Semaphore . 15->AlignmentMorph . 14->MCRemoval . 14->StringMorph . 12->CharacterBlock . 12->LiteralNode . 12->ScrollBar . 11->MCDiffyVersion . 11->MCPatch . 11->MorphicTransform . 11->PluggableButtonMorphPlus . 9->ChangeSet . 9->ByteSymbol . 9->RWBinaryOrTextStream . 9->MultiByteFileStream . 8->MenuLineMorph . 8->MCMcdReader . 8->ZipArchive . 7->LiteralVariableNode . 7->Process . 7->MCClassInstanceVariableDefinition . 7->MCWorkingAncestry . 7->ObjectHistoryMark . 7->UTF8TextConverter . 6->SystemWindowButton . 6->MethodDictionary . 6->SketchMorph . 6->TransformMorph . 6->TextStyle . 5->MCSnapshot . 5->PluggableSet . 4->LazyListMorph . 4->ProportionalSplitterMorph . 4->TranslucentColor . 4->SmalltalkEditor . 4->MCPostscriptDefinition . 4->Pragma . 4->PluggableListMorphPlus . 4->MultiNewParagraph . 3->GrafPort . 3->Metaclass . 3->ProportionalLayout . 3->FormCanvas . 3->PluggablePanelMorph . 3->Delay . 3->MCOrganizationDefinition . 2->Object . 2->TextEmphasis . 2->StepMessage . 2->SharedQueue . 2->BraceNode . 2->PluggableTextMorphPlus . 2->RemoteString . 2->MethodReference . 2->WeakSet . 2->DamageRecorder . 2->TextMorphForEditView . 2->MouseEvent . 1->HandMorph . 1->CascadeNode . 1->MCMcmReader . 1->BottomLeftGripMorph . 1->ProjectLauncher . 1->MenuMorph . 1->WorldState . 1->TheWorldMenu . 1->ClassDiffBuilder class . 1->MCConfiguration . 1->Browser . 1->TopRightGripMorph . 1->PluggableSystemWindow . 1->LeftGripMorph . 1->SHParserST80 . 1->TopLeftGripMorph . 1->SHTextStylerST80 . 1->BottomGripMorph . 1->Message . 1->PrettyTextDiffBuilder class . 1->ExpandedSourceFileArray . 1->BottomRightGripMorph . 1->MCPreambleDefinition . 1->TextDiffBuilder class . 1->RightGripMorph . 1->Monitor . 1->TopGripMorph . 1->UpdatingMenuItemMorph . 1->MCVersion . 1->Heap . 1->UnixFileDirectory . 1->PasteUpMorph . 1->MouseOverHandler . 1->FilePath . 1->Morph} 2: 17 September 2013->{183->Point . 87->Rectangle . 63->Array . 42->Color . 31->Association . 25->SHRange . 11->MorphExtension . 10->RectangleMorph . 9->IdentityDictionary . 8->Float . 8->SimpleBorder . 8->EventHandler . 7->MethodContext . 7->ByteString . 6->MorphicTransform . 6->BlockClosure . 5->GradientFillStyle . 5->Interval . 5->Text . 4->ImageMorph . 4->CharacterBlock . 4->RunArray . 4->Bitmap . 3->TextLine . 3->PluggableSet . 2->Dictionary . 2->MultiNewParagraph . 2->TextStyle . 2->SmalltalkEditor . 2->Form . 1->LayoutFrame . 1->StringMorph . 1->Semaphore . 1->TextEmphasis . 1->WriteStream . 1->AlignmentMorph . 1->StepMessage . 1->TableLayoutProperties . 1->Process . 1->TableLayout} 3: 13 September 2013->{376->Point . 174->Association . 165->Rectangle . 138->Array . 82->Color . 73->MorphExtension . 58->IdentityDictionary . 32->RectangleMorph . 28->SimpleBorder . 25->EventHandler . 22->ByteArray . 18->TableLayoutProperties . 17->ByteString . 17->TableLayout . 16->LayoutFrame . 13->GradientFillStyle . 12->ImageMorph . 12->Float . 9->LayoutProperties . 8->StringMorph . 8->PluggableButtonMorphPlus . 8->AlignmentMorph . 6->ScrollBar . 5->OrderedCollection . 5->Bitmap . 4->MethodReference . 4->SystemWindowButton . 4->SketchMorph . 4->Form . 3->TransformMorph . 2->TextStyle . 2->Semaphore . 2->PluggableTextMorphPlus . 2->PluggablePanelMorph . 2->ProportionalLayout . 2->TextMorphForEditView . 1->MethodContext . 1->Dictionary . 1->Monitor . 1->MorphicTransform . 1->PluggableSet . 1->ProportionalSplitterMorph . 1->TextLine . 1->Morph . 1->RightGripMorph . 1->PluggableListMorphPlus . 1->DependentsArray . 1->PluggableSystemWindow . 1->StepMessage . 1->LeftGripMorph . 1->BottomRightGripMorph . 1->SHTextStylerST80 . 1->BlockClosure . 1->MessageSet . 1->TranslucentColor . 1->TopGripMorph . 1->BottomLeftGripMorph . 1->LazyListMorph . 1->BottomGripMorph . 1->SHParserST80 . 1->TopLeftGripMorph . 1->TopRightGripMorph} 4: 12 September 2013->{1045->Point . 771->Array . 758->Association . 450->Rectangle . 329->ByteString . 262->TextLine . 250->Color . 216->IdentityDictionary . 209->SHRange . 191->CompiledMethod . 185->MorphExtension . 160->MethodChangeRecord . 89->RectangleMorph . 82->SimpleBorder . 70->EventHandler . 67->IdentitySet . 67->ClassChangeRecord . 52->Float . 44->TableLayoutProperties . 41->LayoutFrame . 41->TableLayout . 37->GradientFillStyle . 36->Dictionary . 34->ImageMorph . 31->OrderedCollection . 28->MCVersionName . 26->UUID . 26->Date . 26->DateAndTime . 26->MCVersionInfo . 26->Time . 25->Bitmap . 25->ChangeSet . 24->LayoutProperties . 20->StringMorph . 20->AlignmentMorph . 20->Form . 19->PluggableButtonMorphPlus . 18->ScrollBar . 13->MCWorkingAncestry . 11->ByteSymbol . 9->MethodReference . 9->TransformMorph . 8->SystemWindowButton . 8->SketchMorph . 7->MethodDictionary . 6->TextStyle . 6->RunArray . 6->CharacterBlock . 6->Text . 6->Interval . 5->WeakArray . 5->MorphicTransform . 5->ProportionalSplitterMorph . 5->PluggableListMorphPlus . 5->PluggablePanelMorph . 5->ProportionalLayout . 5->LazyListMorph . 4->Semaphore . 4->PluggableTextMorphPlus . 4->TextMorphForEditView . 3->MethodContext . 3->PluggableSet . 3->Set . 3->StepMessage . 3->BlockClosure . 3->Metaclass . 3->WeakSet . 3->AdditionalMethodState . 3->Object . 2->ClassOrganizer . 2->Monitor . 2->LargePositiveInteger . 2->Morph . 2->RightGripMorph . 2->DependentsArray . 2->PluggableSystemWindow . 2->LeftGripMorph . 2->BottomRightGripMorph . 2->SHTextStylerST80 . 2->Pragma . 2->PragmaPreference . 2->TranslucentColor . 2->TopGripMorph . 2->BottomLeftGripMorph . 2->BottomGripMorph . 2->MultiNewParagraph . 2->SHParserST80 . 2->TopLeftGripMorph . 2->TopRightGripMorph . 2->SmalltalkEditor . 1->WriteStream . 1->TimeStamp . 1->JPEGReadWriter2Test class . 1->Bag . 1->ClassBinding . 1->MessageSet . 1->MultiDisplayScanner class . 1->RemoteString . 1->Browser . 1->ProportionalSplitterMorph class} 5: 11 September 2013->{4270->ByteString . 4269->Array . 2141->MCVersionName . 2134->UUID . 2134->Time . 2134->Date . 2134->DateAndTime . 2134->MCVersionInfo . 35->ByteSymbol . 4->Point . 2->Rectangle . 1->TextLine} 6: 9 September 2013->{3->TextLine . 2->ByteArray . 1->Rectangle . 1->Interval . 1->Point} 7: 6 September 2013->{4->Association . 4->Array . 3->ByteString . 2->ClassChangeRecord . 2->IdentityDictionary . 2->MethodReference . 2->MethodChangeRecord . 2->IdentitySet . 1->ByteSymbol . 1->Global . 1->ObjectHistoryMark . 1->DateAndTime . 1->WriteStream . 1->WeakArray . 1->CompiledMethod . 1->BrowserRequestor} 8: 5 September 2013->{343->Point . 310->Association . 218->Array . 211->ByteString . 176->MethodChangeRecord . 176->CompiledMethod . 172->ByteSymbol . 160->Rectangle . 54->Color . 28->IdentityDictionary . 24->AdditionalMethodState . 21->Pragma . 18->ClassOrganizer . 18->MethodDictionary . 18->ByteArray . 18->IdentitySet . 18->ClassChangeRecord . 16->Bitmap . 15->RectangleMorph . 15->MorphExtension . 14->Float . 12->EventHandler . 10->Form . 9->MethodReference . 9->ClassBinding . 9->SimpleBorder . 9->Metaclass . 9->RemoteString . 7->GradientFillStyle . 6->ImageMorph . 4->Dictionary . 4->OrderedCollection . 4->UUID . 4->LargePositiveInteger . 4->MCVersionName . 4->Date . 4->DateAndTime . 4->MCVersionInfo . 4->Time . 3->WeakArray . 3->TextLine . 3->ChangeSet . 2->PackageInfo . 2->MCWorkingAncestry . 2->MCWorkingCopy . 2->WeakKeyAssociation . 2->MCRepositoryGroup . 2->MCPackage . 1->Interval . 1->WriteStream . 1->ComponentInstance class . 1->OSAID class . 1->AEDesc class . 1->ApplescriptInstance class . 1->DescType class . 1->MCHttpRepository . 1->Applescript class . 1->MacExternalData class . 1->ApplescriptError class . 1->CompiledApplescript class . 1->BrowserRequestor} 9: 4 September 2013->{3->Association . 3->Float . 3->Color . 2->Bitmap . 2->Array . 1->TextLine} 10: 2 September 2013->{3->Association . 3->Float . 3->Color . 2->Bitmap . 2->Array} 11: 31 August 2013->{3->Association . 3->Float . 3->Color . 2->Bitmap . 2->Array . 1->TextLine} 12: 22 August 2013->{329->Array . 258->ByteString . 145->Association . 105->MCVersionName . 104->Date . 104->UUID . 104->DateAndTime . 104->Time . 104->MCVersionInfo . 41->Interval . 19->MethodChangeRecord . 17->Point . 16->MessageNode . 15->TextLine . 15->CompiledMethod . 15->OrderedCollection . 14->ByteArray . 12->Dictionary . 11->SelectorNode . 9->ClassChangeRecord . 9->InstanceVariableNode . 9->IdentityDictionary . 9->IdentitySet . 6->ByteSymbol . 5->LiteralNode . 5->MethodReference . 3->BlockNode . 3->ChangeSet . 2->RemoteString . 2->MethodDictionary . 2->RunArray . 2->Rectangle . 2->WriteStream . 2->Text . 2->WeakArray . 2->Form . 1->MethodNode . 1->Parser . 1->MethodContext . 1->Metaclass . 1->ReadStream . 1->ReturnNode . 1->LargePositiveInteger . 1->LiteralVariableNode . 1->EncoderForV3PlusClosures . 1->TextEmphasis . 1->CompilationCue . 1->DebuggerMethodMapForClosureCompiledMethods . 1->BlockClosure . 1->PluggableDictionary . 1->Set . 1->MCInfoProxy class . 1->AssignmentNode . 1->TempVariableNode . 1->LeafNode . 1->AdditionalMethodState . 1->WeakKeyAssociation} 13: 20 August 2013->{231->Array . 228->ByteString . 114->Association . 57->MethodChangeRecord . 50->CompiledMethod . 47->AdditionalMethodState . 45->MCVersionName . 44->Pragma . 43->DateAndTime . 43->Date . 43->Time . 43->MCVersionInfo . 43->UUID . 31->ByteSymbol . 30->TextLine . 12->IdentityDictionary . 11->ClassChangeRecord . 11->IdentitySet . 8->Dictionary . 3->ChangeSet . 2->MethodDictionary . 2->ClassOrganizer . 2->ByteArray . 2->OrderedCollection . 1->ClassBinding . 1->FullVocabulary . 1->InstallerTestSuite class . 1->Bitmap . 1->MCWorkingAncestry . 1->Metaclass} 14: 16 August 2013->{509->ByteString . 422->CompiledMethod . 284->Array . 189->Association . 77->TextLine . 68->MethodChangeRecord . 54->MCVersionName . 47->Date . 47->Time . 47->UUID . 47->DateAndTime . 47->MCVersionInfo . 38->ByteSymbol . 33->ClassChangeRecord . 33->IdentityDictionary . 33->IdentitySet . 31->MethodDictionary . 15->Metaclass . 14->Dictionary . 9->Float . 9->Color . 8->ChangeSet . 6->Bitmap . 4->AdditionalMethodState . 4->ClassOrganizer . 3->MethodReference . 3->Point . 2->RemoteString . 2->Rectangle . 2->ClassBinding . 2->ByteArray . 1->Form . 1->PragmaPreference . 1->Pragma . 1->InstallerUrl class . 1->StringHolder . 1->InstallerUpdateStream class . 1->InstallerSqueakMap class . 1->InstallerUniverse class . 1->InstallerFile class . 1->InstallerWebBased class . 1->Installer class . 1->InstallerWeb class . 1->InstallerMonticello class . 1->UTF8ClipboardInterpreter . 1->InstallerWebSqueakMap class . 1->InstallerCruft class . 1->MCProxyMaterialization class . 1->InstallerInternetBased class . 1->InstallerMantis class . 1->InstallerSake class} 15: 14 August 2013->{980->ByteString . 832->Array . 531->Association . 307->MCVersionName . 301->Date . 301->Time . 301->UUID . 301->DateAndTime . 301->MCVersionInfo . 281->MethodChangeRecord . 277->CompiledMethod . 63->ClassChangeRecord . 63->IdentityDictionary . 63->IdentitySet . 55->TextLine . 22->Dictionary . 20->ChangeSet . 9->ByteSymbol . 6->Float . 5->OrderedCollection . 5->MCWorkingAncestry . 4->Pragma . 4->AdditionalMethodState . 2->RemoteString . 2->MethodReference . 1->PragmaPreference . 1->PackageInfo . 1->MethodDictionary . 1->MCPackage . 1->MCWorkingCopy . 1->WriteStream . 1->WeakArray . 1->Point . 1->WeakKeyAssociation . 1->MCRepositoryGroup} 16: 7 August 2013->{4764->Array . 3999->ByteString . 2820->Association . 1806->CompiledMethod . 1775->MethodChangeRecord . 1441->MCVersionName . 1365->UUID . 1365->Date . 1365->DateAndTime . 1365->MCVersionInfo . 1365->Time . 393->IdentitySet . 393->IdentityDictionary . 393->ClassChangeRecord . 159->ByteSymbol . 137->Dictionary . 116->Float . 90->ChangeSet . 72->LinedTTCFont . 69->RemoteString . 33->WeakArray . 31->WeakKeyAssociation . 27->MethodDictionary . 26->LargePositiveInteger . 16->OrderedCollection . 13->Set . 12->ClassOrganizer . 12->MCWorkingAncestry . 11->Metaclass . 6->Pragma . 6->AdditionalMethodState . 5->TTCFont . 5->ClassBinding . 4->TimeStamp . 4->Point . 2->PackageInfo . 2->MCWorkingCopy . 2->MCRepositoryGroup . 2->Rectangle . 2->MCPackage . 1->ST80PackageDependencyTest class . 1->WeakFinalizationList . 1->NullMutex class . 1->MacUnicodeInputInterpreter . 1->ClassVarScopeTest class . 1->ReplaceExistingFileException class . 1->StickynessBugz class . 1->PluggableMenuItemSpec class . 1->ScrapBook class . 1->InfiniteForm . 1->PluggableMenuItemSpecTests class . 1->Promise class . 1->Environment class . 1->ImmAbstractPlatform . 1->WeakIdentityKeyDictionary . 1->MorphicProject class . 1->MorphicProject . 1->Environment . 1->StringHolder . 1->Form} 17: 26 May 2013->{478->CompiledMethod . 424->Association . 368->Array . 338->ByteString . 235->MethodChangeRecord . 58->ClassChangeRecord . 58->IdentityDictionary . 58->IdentitySet . 51->MCVersionName . 46->ByteSymbol . 33->Dictionary . 32->Date . 32->Time . 32->UUID . 32->DateAndTime . 32->MCVersionInfo . 30->MethodDictionary . 21->ChangeSet . 15->Metaclass . 9->Set . 7->Float . 7->RemoteString . 6->AdditionalMethodState . 6->ClassOrganizer . 4->MCHttpRepository . 3->Pragma . 3->TimeStamp . 3->LargePositiveInteger . 3->ClassBinding . 3->OrderedCollection . 2->MCWorkingAncestry . 2->LazyListMorph . 2->SystemNavigation . 1->PragmaPreference . 1->MCFtpRepository class . 1->MCHttpRepository class . 1->MCSubDirectoryRepository class . 1->MCFileBasedRepository class . 1->MCCacheRepository class . 1->MCDirectoryRepository . 1->MCCacheRepository . 1->MethodReferenceTest class . 1->MCSMCacheRepository class . 1->SystemNavigation class . 1->MethodReference class . 1->Bitmap . 1->LazyListMorph class . 1->MulticolumnLazyListMorph class . 1->MCDirectoryRepository class . 1->InvalidUTF8 class . 1->SystemNavigationTest class . 1->MCEnvironmentLoadTest class} 18: 24 May 2013->{265->Association . 210->Array . 158->Point . 75->MethodChangeRecord . 70->ByteString . 67->Rectangle . 56->WeakArray . 56->WeakKeyAssociation . 52->IdentityDictionary . 33->MorphExtension . 23->Float . 20->TableLayoutProperties . 20->IdentitySet . 20->TableLayout . 20->ClassChangeRecord . 19->Dictionary . 18->CompiledMethod . 16->SimpleBorder . 15->Set . 13->AdditionalMethodState . 12->Color . 11->Form . 10->StringMorph . 10->Bitmap . 10->AlignmentMorph . 9->ChangeSet . 9->PluggableButtonMorphPlus . 8->MCVersionName . 5->TimeStamp . 5->LayoutFrame . 4->ScrollBar . 4->MethodDictionary . 4->SystemWindowButton . 4->SketchMorph . 3->MethodContext . 3->RaisedBorder . 3->LayoutProperties . 3->ByteSymbol . 3->ByteArray . 3->GradientFillStyle . 2->ClassOrganizer . 2->UUID . 2->LargePositiveInteger . 2->PluggableListMorphPlus . 2->TransformMorph . 2->BlockClosure . 2->RectangleMorph . 2->Metaclass . 2->RemoteString . 2->Date . 2->DateAndTime . 2->MCVersionInfo . 2->Time . 1->OrderedCollection . 1->MCWorkingAncestry . 1->Morph . 1->DependentsArray . 1->ClassBinding . 1->PluggableSystemWindow . 1->MorphicUIManagerTest class . 1->RenderBugz class . 1->PluggablePanelMorph . 1->TranslucentColor . 1->ProportionalLayout . 1->Process . 1->MCWorkingCopyBrowser . 1->EventHandler} 19: 8 May 2013->{25->Association . 25->Array . 19->MethodChangeRecord . 15->ByteString . 14->CompiledMethod . 6->ByteSymbol . 5->ClassChangeRecord . 5->IdentityDictionary . 5->IdentitySet . 4->ChangeSet . 4->MCVersionName . 4->Dictionary . 2->MethodDictionary . 2->ByteArray . 1->OrderedCollection . 1->MCWorkingAncestry} 20: 7 May 2013->{123->Point . 67->Association . 52->Rectangle . 52->Array . 32->MorphExtension . 27->IdentityDictionary . 27->Color . 12->RectangleMorph . 11->LayoutFrame . 11->SimpleBorder . 9->EventHandler . 8->Float . 8->Form . 6->LayoutProperties . 4->ImageMorph . 4->ByteString . 4->SketchMorph . 4->GradientFillStyle . 4->SystemWindowButton . 3->Bitmap . 2->ScrollBar . 2->TextStyle . 1->StringMorph . 1->Dictionary . 1->ProportionalLayout . 1->TranslucentColor . 1->DependentsArray . 1->BottomLeftGripMorph . 1->PluggableTextMorphPlus . 1->Workspace . 1->BottomRightGripMorph . 1->TransformMorph . 1->LeftGripMorph . 1->TopRightGripMorph . 1->MultiNewParagraph . 1->PluggableSystemWindow . 1->Morph . 1->SHTextStylerST80 . 1->AlignmentMorph . 1->TextMorphForEditView . 1->BottomGripMorph . 1->RightGripMorph . 1->SmalltalkEditor . 1->Text . 1->TableLayoutProperties . 1->TopLeftGripMorph . 1->TopGripMorph . 1->TableLayout} 21: 6 May 2013->{827->ByteString . 765->Array . 321->MCVersionName . 316->Date . 316->UUID . 316->Time . 316->DateAndTime . 316->MCVersionInfo . 154->Association . 94->MethodChangeRecord . 65->CompiledMethod . 17->ClassChangeRecord . 17->IdentityDictionary . 17->IdentitySet . 16->AdditionalMethodState . 14->ByteArray . 13->Dictionary . 12->ByteSymbol . 8->Color . 8->ChangeSet . 7->Float . 4->MethodDictionary . 4->Set . 4->Bitmap . 4->ClassOrganizer . 4->WeakArray . 2->Metaclass . 2->PackageInfo . 2->ClassBinding . 2->WeakKeyAssociation . 1->MCWorkingAncestry . 1->MCReorganizationPreloader class . 1->BrowserRequestor . 1->RemoteString . 1->TimeStamp . 1->LargePositiveInteger . 1->InstallerUrlTest class . 1->FilePath . 1->UnixFileDirectory . 1->WriteStream . 1->OrderedCollection . 1->Form . 1->UTF8TextConverter} 22: 4 May 2013->{288->Array . 237->Association . 133->MethodChangeRecord . 87->ByteString . 48->IdentityDictionary . 32->CompiledMethod . 32->MorphExtension . 27->Color . 23->ByteSymbol . 21->IdentitySet . 21->ClassChangeRecord . 16->Point . 13->Float . 12->RectangleMorph . 11->LayoutFrame . 11->SimpleBorder . 9->EventHandler . 8->MethodDictionary . 8->Form . 6->ClassOrganizer . 6->LayoutProperties . 5->ByteArray . 4->Dictionary . 4->ImageMorph . 4->Bitmap . 4->SystemWindowButton . 4->SketchMorph . 4->GradientFillStyle . 3->ColorArray . 3->WeakArray . 3->ClassBinding . 3->Metaclass . 2->ScrollBar . 2->TextStyle . 2->ChangeSet . 2->MCVersionName . 2->WeakKeyAssociation . 2->RemoteString . 2->Rectangle . 1->StringMorph . 1->PluggableTextMorphPlus . 1->ReleaseBuilderDeveloper class . 1->Morph . 1->RightGripMorph . 1->PluggableSystemWindow . 1->TransformMorph . 1->LeftGripMorph . 1->BottomRightGripMorph . 1->ReleaseBuilderSqueakland class . 1->TranslucentColor . 1->TopGripMorph . 1->TableLayoutProperties . 1->BottomLeftGripMorph . 1->Text . 1->ProportionalLayout . 1->TableLayout . 1->BottomGripMorph . 1->AdditionalMethodState . 1->MultiNewParagraph . 1->AlignmentMorph . 1->TopLeftGripMorph . 1->ReleaseBuilderNihongo class . 1->TextMorphForEditView . 1->TopRightGripMorph . 1->SmalltalkEditor} 23: 3 May 2013->{2381->ClassBinding . 1217->Array . 771->CompiledMethod . 741->ByteString . 682->Association . 400->MethodChangeRecord . 146->IdentitySet . 146->IdentityDictionary . 146->ClassChangeRecord . 115->MCVersionName . 92->Float . 74->ByteSymbol . 67->UUID . 67->Date . 67->DateAndTime . 67->MCVersionInfo . 67->Time . 66->Dictionary . 53->MethodDictionary . 51->ChangeSet . 41->Alias . 25->Global . 23->Metaclass . 15->Set . 13->LargePositiveInteger . 12->ClassOrganizer . 9->AdditionalMethodState . 8->OrderedCollection . 7->Pragma . 7->LargeNegativeInteger . 6->MCWorkingAncestry . 5->TimeStamp . 3->RemoteString . 2->WeakArray . 2->PackageInfo . 2->MCWorkingCopy . 2->WeakKeyAssociation . 2->MCRepositoryGroup . 2->MCPackage . 1->MCVersionInspector class . 1->Binding class . 1->MCWorkingHistoryBrowser class . 1->ClassBindingTest class . 1->MCSnapshotBrowser class . 1->MCTool class . 1->SystemReporter class . 1->MCFileRepositoryInspector class . 1->Bitmap . 1->MCPatchBrowser class . 1->MCVersionHistoryBrowser class . 1->ScaledDecimal . 1->EToyExpressionTransformer2 class . 1->MCRepositoryInspector class . 1->Global class . 1->DebuggerExtensionsTest class . 1->ByteEncoderTest class . 1->MCMergeBrowser class . 1->MCChangeSelector class . 1->MCSaveVersionDialog class . 1->MCCodeTool class . 1->MCConfigurationBrowser class . 1->Alias class . 1->MCWorkingCopyBrowser class . 1->ClassBinding class} 24: 25 March 2013->{715->Association . 633->Array . 444->CompiledMethod . 408->MethodChangeRecord . 282->ByteString . 195->ByteSymbol . 133->Point . 104->IdentityDictionary . 98->Float . 97->IdentitySet . 97->ClassChangeRecord . 68->MethodDictionary . 60->Rectangle . 48->ClassOrganizer . 48->Dictionary . 48->Set . 33->Metaclass . 24->ChangeSet . 24->MCVersionName . 22->Bitmap . 18->AdditionalMethodState . 17->Color . 16->TimeStamp . 12->WeakArray . 12->WeakKeyAssociation . 10->Pragma . 8->RemoteString . 4->OrderedCollection . 3->AllNamePolicy . 3->GradientFillStyle . 2->Import . 2->RaisedBorder . 2->WeakSet . 2->Object . 1->FunctionTile class . 1->CharacterBlockScanner class . 1->NamePolicy class . 1->PackageInfo . 1->EnvironmentInfo . 1->ImportTest class . 1->MultiCompositionScanner class . 1->MCWorkingAncestry . 1->MultiCanvasCharacterScanner class . 1->CompositionScanner class . 1->ExplicitNamePolicy class . 1->AliasTest class . 1->LargePositiveInteger . 1->DisplayScanner class . 1->AddPrefixNamePolicyTest class . 1->AddPrefixNamePolicy class . 1->CharacterScanner class . 1->RemovePrefixNamePolicyTest class . 1->MorphExtensionPlus class . 1->FunctionNameTile class . 1->ExplicitNamePolicyTest class . 1->NamePolicyTest class . 1->MultiCharacterBlockScanner class . 1->GlobalTest class . 1->Export . 1->MCWorkingCopy . 1->AllNamePolicy class . 1->Import class . 1->BindingPolicy class . 1->EToysLauncher class . 1->MCRepositoryGroup . 1->MultiCharacterScanner class . 1->RemovePrefixNamePolicy class . 1->WideSymbol . 1->AllNamePolicyTest class . 1->Export class . 1->ExportTest class . 1->MCPackage . 1->BindingPolicyTest class . 1->SegmentScanner class . 1->Form . 1->AnObsoleteBindingTest class . 1->CanvasCharacterScanner class} 25: 4 March 2013->{294->Array . 185->ByteString . 94->MCVersionName . 81->DateAndTime . 81->Date . 81->Time . 81->MCVersionInfo . 81->UUID . 65->Association . 34->MethodChangeRecord . 30->ClassChangeRecord . 30->IdentityDictionary . 30->IdentitySet . 14->Dictionary . 14->ChangeSet . 9->CompiledMethod . 2->MCWorkingAncestry . 2->OrderedCollection . 1->Point . 1->ByteSymbol} 26: 27 February 2013->{115->Association . 115->Array . 105->TextColor . 67->Color . 18->TextEmphasis . 4->Float . 2->Bitmap . 1->IdentityDictionary . 1->Dictionary} 27: 26 February 2013->{1->Association . 1->AdditionalMethodState}
Being able to enumerate in order of creation time is something the VM could maintain in old space, keeping objects in order there-in. But I'd rather drop this property and allow the Spur GC to compact using best-fit and lazy become, which will change the order of objects. How important is being able to enumerate in order of creation? I think it's nice to have but hardly essential. What do y'all think?
Any application that would care about it would not rely on some lucky implementation side-effect, it would solve the ordering requirement explicitly by its own code.
On 22 September 2013 20:29, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Levente,
On Sat, Sep 21, 2013 at 11:39 AM, Levente Uzonyi leves@elte.hu wrote:
Great progress. I've got a few questions about Spur and the bootstrap:
- How will HashedCollections other than MethodDictionaries get rehashed?
How #hash be calculated from #identityHash?
The bootstrap first builds an image in the new format and then finds all classes in it that implement rehash and then uses the simulator to send rehash to all objects to use their own rehash method to rehash them, instead of assuming their format. In fact, the bootstrap does not rehash all objects that understand rehash since Symbol is one of them, and its Symbol table is rehashed anyway.
hash is computed from identityHash without change, but scaledIdentityHash is changed to avoid creating LargeIntegers (the larger identityHash needs less shift also): ProtoObjectPROTOTYPEscaledIdentityHash "For identityHash values returned by primitive 75, answer such values times 2^8. Otherwise, match the existing identityHash implementation"
^self identityHash * 256 "bitShift: 8"
- What will happen to primitive 138 and 139? Will we still be able to
iterate over the objects in the order of their creation time?
That's much more difficult. I'll add allInstances and allObjects primitives which can answer the objects instantianeously without needing to enumerate. One issue is that the scavenger could fire at any time during the enumeration and could tenure objects to old space so its in general not possible to reliably enumerate via firstInstance/nextInstance firstObject/nextObject.
indeed. i think we could breathe more freely if VM wouldn't be obliged to expose heap enumeration/walking primitives.. I just wonder, if it is possible to provide alternatives for most cases: - finding all instances - finding all pointers to subject so the heap walking primitives can R.I.P. .. but there could be more, which we may need and which would require heap walking. what you think?
Being able to enumerate in order of creation time is something the VM could maintain in old space, keeping objects in order there-in. But I'd rather drop this property and allow the Spur GC to compact using best-fit and lazy become, which will change the order of objects. How important is being able to enumerate in order of creation? I think it's nice to have but hardly essential. What do y'all think?
IMO, i would never make any assumptions, in which order objects are located on heap nor in which order they get enumerated. "Heap" word says for itself here. (and anyone who thinks otherwise should be shot on sight :)
- IIUC the memory is more segmented. Will there be new GC primitives for
various levels of garbage collection? What about the GC parameters?
That remains to be seen. I take requests. I take design suggestions. But I also like to keep things simple.
My current approach is to keep things as compatible as possible to ease adoption. Once it is adopted we can start to evolve to provide appropriate and.or improved management facilities.
- Are #become: and #becomeForward: optimized for the case where both
objects use the same amount of memory?
Yes. This case simply swaps contents and adjusts the remembered table accordingly. Tim also suggested optimizing the other case, copying into the smaller object and only allocating one extra clone, which I'll implement soon.
Levente
On Sat, 21 Sep 2013, btc@openinworld.com wrote:
commits@source.squeak.org wrote:
Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/**VMMaker/VMMaker.oscog-eem.399.**mczhttp://source.squeak.org/VMMaker/VMMaker.oscog-eem.399.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.399 Author: eem Time: 20 September 2013, 6:28:56.308 pm UUID: 89f8fefe-b59d-42d7-9c11-**7f848d0e5131 Ancestors: VMMaker.oscog-eem.398
A few isIntegerObject:'s => isImmediate:'s in primitives.
More protocol.
The Spur VM now draws its first window!!
A cheer goes up in the crowd of interested spectators.
Probably lots still to do, but its a nice concrete milestone. Contributing is beyond me at this time, so I especially like to thank you for this important initiative.
cheers -ben
-- best, Eliot
On 20 October 2013 15:42, Igor Stasenko siguctua@gmail.com wrote:
On 22 September 2013 20:29, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Levente,
On Sat, Sep 21, 2013 at 11:39 AM, Levente Uzonyi leves@elte.hu wrote:
Great progress. I've got a few questions about Spur and the bootstrap:
- How will HashedCollections other than MethodDictionaries get rehashed?
How #hash be calculated from #identityHash?
The bootstrap first builds an image in the new format and then finds all classes in it that implement rehash and then uses the simulator to send rehash to all objects to use their own rehash method to rehash them, instead of assuming their format. In fact, the bootstrap does not rehash all objects that understand rehash since Symbol is one of them, and its Symbol table is rehashed anyway.
hash is computed from identityHash without change, but scaledIdentityHash is changed to avoid creating LargeIntegers (the larger identityHash needs less shift also): ProtoObjectPROTOTYPEscaledIdentityHash "For identityHash values returned by primitive 75, answer such values times 2^8. Otherwise, match the existing identityHash implementation"
^self identityHash * 256 "bitShift: 8"
- What will happen to primitive 138 and 139? Will we still be able to
iterate over the objects in the order of their creation time?
That's much more difficult. I'll add allInstances and allObjects primitives which can answer the objects instantianeously without needing to enumerate. One issue is that the scavenger could fire at any time during the enumeration and could tenure objects to old space so its in general not possible to reliably enumerate via firstInstance/nextInstance firstObject/nextObject.
indeed. i think we could breathe more freely if VM wouldn't be obliged to expose heap enumeration/walking primitives.. I just wonder, if it is possible to provide alternatives for most cases:
- finding all instances
- finding all pointers to subject
so the heap walking primitives can R.I.P. .. but there could be more, which we may need and which would require heap walking. what you think?
more for this matter: i think for all kinds of queries , like "gimme all
object meeting certain criteria", there could be other way.
Basically, we need 1 primitive: primitive takes 2 arguments: an object, and array the first argument defines the root object which VM should traverse and answer the (possibly) huge array of all objects it can reach starting from it. the second argument is simply a list of objects which VM should ignore while traversing the graph this gives the opportunity for user to define a filter which can be used during scanning, instead of filtering afterwards.
(as variant, there could be a 3rd argument - an array which will contain the answer, like that if array not big enough to fit all found references, primitive fails, otherwise primitive answers the number of filled indices in that array. this gives opportunity for user to control the size of array and preallocate it before scanning.)
I think, with such primitive, we will no longer need heap walking primitives.
Being able to enumerate in order of creation time is something the VM
could maintain in old space, keeping objects in order there-in. But I'd rather drop this property and allow the Spur GC to compact using best-fit and lazy become, which will change the order of objects. How important is being able to enumerate in order of creation? I think it's nice to have but hardly essential. What do y'all think?
IMO, i would never make any assumptions, in which order objects are located on heap nor in which order they get enumerated. "Heap" word says for itself here. (and anyone who thinks otherwise should be shot on sight :)
- IIUC the memory is more segmented. Will there be new GC primitives for
various levels of garbage collection? What about the GC parameters?
That remains to be seen. I take requests. I take design suggestions. But I also like to keep things simple.
My current approach is to keep things as compatible as possible to ease adoption. Once it is adopted we can start to evolve to provide appropriate and.or improved management facilities.
- Are #become: and #becomeForward: optimized for the case where both
objects use the same amount of memory?
Yes. This case simply swaps contents and adjusts the remembered table accordingly. Tim also suggested optimizing the other case, copying into the smaller object and only allocating one extra clone, which I'll implement soon.
Levente
On Sat, 21 Sep 2013, btc@openinworld.com wrote:
commits@source.squeak.org wrote:
Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/**VMMaker/VMMaker.oscog-eem.399.**mczhttp://source.squeak.org/VMMaker/VMMaker.oscog-eem.399.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.399 Author: eem Time: 20 September 2013, 6:28:56.308 pm UUID: 89f8fefe-b59d-42d7-9c11-**7f848d0e5131 Ancestors: VMMaker.oscog-eem.398
A few isIntegerObject:'s => isImmediate:'s in primitives.
More protocol.
The Spur VM now draws its first window!!
A cheer goes up in the crowd of interested spectators.
Probably lots still to do, but its a nice concrete milestone. Contributing is beyond me at this time, so I especially like to thank you for this important initiative.
cheers -ben
-- best, Eliot
-- Best regards, Igor Stasenko.
On 20 October 2013 15:59, Igor Stasenko siguctua@gmail.com wrote:
On 20 October 2013 15:42, Igor Stasenko siguctua@gmail.com wrote:
On 22 September 2013 20:29, Eliot Miranda eliot.miranda@gmail.comwrote:
Hi Levente,
On Sat, Sep 21, 2013 at 11:39 AM, Levente Uzonyi leves@elte.hu wrote:
Great progress. I've got a few questions about Spur and the bootstrap:
- How will HashedCollections other than MethodDictionaries get
rehashed? How #hash be calculated from #identityHash?
The bootstrap first builds an image in the new format and then finds all classes in it that implement rehash and then uses the simulator to send rehash to all objects to use their own rehash method to rehash them, instead of assuming their format. In fact, the bootstrap does not rehash all objects that understand rehash since Symbol is one of them, and its Symbol table is rehashed anyway.
hash is computed from identityHash without change, but scaledIdentityHash is changed to avoid creating LargeIntegers (the larger identityHash needs less shift also): ProtoObjectPROTOTYPEscaledIdentityHash "For identityHash values returned by primitive 75, answer such values times 2^8. Otherwise, match the existing identityHash implementation"
^self identityHash * 256 "bitShift: 8"
- What will happen to primitive 138 and 139? Will we still be able to
iterate over the objects in the order of their creation time?
That's much more difficult. I'll add allInstances and allObjects primitives which can answer the objects instantianeously without needing to enumerate. One issue is that the scavenger could fire at any time during the enumeration and could tenure objects to old space so its in general not possible to reliably enumerate via firstInstance/nextInstance firstObject/nextObject.
indeed. i think we could breathe more freely if VM wouldn't be obliged to expose heap enumeration/walking primitives.. I just wonder, if it is possible to provide alternatives for most cases:
- finding all instances
- finding all pointers to subject
so the heap walking primitives can R.I.P. .. but there could be more, which we may need and which would require heap walking. what you think?
more for this matter: i think for all kinds of queries , like "gimme all
object meeting certain criteria", there could be other way.
Basically, we need 1 primitive: primitive takes 2 arguments: an object, and array the first argument defines the root object which VM should traverse and answer the (possibly) huge array of all objects it can reach starting from it. the second argument is simply a list of objects which VM should ignore while traversing the graph this gives the opportunity for user to define a filter which can be used during scanning, instead of filtering afterwards.
(as variant, there could be a 3rd argument - an array which will contain the answer, like that if array not big enough to fit all found references, primitive fails, otherwise primitive answers the number of filled indices in that array. this gives opportunity for user to control the size of array and preallocate it before scanning.)
I think, with such primitive, we will no longer need heap walking primitives.
please, note that this primitive does not walks heap linearly but scans the graph. also note, that such primitive can be really useful for serialization (to find all objects one needs to serialize for a certain subgraph) , and improve speed of this particular task considerably. also note, that with such primitive you no longer need to worry about "objects moving/new objects created while walking the heap", which certainly a weak point of current design. And as a bonus, one don't needs to be concerned with the order of objects on heap anymore. :)
Being able to enumerate in order of creation time is something the VM
could maintain in old space, keeping objects in order there-in. But I'd rather drop this property and allow the Spur GC to compact using best-fit and lazy become, which will change the order of objects. How important is being able to enumerate in order of creation? I think it's nice to have but hardly essential. What do y'all think?
IMO, i would never make any assumptions, in which order objects are located on heap nor in which order they get enumerated. "Heap" word says for itself here. (and anyone who thinks otherwise should be shot on sight :)
- IIUC the memory is more segmented. Will there be new GC primitives
for various levels of garbage collection? What about the GC parameters?
That remains to be seen. I take requests. I take design suggestions. But I also like to keep things simple.
My current approach is to keep things as compatible as possible to ease adoption. Once it is adopted we can start to evolve to provide appropriate and.or improved management facilities.
- Are #become: and #becomeForward: optimized for the case where both
objects use the same amount of memory?
Yes. This case simply swaps contents and adjusts the remembered table accordingly. Tim also suggested optimizing the other case, copying into the smaller object and only allocating one extra clone, which I'll implement soon.
Levente
On Sat, 21 Sep 2013, btc@openinworld.com wrote:
commits@source.squeak.org wrote:
Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/**VMMaker/VMMaker.oscog-eem.399.**mczhttp://source.squeak.org/VMMaker/VMMaker.oscog-eem.399.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.399 Author: eem Time: 20 September 2013, 6:28:56.308 pm UUID: 89f8fefe-b59d-42d7-9c11-**7f848d0e5131 Ancestors: VMMaker.oscog-eem.398
A few isIntegerObject:'s => isImmediate:'s in primitives.
More protocol.
The Spur VM now draws its first window!!
A cheer goes up in the crowd of interested spectators.
Probably lots still to do, but its a nice concrete milestone. Contributing is beyond me at this time, so I especially like to thank you for this important initiative.
cheers -ben
-- best, Eliot
-- Best regards, Igor Stasenko.
-- Best regards, Igor Stasenko.
Hi Igor,
On Sun, Oct 20, 2013 at 6:42 AM, Igor Stasenko siguctua@gmail.com wrote:
On 22 September 2013 20:29, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Levente,
On Sat, Sep 21, 2013 at 11:39 AM, Levente Uzonyi leves@elte.hu wrote:
Great progress. I've got a few questions about Spur and the bootstrap:
- How will HashedCollections other than MethodDictionaries get rehashed?
How #hash be calculated from #identityHash?
The bootstrap first builds an image in the new format and then finds all classes in it that implement rehash and then uses the simulator to send rehash to all objects to use their own rehash method to rehash them, instead of assuming their format. In fact, the bootstrap does not rehash all objects that understand rehash since Symbol is one of them, and its Symbol table is rehashed anyway.
hash is computed from identityHash without change, but scaledIdentityHash is changed to avoid creating LargeIntegers (the larger identityHash needs less shift also): ProtoObjectPROTOTYPEscaledIdentityHash "For identityHash values returned by primitive 75, answer such values times 2^8. Otherwise, match the existing identityHash implementation"
^self identityHash * 256 "bitShift: 8"
- What will happen to primitive 138 and 139? Will we still be able to
iterate over the objects in the order of their creation time?
That's much more difficult. I'll add allInstances and allObjects primitives which can answer the objects instantianeously without needing to enumerate. One issue is that the scavenger could fire at any time during the enumeration and could tenure objects to old space so its in general not possible to reliably enumerate via firstInstance/nextInstance firstObject/nextObject.
indeed. i think we could breathe more freely if VM wouldn't be obliged to expose heap enumeration/walking primitives.. I just wonder, if it is possible to provide alternatives for most cases:
- finding all instances
- finding all pointers to subject
so the heap walking primitives can R.I.P. .. but there could be more, which we may need and which would require heap walking. what you think?
I much prefer having allInstances and allObjects primitives. I'll be implementing them. In VisualWorks we also implemented allInstancesOfAny: to collect all instances of a set of classes, useful for the ClassBuilder, and also for odd uses like enumerating all fonts in one pass.
Being able to enumerate in order of creation time is something the VM
could maintain in old space, keeping objects in order there-in. But I'd rather drop this property and allow the Spur GC to compact using best-fit and lazy become, which will change the order of objects. How important is being able to enumerate in order of creation? I think it's nice to have but hardly essential. What do y'all think?
IMO, i would never make any assumptions, in which order objects are located on heap nor in which order they get enumerated. "Heap" word says for itself here. (and anyone who thinks otherwise should be shot on sight :)
Agreed, but once the VM was simple and so we could get away with it.
- IIUC the memory is more segmented. Will there be new GC primitives for
various levels of garbage collection? What about the GC parameters?
That remains to be seen. I take requests. I take design suggestions. But I also like to keep things simple.
My current approach is to keep things as compatible as possible to ease adoption. Once it is adopted we can start to evolve to provide appropriate and.or improved management facilities.
- Are #become: and #becomeForward: optimized for the case where both
objects use the same amount of memory?
Yes. This case simply swaps contents and adjusts the remembered table accordingly. Tim also suggested optimizing the other case, copying into the smaller object and only allocating one extra clone, which I'll implement soon.
Levente
On Sat, 21 Sep 2013, btc@openinworld.com wrote:
commits@source.squeak.org wrote:
Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/**VMMaker/VMMaker.oscog-eem.399.**mczhttp://source.squeak.org/VMMaker/VMMaker.oscog-eem.399.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.399 Author: eem Time: 20 September 2013, 6:28:56.308 pm UUID: 89f8fefe-b59d-42d7-9c11-**7f848d0e5131 Ancestors: VMMaker.oscog-eem.398
A few isIntegerObject:'s => isImmediate:'s in primitives.
More protocol.
The Spur VM now draws its first window!!
A cheer goes up in the crowd of interested spectators.
Probably lots still to do, but its a nice concrete milestone. Contributing is beyond me at this time, so I especially like to thank you for this important initiative.
cheers -ben
--
best, Eliot
-- Best regards, Igor Stasenko.
On 21 October 2013 00:48, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Igor,
On Sun, Oct 20, 2013 at 6:42 AM, Igor Stasenko siguctua@gmail.com wrote:
On 22 September 2013 20:29, Eliot Miranda eliot.miranda@gmail.comwrote:
Hi Levente,
On Sat, Sep 21, 2013 at 11:39 AM, Levente Uzonyi leves@elte.hu wrote:
Great progress. I've got a few questions about Spur and the bootstrap:
- How will HashedCollections other than MethodDictionaries get
rehashed? How #hash be calculated from #identityHash?
The bootstrap first builds an image in the new format and then finds all classes in it that implement rehash and then uses the simulator to send rehash to all objects to use their own rehash method to rehash them, instead of assuming their format. In fact, the bootstrap does not rehash all objects that understand rehash since Symbol is one of them, and its Symbol table is rehashed anyway.
hash is computed from identityHash without change, but scaledIdentityHash is changed to avoid creating LargeIntegers (the larger identityHash needs less shift also): ProtoObjectPROTOTYPEscaledIdentityHash "For identityHash values returned by primitive 75, answer such values times 2^8. Otherwise, match the existing identityHash implementation"
^self identityHash * 256 "bitShift: 8"
- What will happen to primitive 138 and 139? Will we still be able to
iterate over the objects in the order of their creation time?
That's much more difficult. I'll add allInstances and allObjects primitives which can answer the objects instantianeously without needing to enumerate. One issue is that the scavenger could fire at any time during the enumeration and could tenure objects to old space so its in general not possible to reliably enumerate via firstInstance/nextInstance firstObject/nextObject.
indeed. i think we could breathe more freely if VM wouldn't be obliged to expose heap enumeration/walking primitives.. I just wonder, if it is possible to provide alternatives for most cases:
- finding all instances
- finding all pointers to subject
so the heap walking primitives can R.I.P. .. but there could be more, which we may need and which would require heap walking. what you think?
I much prefer having allInstances and allObjects primitives. I'll be implementing them. In VisualWorks we also implemented allInstancesOfAny: to collect all instances of a set of classes, useful for the ClassBuilder, and also for odd uses like enumerating all fonts in one pass.
and what about graph traversal prim? by having it (and possibly one extra prim, which answers all roots known by VM, if we cannot get roots in some other way already), one can easily implement all of the rest:
roots := VM primAllRoots. allObjects := VM primGraphScan: roots ignoring: #().
allInstances := allObjects select: [:ea | myclasses includes: ea class].
and in constrast to heap walking, this graph traversal will not give false positives (objects which not reachable anymore, but listed because GC didn't caught them yet).
Being able to enumerate in order of creation time is something the VM
could maintain in old space, keeping objects in order there-in. But I'd rather drop this property and allow the Spur GC to compact using best-fit and lazy become, which will change the order of objects. How important is being able to enumerate in order of creation? I think it's nice to have but hardly essential. What do y'all think?
IMO, i would never make any assumptions, in which order objects are located on heap nor in which order they get enumerated. "Heap" word says for itself here. (and anyone who thinks otherwise should be shot on sight :)
Agreed, but once the VM was simple and so we could get away with it.
- IIUC the memory is more segmented. Will there be new GC primitives
for various levels of garbage collection? What about the GC parameters?
That remains to be seen. I take requests. I take design suggestions. But I also like to keep things simple.
My current approach is to keep things as compatible as possible to ease adoption. Once it is adopted we can start to evolve to provide appropriate and.or improved management facilities.
- Are #become: and #becomeForward: optimized for the case where both
objects use the same amount of memory?
Yes. This case simply swaps contents and adjusts the remembered table accordingly. Tim also suggested optimizing the other case, copying into the smaller object and only allocating one extra clone, which I'll implement soon.
Levente
On Sat, 21 Sep 2013, btc@openinworld.com wrote:
commits@source.squeak.org wrote:
Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/**VMMaker/VMMaker.oscog-eem.399.**mczhttp://source.squeak.org/VMMaker/VMMaker.oscog-eem.399.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.399 Author: eem Time: 20 September 2013, 6:28:56.308 pm UUID: 89f8fefe-b59d-42d7-9c11-**7f848d0e5131 Ancestors: VMMaker.oscog-eem.398
A few isIntegerObject:'s => isImmediate:'s in primitives.
More protocol.
The Spur VM now draws its first window!!
A cheer goes up in the crowd of interested spectators.
Probably lots still to do, but its a nice concrete milestone. Contributing is beyond me at this time, so I especially like to thank you for this important initiative.
cheers -ben
--
best, Eliot
-- Best regards, Igor Stasenko.
-- best, Eliot
Hi Igor,
On Sun, Oct 20, 2013 at 6:38 PM, Igor Stasenko siguctua@gmail.com wrote:
On 21 October 2013 00:48, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Igor,
On Sun, Oct 20, 2013 at 6:42 AM, Igor Stasenko siguctua@gmail.comwrote:
On 22 September 2013 20:29, Eliot Miranda eliot.miranda@gmail.comwrote:
Hi Levente,
On Sat, Sep 21, 2013 at 11:39 AM, Levente Uzonyi leves@elte.hu wrote:
Great progress. I've got a few questions about Spur and the bootstrap:
- How will HashedCollections other than MethodDictionaries get
rehashed? How #hash be calculated from #identityHash?
The bootstrap first builds an image in the new format and then finds all classes in it that implement rehash and then uses the simulator to send rehash to all objects to use their own rehash method to rehash them, instead of assuming their format. In fact, the bootstrap does not rehash all objects that understand rehash since Symbol is one of them, and its Symbol table is rehashed anyway.
hash is computed from identityHash without change, but scaledIdentityHash is changed to avoid creating LargeIntegers (the larger identityHash needs less shift also): ProtoObjectPROTOTYPEscaledIdentityHash "For identityHash values returned by primitive 75, answer such values times 2^8. Otherwise, match the existing identityHash implementation"
^self identityHash * 256 "bitShift: 8"
- What will happen to primitive 138 and 139? Will we still be able to
iterate over the objects in the order of their creation time?
That's much more difficult. I'll add allInstances and allObjects primitives which can answer the objects instantianeously without needing to enumerate. One issue is that the scavenger could fire at any time during the enumeration and could tenure objects to old space so its in general not possible to reliably enumerate via firstInstance/nextInstance firstObject/nextObject.
indeed. i think we could breathe more freely if VM wouldn't be obliged to expose heap enumeration/walking primitives.. I just wonder, if it is possible to provide alternatives for most cases:
- finding all instances
- finding all pointers to subject
so the heap walking primitives can R.I.P. .. but there could be more, which we may need and which would require heap walking. what you think?
I much prefer having allInstances and allObjects primitives. I'll be implementing them. In VisualWorks we also implemented allInstancesOfAny: to collect all instances of a set of classes, useful for the ClassBuilder, and also for odd uses like enumerating all fonts in one pass.
and what about graph traversal prim? by having it (and possibly one extra prim, which answers all roots known by VM, if we cannot get roots in some other way already), one can easily implement all of the rest:
roots := VM primAllRoots. allObjects := VM primGraphScan: roots ignoring: #().
allInstances := allObjects select: [:ea | myclasses includes: ea class].
and in constrast to heap walking, this graph traversal will not give false positives (objects which not reachable anymore, but listed because GC didn't caught them yet).
That's a great idea. This is guts of the segment writing primitive isn't it? And we need that. For allObjects it seems to me that simplest is just a full GC, which can count the number of objects as a side-effect, and then create a container large enough to hold that number of large objects. And that makes me think that allInstances is a good time to do a GC also. So you might combine the primitive with a GC pass.
I *think* all you need is in the repository now. You can create a bootstrap image using
(SpurBootstrap32 new on: 'pharo.image') transform; saveTransformedImage
You then can either play with that directly using
SpurBootstrap32 new launchSaved
or write it to a snapshot using
SpurBootstrap32 new writeSnapshotOfTransformedImage
and then launch via
| vm | vm := StackInterpreterSimulator newWithOptions: #(ObjectMemory Spur32BitMemoryManager). vm objectMemory setCheckForLeaks: -1. vm openOn: 'spur.image'. vm openAsMorph; run
I use an image with a read-eval-print-loop in it so I can type expressions at it for testing. But you can also do hacks like
| vm | vm := StackInterpreterSimulator newWithOptions: #(ObjectMemory Spur32BitMemoryManager). vm objectMemory setCheckForLeaks: -1. vm openOn: 'spur.image'. vm initStackPages. vm push: ...expression to push receiver... vm push: ...expression to create arg... vm primitiveMyNewPrimitive
to get to to run your primitive directly.
Being able to enumerate in order of creation time is something the VM
could maintain in old space, keeping objects in order there-in. But I'd rather drop this property and allow the Spur GC to compact using best-fit and lazy become, which will change the order of objects. How important is being able to enumerate in order of creation? I think it's nice to have but hardly essential. What do y'all think?
IMO, i would never make any assumptions, in which order objects are located on heap nor in which order they get enumerated. "Heap" word says for itself here. (and anyone who thinks otherwise should be shot on sight :)
Agreed, but once the VM was simple and so we could get away with it.
- IIUC the memory is more segmented. Will there be new GC primitives
for various levels of garbage collection? What about the GC parameters?
That remains to be seen. I take requests. I take design suggestions. But I also like to keep things simple.
My current approach is to keep things as compatible as possible to ease adoption. Once it is adopted we can start to evolve to provide appropriate and.or improved management facilities.
- Are #become: and #becomeForward: optimized for the case where both
objects use the same amount of memory?
Yes. This case simply swaps contents and adjusts the remembered table accordingly. Tim also suggested optimizing the other case, copying into the smaller object and only allocating one extra clone, which I'll implement soon.
Levente
On Sat, 21 Sep 2013, btc@openinworld.com wrote:
commits@source.squeak.org wrote:
> Eliot Miranda uploaded a new version of VMMaker to project VM Maker: > http://source.squeak.org/**VMMaker/VMMaker.oscog-eem.399.**mczhttp://source.squeak.org/VMMaker/VMMaker.oscog-eem.399.mcz > > ==================== Summary ==================== > > Name: VMMaker.oscog-eem.399 > Author: eem > Time: 20 September 2013, 6:28:56.308 pm > UUID: 89f8fefe-b59d-42d7-9c11-**7f848d0e5131 > Ancestors: VMMaker.oscog-eem.398 > > A few isIntegerObject:'s => isImmediate:'s in primitives. > > More protocol. > > The Spur VM now draws its first window!! > > > A cheer goes up in the crowd of interested spectators. Probably lots still to do, but its a nice concrete milestone. Contributing is beyond me at this time, so I especially like to thank you for this important initiative.
cheers -ben
--
best, Eliot
-- Best regards, Igor Stasenko.
-- best, Eliot
-- Best regards, Igor Stasenko.
- Are #become: and #becomeForward: optimized for the case where both
objects use the same amount of memory?
Yes. This case simply swaps contents and adjusts the remembered table accordingly. Tim also suggested optimizing the other case, copying into the smaller object and only allocating one extra clone, which I'll implement soon.
Just curious....This "simply swaps contents" isn't only for #become: ? Or there is a way to do something to avoid updating pointers for #becomeForward as well? Maybe there is something related about lazy become / forwarding pointers / proxies or whatever that could help?
Thanks,
Levente
On Sat, 21 Sep 2013, btc@openinworld.com wrote:
commits@source.squeak.org wrote:
Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/**VMMaker/VMMaker.oscog-eem.399.**mczhttp://source.squeak.org/VMMaker/VMMaker.oscog-eem.399.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.399 Author: eem Time: 20 September 2013, 6:28:56.308 pm UUID: 89f8fefe-b59d-42d7-9c11-**7f848d0e5131 Ancestors: VMMaker.oscog-eem.398
A few isIntegerObject:'s => isImmediate:'s in primitives.
More protocol.
The Spur VM now draws its first window!!
A cheer goes up in the crowd of interested spectators.
Probably lots still to do, but its a nice concrete milestone. Contributing is beyond me at this time, so I especially like to thank you for this important initiative.
cheers -ben
-- best, Eliot
Hi Mario,
On Mon, Oct 21, 2013 at 4:15 PM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
- Are #become: and #becomeForward: optimized for the case where both
objects use the same amount of memory?
Yes. This case simply swaps contents and adjusts the remembered table accordingly. Tim also suggested optimizing the other case, copying into the smaller object and only allocating one extra clone, which I'll implement soon.
Just curious....This "simply swaps contents" isn't only for #become: ? Or there is a way to do something to avoid updating pointers for #becomeForward as well?
Forwarding is the case one needs for becomeForward:. Think about the semantics; references to object A must change to be references to object B. object B must be unchanged. object A can be collected. So one either forwards object A to object B or sweeps the heap replacing references to object A with references to object B. There is no opportunity to swap contents with becomeForward:. Do you agree?
Maybe there is something related about lazy become / forwarding pointers /
proxies or whatever that could help?
Yes, forwarding means all the system has to do is change object A into a forwarder to object B. That's what Spur does. No heap sweep. It just (*) changes the class and format of object A to say "I'm a forwarder" and replaces the first slot with a reference to object B.
(*) in fact, it must then do more; ensuring that nothing along the superclass chain of any class in use is forwarded, because otherwise the VM would have to check for forwarding pointers when doing message lookup. But this is a small subset of the heap to scan, all reachable from the class table.
HTH, Eliot
Thanks,
Levente
On Sat, 21 Sep 2013, btc@openinworld.com wrote:
commits@source.squeak.org wrote:
Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/**VMMaker/VMMaker.oscog-eem.399.**mczhttp://source.squeak.org/VMMaker/VMMaker.oscog-eem.399.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.399 Author: eem Time: 20 September 2013, 6:28:56.308 pm UUID: 89f8fefe-b59d-42d7-9c11-**7f848d0e5131 Ancestors: VMMaker.oscog-eem.398
A few isIntegerObject:'s => isImmediate:'s in primitives.
More protocol.
The Spur VM now draws its first window!!
A cheer goes up in the crowd of interested spectators.
Probably lots still to do, but its a nice concrete milestone. Contributing is beyond me at this time, so I especially like to thank you for this important initiative.
cheers -ben
-- best, Eliot
-- Mariano http://marianopeck.wordpress.com
vm-dev@lists.squeakfoundation.org
-
Bert Freudenberg -
btc＠openinworld.com -
Chris Muller -
Eliot Miranda -
Igor Stasenko -
Levente Uzonyi -
Mariano Martinez Peck -
stephane ducasse