I discovered that a recent update in VMMaker-dtl.387 (November 2016) breaks the VM for older images (including my own from about 10 or 15 years ago, not good). The reason is that some obsolete numbered primitives were removed to keep the primitive table in sync with Cog/Spur as much as possible.
It is easy to revert the changes, and I plan to do that, but the issue begs for a more general solution. Given that the numbered primitive assignments change over time, and old primitives need to be retired to make room for the new, how best to provide continued support for older images with a current VM?
The primitive table is initialized at compile time, and this works fine as long as the assignments do not change too often. On the other hand, if the table was initialized at run time, i.e. when first entering interpret(), then in principle it would be possible for the VM to examine the image to deduce its vintage, or it would be possible to explicitly request some version of the primitive table by means of a command line option to the VM.
I played around wth this a bit in VMMaker, and I think that defining entire versions of the primitive table at runtime would require large methods that would probably to hard to understand and maintain. However, if we adopt a policy of using the latest primitive assignments as they appear in Cog/Spur for the default table at compile time, and then overriding those assignments as needed at run time, then it should be possible to maintain backward compatibility for old images without making too much of a mess out of the primitive table initialization.
Thoughts?
Dave
On Thu, Dec 22, 2016 at 7:05 AM, David T. Lewis lewis@mail.msen.com wrote:
I discovered that a recent update in VMMaker-dtl.387 (November 2016) breaks the VM for older images (including my own from about 10 or 15 years ago, not good). The reason is that some obsolete numbered primitives were removed to keep the primitive table in sync with Cog/Spur as much as possible.
It is easy to revert the changes, and I plan to do that, but the issue begs for a more general solution. Given that the numbered primitive assignments change over time, and old primitives need to be retired to make room for the new, how best to provide continued support for older images with a current VM?
The primitive table is initialized at compile time, and this works fine as long as the assignments do not change too often. On the other hand, if the table was initialized at run time, i.e. when first entering interpret(), then in principle it would be possible for the VM to examine the image to deduce its vintage, or it would be possible to explicitly request some version of the primitive table by means of a command line option to the VM.
I played around with this a bit in VMMaker, and I think that defining entire versions of the primitive table at runtime would require large methods that would probably to hard to understand and maintain. However, if we adopt a policy of using the latest primitive assignments as they appear in Cog/Spur for the default table at compile time, and then overriding those assignments as needed at run time, then it should be possible to maintain backward compatibility for old images without making too much of a mess out of the primitive table initialization.
Thoughts?
How far back in time do images maintain the ImageFormat in its header? I presume you would just work off that.
Could this backward compatibility be organised as a plugin? Or is the initialization too far advanced when plugins are loaded? It would be good if this code is easily separable for possibly producing a minimal VM for embedded devices. cheers -ben
On Thu, Dec 22, 2016 at 09:48:09AM +0800, Ben Coman wrote:
On Thu, Dec 22, 2016 at 7:05 AM, David T. Lewis lewis@mail.msen.com wrote:
I discovered that a recent update in VMMaker-dtl.387 (November 2016) breaks the VM for older images (including my own from about 10 or 15 years ago, not good). The reason is that some obsolete numbered primitives were removed to keep the primitive table in sync with Cog/Spur as much as possible.
It is easy to revert the changes, and I plan to do that, but the issue begs for a more general solution. Given that the numbered primitive assignments change over time, and old primitives need to be retired to make room for the new, how best to provide continued support for older images with a current VM?
The primitive table is initialized at compile time, and this works fine as long as the assignments do not change too often. On the other hand, if the table was initialized at run time, i.e. when first entering interpret(), then in principle it would be possible for the VM to examine the image to deduce its vintage, or it would be possible to explicitly request some version of the primitive table by means of a command line option to the VM.
I played around with this a bit in VMMaker, and I think that defining entire versions of the primitive table at runtime would require large methods that would probably to hard to understand and maintain. However, if we adopt a policy of using the latest primitive assignments as they appear in Cog/Spur for the default table at compile time, and then overriding those assignments as needed at run time, then it should be possible to maintain backward compatibility for old images without making too much of a mess out of the primitive table initialization.
Thoughts?
How far back in time do images maintain the ImageFormat in its header? I presume you would just work off that.
The image format numbers go back to the very first Squeak images, although I don't think they are going to provide enough granularity. They might get us close though, or at least be enough to provide some hints. So for example if the image format was pre-closure, we can be fairly sure it needs access to some of the older numbered primitives.
Could this backward compatibility be organised as a plugin? Or is the initialization too far advanced when plugins are loaded? It would be good if this code is easily separable for possibly producing a minimal VM for embedded devices. cheers -ben
The initialization needs to be in place before any primitives are called, so it needs to happen before or at the same time as you first enter the interpret loop. So I expect that any initialization would need to be either passed to the VM (via command line option or some configuration), or deduced from the image format as you suggest above.
Dave
On Thu, Dec 22, 2016 at 3:50 AM, David T. Lewis lewis@mail.msen.com wrote:
Given that the numbered primitive assignments
change over time, and old primitives need to be retired to make room
for
the new, how best to provide continued support for older images with a current VM?
In SqueakJS I have a single flag oldPrims to switch between two set of primitives. This appears to work fine all the way back to Squeak 1.x. The flag is set based on whether the image format has closures or not.
Just search for "oldPrims" in https://github.com/bertfreudenberg/SqueakJS/blob/master/vm.js
- Bert -
Thanks Bert, that definitely will help.
Dave
On Thu, Dec 22, 2016 at 3:50 AM, David T. Lewis lewis@mail.msen.com wrote:
Given that the numbered primitive assignments
change over time, and old primitives need to be retired to make room
for
the new, how best to provide continued support for older images with
a
current VM?
In SqueakJS I have a single flag oldPrims to switch between two set of primitives. This appears to work fine all the way back to Squeak 1.x. The flag is set based on whether the image format has closures or not.
Just search for "oldPrims" in https://github.com/bertfreudenberg/SqueakJS/blob/master/vm.js
- Bert -
Bert Freudenberg wrote on Thu, 22 Dec 2016 16:40:51 +0100
In SqueakJS I have a single flag oldPrims to switch between two set of primitives. This appears to work fine all the way back to Squeak 1.x. The flag is set based on whether the image format has closures or not.
I had the impression, based on some difficulties on getting the simulator to work (around 2008, I think), that at one point an instance variable in a key class (Process, most likely) stopped being used and then a few versions later it got reused in a different way. This also happened to a numbered primitive or two. And the version number stayed 6502 though all these changes. So a newer VM would appear to run an older image just fine but once in a while you got a really strange crash.
My hope is that this was just some horrible nightmare of mine and not actual memories, but I will have to take some time searching through old email messages to be sure.
-- Jecel
On Thu, Dec 22, 2016 at 09:10:05PM -0300, Jecel Assumpcao Jr. wrote:
Bert Freudenberg wrote on Thu, 22 Dec 2016 16:40:51 +0100
In SqueakJS I have a single flag??oldPrims??to switch between two set of primitives. This appears to work fine all the way back to Squeak 1.x. The flag is set based on whether the image format has closures or not.
I had the impression, based on some difficulties on getting the simulator to work (around 2008, I think), that at one point an instance variable in a key class (Process, most likely) stopped being used and then a few versions later it got reused in a different way. This also happened to a numbered primitive or two. And the version number stayed 6502 though all these changes. So a newer VM would appear to run an older image just fine but once in a while you got a really strange crash.
My hope is that this was just some horrible nightmare of mine and not actual memories, but I will have to take some time searching through old email messages to be sure.
The other thing that I expect may be a source of trouble is the special objects array. But I have not actually checked to see if it's a problem yet.
It is quite encouraging that SqueakJS is able to run all the way back to Squeak 1.x with just two variations of the primitive table.
Dave
On Thu, Dec 22, 2016 at 04:40:51PM +0100, Bert Freudenberg wrote:
On Thu, Dec 22, 2016 at 3:50 AM, David T. Lewis lewis@mail.msen.com wrote:
Given that the numbered primitive assignments
change over time, and old primitives need to be retired to make room
for
the new, how best to provide continued support for older images with a current VM?
In SqueakJS I have a single flag oldPrims to switch between two set of primitives. This appears to work fine all the way back to Squeak 1.x. The flag is set based on whether the image format has closures or not.
Just search for "oldPrims" in https://github.com/bertfreudenberg/SqueakJS/blob/master/vm.js
I did some further digging into the various versions of the primitive table to be found on files.squeak.org and the VMMaker repository. I'm not sure if this is of any general interest, but the attached change set contains everything I could find going back to Squeak 1.1 from 1996.
Preamble:
"Change Set: PrimitiveTableHistory-dtl Date: 26 December 2016 Author: David T Lewis
PrimitiveTableHistory shows history and differences between versions of the primitive table.
Browse all recorded changes: PrimitiveTableHistory enumerate
Changes from last pre-closure version to present: PrimitiveTableHistory differenceFrom: self lastPreClosuresVersion to: PrimitiveTableHistory current
Early versions through Squeak 3.0 are extracted from changes and sources files found on files.squeak.org. After Squeak 3.0 the VM was maintained externally. Beginning in 2004, the VM was maintained in Monticello. Versions between Feb 2001 and Feb 2004 are not recorded here. "
Dave
vm-dev@lists.squeakfoundation.org
-
Ben Coman -
Bert Freudenberg -
David T. Lewis -
Jecel Assumpcao Jr.