Fwd from John Maloney ...
- Bert -
Begin forwarded message:
From: John Maloney jmaloney@media.mit.edu Date: 16. Januar 2010 16:21:59 MEZ To: Bert Freudenberg bert@freudenbergs.de Cc: Ian Piumarta ian@vpri.org Subject: Re: [Vm-dev] Re: [squeak-dev] Numbered primitives in images
Hi, Bert.
Thanks for the alert.
Yep, this would be a problem for Scratch. It would prevent sharing projects on the website. Normally, I would not mind sticking with an older VM, but that there is also a change in the sound support on Linux from ALSA to Pulse Audio that we need, and work on that is still ongoing. So we may need to use a current VM, at least on Linux.
Any chance there are some other numbered primitives that could be co-opted for the closure prims? As I recall there were lots of unused numbered primitives.
-- John
On Jan 16, 2010, at 9:57 AM, Bert Freudenberg wrote:
That may be a problem ...
- Bert -
Begin forwarded message:
From: "David T. Lewis" lewis@mail.msen.com Date: 16. Januar 2010 15:55:02 MEZ To: "K. K. Subramaniam" subbukk@gmail.com Cc: vm-dev@lists.squeakfoundation.org Subject: [Vm-dev] Re: [squeak-dev] Numbered primitives in images Reply-To: Squeak Virtual Machine Development Discussion vm-dev@lists.squeakfoundation.org
On Sat, Jan 16, 2010 at 08:16:06AM +0530, K. K. Subramaniam wrote:
On Friday 15 January 2010 10:46:45 pm Bert Freudenberg wrote:
On 15.01.2010, at 16:58, K. K. Subramaniam wrote:
Scratch 1.4 (Linux) of 30-Nov-2009 on 3.11-3 VM. Break into Scratch and do "Socket initializeNetwork".
So it does not occur in normal operation?
Only if you define "normal" as "offline" ;-). Any operation which uses networking (url block) goes through this method and it will fail when run under the 3.11.3 VM built from unix-3.11.3-vmm or svn trunk.
BTW, Scratch image is just an example which has numbered primitives.
The 3.10-4 VM has primitiveObsoleteIndexedPrimitive in slot 207 in platforms/unix/src/vm/interp.c while the 3.11.3 tar.gz and the trunk contain primitiveFail. The change crept in r1935 through vmm-dtl-126.
Is this intentional or an oversight?
Sorry I was not able to reply earlier. The change was intentional, but the effect on Scratch was not.
Some of the obsolete primitives had to be dropped when we added the block closure support. This happened in February 2009 (VMMaker-eem.114 in the SqS/VMMaker). Specifically, we changed from this, which redirects the numbered primitives:
"Networking Primitives (200-225) - NO LONGER INDEXED" (200 225 primitiveObsoleteIndexedPrimitive)to this, which reallocates the numbers for closure support and for future COG integration:
"new closure primitives (were Networking primitives)" (200 primitiveClosureCopyWithCopiedValues) (201 primitiveClosureValue) "value" (202 primitiveClosureValue) "value:" (203 primitiveClosureValue) "value:value:" (204 primitiveClosureValue) "value:value:value:" (205 primitiveClosureValue) "value:value:value:value:" (206 primitiveClosureValueWithArgs) "valueWithArguments:" (207 209 primitiveFail) "reserved for Cog primitives"The obsolete primitive redirection had been in place for at least five years, so backward compatibility had been preserved for some time.
Question for Subbu: Is this an important issue may affect a large number of Scratch users? In other words, are there kids trying to run Scratch, and their Scratch image stops working properly after Squeak gets updated to use a new VM?
And a thought for the vm-dev list: I don't know if there might be a technical solution to this, but it might be possible to use a strategy in which the some of the old primitive lookups are preserved (at least for the range 207 through 225) if and only if the image format is pre-closures. Follow ups to this on vm-only.
Dave
The issue is now on Mantis:
http://bugs.squeak.org/view.php?id=7454
Moving the closure primitives would be difficult at this point, as we have a large installed base of closure-enabled images by now.
In principle, it should be possible to update the primitiveTable at run time based on imageFormatVersion, or perhaps as an option to be supplied to the VM at startup time, or a combination of the two.
Doing it automatically based on imageFormatVersion would probably work, but has a catch-22 for someone adding closures to an older image (e.g. adding the closure support to Etoys). For that situation you want to be able to turn on closure support even though the image does not yet need them, so perhaps e.g. a "-closures" command line option turn on closure support in that case.
Any other ideas?
Dave
On Sat, Jan 16, 2010 at 06:56:47PM +0100, Bert Freudenberg wrote:
Fwd from John Maloney ...
- Bert -
Begin forwarded message:
From: John Maloney jmaloney@media.mit.edu Date: 16. Januar 2010 16:21:59 MEZ To: Bert Freudenberg bert@freudenbergs.de Cc: Ian Piumarta ian@vpri.org Subject: Re: [Vm-dev] Re: [squeak-dev] Numbered primitives in images
Hi, Bert.
Thanks for the alert.
Yep, this would be a problem for Scratch. It would prevent sharing projects on the website. Normally, I would not mind sticking with an older VM, but that there is also a change in the sound support on Linux from ALSA to Pulse Audio that we need, and work on that is still ongoing. So we may need to use a current VM, at least on Linux.
Any chance there are some other numbered primitives that could be co-opted for the closure prims? As I recall there were lots of unused numbered primitives.
-- John
On Jan 16, 2010, at 9:57 AM, Bert Freudenberg wrote:
That may be a problem ...
- Bert -
Begin forwarded message:
From: "David T. Lewis" lewis@mail.msen.com Date: 16. Januar 2010 15:55:02 MEZ To: "K. K. Subramaniam" subbukk@gmail.com Cc: vm-dev@lists.squeakfoundation.org Subject: [Vm-dev] Re: [squeak-dev] Numbered primitives in images Reply-To: Squeak Virtual Machine Development Discussion vm-dev@lists.squeakfoundation.org
On Sat, Jan 16, 2010 at 08:16:06AM +0530, K. K. Subramaniam wrote:
On Friday 15 January 2010 10:46:45 pm Bert Freudenberg wrote:
On 15.01.2010, at 16:58, K. K. Subramaniam wrote: > Scratch 1.4 (Linux) of 30-Nov-2009 on 3.11-3 VM. > Break into Scratch and do "Socket initializeNetwork".
So it does not occur in normal operation?
Only if you define "normal" as "offline" ;-). Any operation which uses networking (url block) goes through this method and it will fail when run under the 3.11.3 VM built from unix-3.11.3-vmm or svn trunk.
BTW, Scratch image is just an example which has numbered primitives.
The 3.10-4 VM has primitiveObsoleteIndexedPrimitive in slot 207 in platforms/unix/src/vm/interp.c while the 3.11.3 tar.gz and the trunk contain primitiveFail. The change crept in r1935 through vmm-dtl-126.
Is this intentional or an oversight?
Sorry I was not able to reply earlier. The change was intentional, but the effect on Scratch was not.
Some of the obsolete primitives had to be dropped when we added the block closure support. This happened in February 2009 (VMMaker-eem.114 in the SqS/VMMaker). Specifically, we changed from this, which redirects the numbered primitives:
"Networking Primitives (200-225) - NO LONGER INDEXED" (200 225 primitiveObsoleteIndexedPrimitive)to this, which reallocates the numbers for closure support and for future COG integration:
"new closure primitives (were Networking primitives)" (200 primitiveClosureCopyWithCopiedValues) (201 primitiveClosureValue) "value" (202 primitiveClosureValue) "value:" (203 primitiveClosureValue) "value:value:" (204 primitiveClosureValue) "value:value:value:" (205 primitiveClosureValue) "value:value:value:value:" (206 primitiveClosureValueWithArgs) "valueWithArguments:" (207 209 primitiveFail) "reserved for Cog primitives"The obsolete primitive redirection had been in place for at least five years, so backward compatibility had been preserved for some time.
Question for Subbu: Is this an important issue may affect a large number of Scratch users? In other words, are there kids trying to run Scratch, and their Scratch image stops working properly after Squeak gets updated to use a new VM?
And a thought for the vm-dev list: I don't know if there might be a technical solution to this, but it might be possible to use a strategy in which the some of the old primitive lookups are preserved (at least for the range 207 through 225) if and only if the image format is pre-closures. Follow ups to this on vm-only.
Dave
John, this may be of some concern if your new VMs are making use of the #extraVmMemory setting.
http://bugs.squeak.org/view.php?id=7455
A 64 bit image (wordSize 8) is currently using a 128 byte image file header, with all the integer values in the header stored as 8 bit ints. The VM is writing the image data at position 64 rather than 128, resulting the value of extraVmMemory being overwritten with garbage. ImageTracer64 produces the correct 128 byte offset, but this is undone by the VM after the first image save.
This is producing no obvious symptoms when running on a unix VM, but is certainly not a good thing.
Short term fix should be to make the VM use headerSize 128 rather than 64 when writing a 64 bit image. IMO the correct thing to do long term is to always write header data as 32 bit int regardless of image wordSize, and use 64 bit file headers in both cases. This however will require some way to retain backward compatibility for reading 64 bit images that have the current file header structure.
Dave
extraVmMemory was an option for macintosh os 9 and earlier machines. Say September 27, 1996 era machines. In checking the flow of this variable it's actually ignored by macintosh VM since the early part of the last decade.
I'd suggest the slot is marked as reserved/obsolete, and the extraVmMemory removed. I note the extraVmMemory is a vm parm value so we can't drop it directly without effecting index, but it can be marked as obsolete.
On 2010-01-16, at 12:30 PM, David T. Lewis wrote:
extraVmMemory
-- =========================================================================== John M. McIntosh johnmci@smalltalkconsulting.com Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
On Sat, Jan 16, 2010 at 11:20 AM, David T. Lewis lewis@mail.msen.comwrote:
The issue is now on Mantis:
http://bugs.squeak.org/view.php?id=7454
Moving the closure primitives would be difficult at this point, as we have a large installed base of closure-enabled images by now.
In principle, it should be possible to update the primitiveTable at run time based on imageFormatVersion, or perhaps as an option to be supplied to the VM at startup time, or a combination of the two.
Doing it automatically based on imageFormatVersion would probably work, but has a catch-22 for someone adding closures to an older image (e.g. adding the closure support to Etoys). For that situation you want to be able to turn on closure support even though the image does not yet need them, so perhaps e.g. a "-closures" command line option turn on closure support in that case.
Any other ideas?
Would it be possible to write a script to replace the numbered primitives in the Scratch image with named prims and run that script on a suitable pre-closure VM? Once the image is saved it should run on the closure-enabled VMs.
Surely this is better than rewriting the primitive table. I would think we would want to move away from obsolete interfaces and if Scratch supports named prims then it works, is compatible with other Squeaks and avoids hacks.
Dave
On Sat, Jan 16, 2010 at 06:56:47PM +0100, Bert Freudenberg wrote:
Fwd from John Maloney ...
- Bert -
Begin forwarded message:
From: John Maloney jmaloney@media.mit.edu Date: 16. Januar 2010 16:21:59 MEZ To: Bert Freudenberg bert@freudenbergs.de Cc: Ian Piumarta ian@vpri.org Subject: Re: [Vm-dev] Re: [squeak-dev] Numbered primitives in images
Hi, Bert.
Thanks for the alert.
Yep, this would be a problem for Scratch. It would prevent sharing
projects on the website. Normally, I would not mind sticking with an older VM, but that there is also a change in the sound support on Linux from ALSA to Pulse Audio that we need, and work on that is still ongoing. So we may need to use a current VM, at least on Linux.
Any chance there are some other numbered primitives that could be
co-opted for the closure prims? As I recall there were lots of unused numbered primitives.
-- JohnOn Jan 16, 2010, at 9:57 AM, Bert Freudenberg wrote:
That may be a problem ...
- Bert -
Begin forwarded message:
From: "David T. Lewis" lewis@mail.msen.com Date: 16. Januar 2010 15:55:02 MEZ To: "K. K. Subramaniam" subbukk@gmail.com Cc: vm-dev@lists.squeakfoundation.org Subject: [Vm-dev] Re: [squeak-dev] Numbered primitives in images Reply-To: Squeak Virtual Machine Development Discussion <
vm-dev@lists.squeakfoundation.org>
On Sat, Jan 16, 2010 at 08:16:06AM +0530, K. K. Subramaniam wrote:
On Friday 15 January 2010 10:46:45 pm Bert Freudenberg wrote: > On 15.01.2010, at 16:58, K. K. Subramaniam wrote: >> Scratch 1.4 (Linux) of 30-Nov-2009 on 3.11-3 VM. >> Break into Scratch and do "Socket initializeNetwork". > > So it does not occur in normal operation? Only if you define "normal" as "offline" ;-). Any operation which
uses networking
(url block) goes through this method and it will fail when run under
the
3.11.3 VM built from unix-3.11.3-vmm or svn trunk.
BTW, Scratch image is just an example which has numbered primitives.
The 3.10-4 VM has primitiveObsoleteIndexedPrimitive in slot 207 in platforms/unix/src/vm/interp.c while the 3.11.3 tar.gz and the trunk
contain
primitiveFail. The change crept in r1935 through vmm-dtl-126.
Is this intentional or an oversight?
Sorry I was not able to reply earlier. The change was intentional, but the effect on Scratch was not.
Some of the obsolete primitives had to be dropped when we added the block closure support. This happened in February 2009
(VMMaker-eem.114
in the SqS/VMMaker). Specifically, we changed from this, which
redirects
the numbered primitives:
"Networking Primitives (200-225) - NO LONGER INDEXED" (200 225 primitiveObsoleteIndexedPrimitive)to this, which reallocates the numbers for closure support and for future COG integration:
"new closure primitives (were Networking primitives)" (200 primitiveClosureCopyWithCopiedValues) (201 primitiveClosureValue) "value" (202 primitiveClosureValue) "value:" (203 primitiveClosureValue) "value:value:" (204 primitiveClosureValue) "value:value:value:" (205 primitiveClosureValue) "value:value:value:value:" (206 primitiveClosureValueWithArgs) "valueWithArguments:" (207 209 primitiveFail) "reserved for Cog primitives"The obsolete primitive redirection had been in place for at least
five
years, so backward compatibility had been preserved for some time.
Question for Subbu: Is this an important issue may affect a large
number
of Scratch users? In other words, are there kids trying to run
Scratch,
and their Scratch image stops working properly after Squeak gets
updated
to use a new VM?
And a thought for the vm-dev list: I don't know if there might be a technical solution to this, but it might be possible to use a
strategy
in which the some of the old primitive lookups are preserved (at
least
for the range 207 through 225) if and only if the image format is pre-closures. Follow ups to this on vm-only.
Dave
On 17.01.2010, at 02:23, Eliot Miranda wrote:
On Sat, Jan 16, 2010 at 11:20 AM, David T. Lewis lewis@mail.msen.com wrote:
The issue is now on Mantis:
http://bugs.squeak.org/view.php?id=7454
Moving the closure primitives would be difficult at this point, as we have a large installed base of closure-enabled images by now.
In principle, it should be possible to update the primitiveTable at run time based on imageFormatVersion, or perhaps as an option to be supplied to the VM at startup time, or a combination of the two.
Doing it automatically based on imageFormatVersion would probably work, but has a catch-22 for someone adding closures to an older image (e.g. adding the closure support to Etoys). For that situation you want to be able to turn on closure support even though the image does not yet need them, so perhaps e.g. a "-closures" command line option turn on closure support in that case.
Any other ideas?
Would it be possible to write a script to replace the numbered primitives in the Scratch image with named prims and run that script on a suitable pre-closure VM? Once the image is saved it should run on the closure-enabled VMs.
Surely this is better than rewriting the primitive table. I would think we would want to move away from obsolete interfaces and if Scratch supports named prims then it works, is compatible with other Squeaks and avoids hacks.
The Scratch image is from around Squeak 2.5 - you might take a look here: http://download.scratch.mit.edu/source-code/ScratchSource1.4.zip
- Bert -
Dave
On Sat, Jan 16, 2010 at 06:56:47PM +0100, Bert Freudenberg wrote:
Fwd from John Maloney ...
- Bert -
Begin forwarded message:
From: John Maloney jmaloney@media.mit.edu Date: 16. Januar 2010 16:21:59 MEZ To: Bert Freudenberg bert@freudenbergs.de Cc: Ian Piumarta ian@vpri.org Subject: Re: [Vm-dev] Re: [squeak-dev] Numbered primitives in images
Hi, Bert.
Thanks for the alert.
Yep, this would be a problem for Scratch. It would prevent sharing projects on the website. Normally, I would not mind sticking with an older VM, but that there is also a change in the sound support on Linux from ALSA to Pulse Audio that we need, and work on that is still ongoing. So we may need to use a current VM, at least on Linux.
Any chance there are some other numbered primitives that could be co-opted for the closure prims? As I recall there were lots of unused numbered primitives.
-- JohnOn Jan 16, 2010, at 9:57 AM, Bert Freudenberg wrote:
That may be a problem ...
- Bert -
Begin forwarded message:
From: "David T. Lewis" lewis@mail.msen.com Date: 16. Januar 2010 15:55:02 MEZ To: "K. K. Subramaniam" subbukk@gmail.com Cc: vm-dev@lists.squeakfoundation.org Subject: [Vm-dev] Re: [squeak-dev] Numbered primitives in images Reply-To: Squeak Virtual Machine Development Discussion vm-dev@lists.squeakfoundation.org
On Sat, Jan 16, 2010 at 08:16:06AM +0530, K. K. Subramaniam wrote:
On Friday 15 January 2010 10:46:45 pm Bert Freudenberg wrote: > On 15.01.2010, at 16:58, K. K. Subramaniam wrote: >> Scratch 1.4 (Linux) of 30-Nov-2009 on 3.11-3 VM. >> Break into Scratch and do "Socket initializeNetwork". > > So it does not occur in normal operation? Only if you define "normal" as "offline" ;-). Any operation which uses networking (url block) goes through this method and it will fail when run under the 3.11.3 VM built from unix-3.11.3-vmm or svn trunk.
BTW, Scratch image is just an example which has numbered primitives.
The 3.10-4 VM has primitiveObsoleteIndexedPrimitive in slot 207 in platforms/unix/src/vm/interp.c while the 3.11.3 tar.gz and the trunk contain primitiveFail. The change crept in r1935 through vmm-dtl-126.
Is this intentional or an oversight?
Sorry I was not able to reply earlier. The change was intentional, but the effect on Scratch was not.
Some of the obsolete primitives had to be dropped when we added the block closure support. This happened in February 2009 (VMMaker-eem.114 in the SqS/VMMaker). Specifically, we changed from this, which redirects the numbered primitives:
"Networking Primitives (200-225) - NO LONGER INDEXED" (200 225 primitiveObsoleteIndexedPrimitive)to this, which reallocates the numbers for closure support and for future COG integration:
"new closure primitives (were Networking primitives)" (200 primitiveClosureCopyWithCopiedValues) (201 primitiveClosureValue) "value" (202 primitiveClosureValue) "value:" (203 primitiveClosureValue) "value:value:" (204 primitiveClosureValue) "value:value:value:" (205 primitiveClosureValue) "value:value:value:value:" (206 primitiveClosureValueWithArgs) "valueWithArguments:" (207 209 primitiveFail) "reserved for Cog primitives"The obsolete primitive redirection had been in place for at least five years, so backward compatibility had been preserved for some time.
Question for Subbu: Is this an important issue may affect a large number of Scratch users? In other words, are there kids trying to run Scratch, and their Scratch image stops working properly after Squeak gets updated to use a new VM?
And a thought for the vm-dev list: I don't know if there might be a technical solution to this, but it might be possible to use a strategy in which the some of the old primitive lookups are preserved (at least for the range 207 through 225) if and only if the image format is pre-closures. Follow ups to this on vm-only.
Dave
Elliot's idea -- just fix the Scratch image to use the named socket prims -- sounds like a winner. It's simple, it doesn't add any complexity to the VM's, and it doesn't break anything. And it puts the burden on Scratch, where it belongs.
Assuming the named socket primitives do exactly the same things the numbered ones did (and it sounds like they do, since the numbered prims have been re-directed to the named ones for about a decade :-)) then nothing should break, right?
Note that Scratch comes bundled with a copy of the Squeak VM, and the bundled VM in the current release still has support for the numbered socket primitives. So this VM change only matters for future releases of Scratch (and the Linux package, which is in active development).
Thanks for thinking about this issue.
-- John
On Jan 17, 2010, at 7:07 AM, Bert Freudenberg wrote:
On 17.01.2010, at 02:23, Eliot Miranda wrote:
On Sat, Jan 16, 2010 at 11:20 AM, David T. Lewis lewis@mail.msen.com wrote:
The issue is now on Mantis:
http://bugs.squeak.org/view.php?id=7454
Moving the closure primitives would be difficult at this point, as we have a large installed base of closure-enabled images by now.
In principle, it should be possible to update the primitiveTable at run time based on imageFormatVersion, or perhaps as an option to be supplied to the VM at startup time, or a combination of the two.
Doing it automatically based on imageFormatVersion would probably work, but has a catch-22 for someone adding closures to an older image (e.g. adding the closure support to Etoys). For that situation you want to be able to turn on closure support even though the image does not yet need them, so perhaps e.g. a "-closures" command line option turn on closure support in that case.
Any other ideas?
Would it be possible to write a script to replace the numbered primitives in the Scratch image with named prims and run that script on a suitable pre-closure VM? Once the image is saved it should run on the closure-enabled VMs.
Surely this is better than rewriting the primitive table. I would think we would want to move away from obsolete interfaces and if Scratch supports named prims then it works, is compatible with other Squeaks and avoids hacks.
The Scratch image is from around Squeak 2.5 - you might take a look here: http://download.scratch.mit.edu/source-code/ScratchSource1.4.zip
- Bert -
Dave
On Sat, Jan 16, 2010 at 06:56:47PM +0100, Bert Freudenberg wrote:
Fwd from John Maloney ...
- Bert -
Begin forwarded message:
From: John Maloney jmaloney@media.mit.edu Date: 16. Januar 2010 16:21:59 MEZ To: Bert Freudenberg bert@freudenbergs.de Cc: Ian Piumarta ian@vpri.org Subject: Re: [Vm-dev] Re: [squeak-dev] Numbered primitives in
images
Hi, Bert.
Thanks for the alert.
Yep, this would be a problem for Scratch. It would prevent
sharing projects on the website. Normally, I would not mind sticking with an older VM, but that there is also a change in the sound support on Linux from ALSA to Pulse Audio that we need, and work on that is still ongoing. So we may need to use a current VM, at least on Linux.
Any chance there are some other numbered primitives that could
be co-opted for the closure prims? As I recall there were lots of unused numbered primitives.
-- JohnOn Jan 16, 2010, at 9:57 AM, Bert Freudenberg wrote:
That may be a problem ...
- Bert -
Begin forwarded message:
From: "David T. Lewis" lewis@mail.msen.com Date: 16. Januar 2010 15:55:02 MEZ To: "K. K. Subramaniam" subbukk@gmail.com Cc: vm-dev@lists.squeakfoundation.org Subject: [Vm-dev] Re: [squeak-dev] Numbered primitives in
images
Reply-To: Squeak Virtual Machine Development Discussion <vm-dev@lists.squeakfoundation.org
On Sat, Jan 16, 2010 at 08:16:06AM +0530, K. K. Subramaniam
wrote:
> On Friday 15 January 2010 10:46:45 pm Bert Freudenberg wrote: >> On 15.01.2010, at 16:58, K. K. Subramaniam wrote: >>> Scratch 1.4 (Linux) of 30-Nov-2009 on 3.11-3 VM. >>> Break into Scratch and do "Socket initializeNetwork". >> >> So it does not occur in normal operation? > Only if you define "normal" as "offline" ;-). Any operation
which uses networking
> (url block) goes through this method and it will fail when
run under the
> 3.11.3 VM built from unix-3.11.3-vmm or svn trunk. > > BTW, Scratch image is just an example which has numbered
primitives.
> > The 3.10-4 VM has primitiveObsoleteIndexedPrimitive in slot
207 in
> platforms/unix/src/vm/interp.c while the 3.11.3 tar.gz and
the trunk contain
> primitiveFail. The change crept in r1935 through vmm-dtl-126. > > Is this intentional or an oversight?
Sorry I was not able to reply earlier. The change was
intentional,
but the effect on Scratch was not.
Some of the obsolete primitives had to be dropped when we
added the
block closure support. This happened in February 2009
(VMMaker-eem.114
in the SqS/VMMaker). Specifically, we changed from this,
which redirects
the numbered primitives:
"Networking Primitives (200-225) - NO LONGER INDEXED" (200 225 primitiveObsoleteIndexedPrimitive)to this, which reallocates the numbers for closure support
and for
future COG integration:
"new closure primitives (were Networking primitives)" (200 primitiveClosureCopyWithCopiedValues) (201 primitiveClosureValue) "value" (202 primitiveClosureValue) "value:" (203 primitiveClosureValue) "value:value:" (204 primitiveClosureValue) "value:value:value:" (205 primitiveClosureValue)"value:value:value:value:"
(206 primitiveClosureValueWithArgs)"valueWithArguments:"
(207 209 primitiveFail) "reserved for Cog primitives"The obsolete primitive redirection had been in place for at
least five
years, so backward compatibility had been preserved for some
time.
Question for Subbu: Is this an important issue may affect a
large number
of Scratch users? In other words, are there kids trying to
run Scratch,
and their Scratch image stops working properly after Squeak
gets updated
to use a new VM?
And a thought for the vm-dev list: I don't know if there
might be a
technical solution to this, but it might be possible to use a
strategy
in which the some of the old primitive lookups are preserved
(at least
for the range 207 through 225) if and only if the image
format is
pre-closures. Follow ups to this on vm-only.
Dave
On Sunday 17 January 2010 12:50:17 am David T. Lewis wrote:
Doing it automatically based on imageFormatVersion would probably work, but has a catch-22 for someone adding closures to an older image (e.g. adding the closure support to Etoys). For that situation you want to be able to turn on closure support even though the image does not yet need them, so perhaps e.g. a "-closures" command line option turn on closure support in that case.
Any other ideas?
An image may run on a VM on multiple platforms and not all may be at the same rev, breaking backward compatibility between 3.10 and 3.11 may cause more grief than necessary.
How about patching the VM to check for env variable (say SQUEAK_PRIM311) in addition to checking for -closures?
Subbu
On Sun, Jan 17, 2010 at 08:43:37PM +0530, K. K. Subramaniam wrote:
On Sunday 17 January 2010 12:50:17 am David T. Lewis wrote:
Doing it automatically based on imageFormatVersion would probably work, but has a catch-22 for someone adding closures to an older image (e.g. adding the closure support to Etoys). For that situation you want to be able to turn on closure support even though the image does not yet need them, so perhaps e.g. a "-closures" command line option turn on closure support in that case.
Any other ideas?
An image may run on a VM on multiple platforms and not all may be at the same rev, breaking backward compatibility between 3.10 and 3.11 may cause more grief than necessary.
How about patching the VM to check for env variable (say SQUEAK_PRIM311) in addition to checking for -closures?
I guess we're still looking for ideas at this point. As Eliot points out, rewriting the primitive table (which is probably what we would do based on s SQUEAK_PRIM311 flag) is a messy hack, and it would be best not to put that sort of thing into the VM.
Perhaps a BackwardCompatibilityHacksPlugin to keep the messy stuff out of the main VM would be possible. To extend the challenge a bit further, it would be interesting to see how far back in time we could go with a single VM. The ftp.squeak.org site has images going all the way back to Squeak 1.1.
Dave
On 17.01.2010, at 16:39, David T. Lewis wrote:
On Sun, Jan 17, 2010 at 08:43:37PM +0530, K. K. Subramaniam wrote:
On Sunday 17 January 2010 12:50:17 am David T. Lewis wrote:
Doing it automatically based on imageFormatVersion would probably work, but has a catch-22 for someone adding closures to an older image (e.g. adding the closure support to Etoys). For that situation you want to be able to turn on closure support even though the image does not yet need them, so perhaps e.g. a "-closures" command line option turn on closure support in that case.
Any other ideas?
An image may run on a VM on multiple platforms and not all may be at the same rev, breaking backward compatibility between 3.10 and 3.11 may cause more grief than necessary.
How about patching the VM to check for env variable (say SQUEAK_PRIM311) in addition to checking for -closures?
I guess we're still looking for ideas at this point. As Eliot points out, rewriting the primitive table (which is probably what we would do based on s SQUEAK_PRIM311 flag) is a messy hack, and it would be best not to put that sort of thing into the VM.
Perhaps a BackwardCompatibilityHacksPlugin to keep the messy stuff out of the main VM would be possible. To extend the challenge a bit further, it would be interesting to see how far back in time we could go with a single VM. The ftp.squeak.org site has images going all the way back to Squeak 1.1.
Dave
IIUC the unix 3.10-5 VM is still okay and 3.10-6 changed this. We're sort-of lucky that the latest VM package in Fedora is still 3.10-5 (meaning the new OLPC OS will still support Scratch) but it is unfortunate that a minor version would break this. At least I wasn't aware the closure support does break backwards compatibility, and apparently Ian neither, or he would have at least made the change in 3.11 (if not 4.x).
- Bert -
On Sun, Jan 17, 2010 at 08:02:23PM +0100, Bert Freudenberg wrote:
IIUC the unix 3.10-5 VM is still okay and 3.10-6 changed this. We're sort-of lucky that the latest VM package in Fedora is still 3.10-5 (meaning the new OLPC OS will still support Scratch) but it is unfortunate that a minor version would break this. At least I wasn't aware the closure support does break backwards compatibility, and apparently Ian neither, or he would have at least made the change in 3.11 (if not 4.x).
Agreed, and with hindsight I'm sorry that I did not communicate the extent of the changes more clearly. Although to be clear, it was a mistake: I did not expect that the closure changes would break compatibility with a 2.5 based image, because the primitives indices that we removed had been marked obsolete for at least five years.
By way of preventing recurrence, the new VM version numbering approach that Ian has since adopted may help prevent this kind of misunderstanding. The version number is now based on VMMaker versionString combined with the Subversion version number, so for example in the case of the closures changes, the VM version would have gone from 3.9.5-nnnn to 3.10-nnnn (and yes, I should have made it 3.10.0 rather than 3.10, I'll keep that in mind in the future).
Maintaining backward compatibility as we move towards COG or other VM enhancements is going to be a challenge, and we'll have to do this carefully with an eye towards supporting a rather diverse installed base of Squeak images.
Dave
On Sun, Jan 17, 2010 at 2:42 PM, David T. Lewis lewis@mail.msen.com wrote:
On Sun, Jan 17, 2010 at 08:02:23PM +0100, Bert Freudenberg wrote:
IIUC the unix 3.10-5 VM is still okay and 3.10-6 changed this. We're
sort-of
lucky that the latest VM package in Fedora is still 3.10-5 (meaning the
new
OLPC OS will still support Scratch) but it is unfortunate that a minor
version
would break this. At least I wasn't aware the closure support does break backwards compatibility, and apparently Ian neither, or he would have at least made the change in 3.11 (if not 4.x).
Agreed, and with hindsight I'm sorry that I did not communicate the extent of the changes more clearly. Although to be clear, it was a mistake: I did not expect that the closure changes would break compatibility with a 2.5 based image, because the primitives indices that we removed had been marked obsolete for at least five years.
By way of preventing recurrence, the new VM version numbering approach that Ian has since adopted may help prevent this kind of misunderstanding. The version number is now based on VMMaker versionString combined with the Subversion version number, so for example in the case of the closures changes, the VM version would have gone from 3.9.5-nnnn to 3.10-nnnn (and yes, I should have made it 3.10.0 rather than 3.10, I'll keep that in mind in the future).
Maintaining backward compatibility as we move towards COG or other VM enhancements is going to be a challenge, and we'll have to do this carefully with an eye towards supporting a rather diverse installed base of Squeak images.
The move to Cog will break backward-compatibility with all pre-closure images as the Cog VM only runs closures. It does not and cannot easily support old BlockContext-based images. So the issue of backward-compatibility is moot.
The issue is thus one of forward migration. Applying the closure bootstrap is not an easy process but is one that has been done by others; Juan Vuletich migrated Cuis to closures without any help at all; most impressive. With a bit of polish an up-to-date version of the bootstrap should be able to take images forward without too much effort. One needs a VM that runs both closures and old BlockContext, i.e. one of the current clutch of pre-Cog VMs. One needs a bootstrap that has been updated with the relevant bug-fixes.
best Eliot
Dave
vm-dev@lists.squeakfoundation.org
-
Bert Freudenberg -
David T. Lewis -
Eliot Miranda -
John M McIntosh -
John Maloney -
K. K. Subramaniam