Well, I for one would not put "John Maloney on the team" in the same sentence with "unfortunately" ;-)
In any case, to bracket the problem a bit I tried running a Scratch image on the 3.7-7 Unix VM from http://squeakvm.org/unix/ and it seems to work fine. This VM was built from source in March 2005. Source code for that VM is at http://squeakvm.org/unix/release/Squeak-3.7-7.src.tar.gz (which may not include the the Scratch plugins, but is otherwise a workable source code base).
Current Squeak VMs do not support a Scratch image, so the differences are due to changes in the VM / image interface that took place since 2005.
Time permitting I will try next week to identify the major changes that that have taken place since 2005 that would affect Scratch support. In principle it should be possible to produce a VM that will run Scratch along with current Squeak/Pharo/etc although I'm afraid that this will be a difficult task in practice, so a separate source distribution for a VM to run Scratch may be a more pragmatic approach.
With respect to patching the image, this is an extremely easy thing to do, but a difficult thing to manage. Unless there is active development of the Scratch image itself, it is probable better to focus effort on providing a VM that runs the Scratch image as currently distributed. But that is an idea that you might want to run by John for a second opinion if you can.
Dave
On Wed, May 09, 2012 at 01:48:19PM -0400, Amos Blanton wrote:
Thanks to everyone who is looking into this issue.
Unfortunately, the only current member of the Scratch Team with Squeak experience is John Maloney, and he is tied up working on Scratch 2.0 at the moment. I'm told it will not be possible to divert his time towards looking deeply into these issues - although he probably can respond to general questions.
So - if bundling the VM seems like the best solution, that would be fine. We do bundle the VM in the Mac / Windows version of Scratch. The latest official Debian package also bundles an older VM (not included in the current GPL'd Scratch source tarball, but still available here: http://www.assembla.com/spaces/scratchonlinux/trac_subversion_tool ). However, we were once told that this may run afoul of packaging guidelines, and that we should try to avoid it if possible. Perhaps those on thread with more knowledge of packaging guidelines for Debian can say more?
In addition - if someone would like to look into making changes to the Scratch.image to improve compatibility with later VMs, that would be great. I'm not sure how patching works with Squeak images, and if it would be possible for packagers to patch the current source. But if that's difficult, we could update the source package based on recommendations / changes from the OSS / Squeak community.
On Tue, May 8, 2012 at 10:52 AM, David T. Lewis lewis@mail.msen.com wrote:
On Tue, May 08, 2012 at 02:39:45PM +0200, Bert Freudenberg wrote:
On 08.05.2012, at 14:21, David T. Lewis wrote:
On Tue, May 08, 2012 at 10:44:26AM +0100, Alex Bradbury wrote:
David and Igor: thanks for your responses. Comments below.
On 7 May 2012 00:28, David T. Lewis lewis@mail.msen.com wrote:
If I remember correctly, Scratch derives from a Squeak 3.6 image and
uses
a VM of similar vintage. Since the days of Squeak 3.6, there have
been
a number of fundamental changes in the Squeak VM. I recall some
discussion
of this perhaps a couple of years ago, and I think that the
prevailing
view was that it made more sense to bring the Scratch image up to
date
with respect to the VM changes, as opposed to trying to make a single VM that could be backward compatible all the way to Squeak 3.6.
Though the current squeak image did work flawlessly in squeak-vm 4.0? So the upstream view is that this is not officially supported and the correct solution if it no longer works on 4.4 is to just use an older vm?
From a license point of view, is there any reason that the Scratch VM and source code could not be distributed on Debian? If this is
acceptable,
then I expect that the fastest way to achieve a working distribution
would
involve arranging for Scratch to run using a VM at e.g.
/usr/bin/scratch,
and Squeak to run with /usr/bin/squeak or /usr/bin/cog.
It does seem that the Ubuntu packages bundles its own scratch_squeak_vm. Either that, or supporting the installation of squeak-vm 4.0 alongside 4.4 might be good options. For my purposes with the Raspberry Pi, I'm probably just going to install and pin the squeeze squeak-vm (4.0) until a better solution comes along.
Alex
On 8 May 2012 10:44, Alex Bradbury asb@asbradbury.org wrote:
Though the current squeak image did work flawlessly in squeak-vm 4.0? So the upstream view is that this is not officially supported and the correct solution if it no longer works on 4.4 is to just use an older vm?
squeak image -> scratch image
Alex
As far as I know, Scratch images have not worked on standard Squeak VMs for a number of years, and Scratch has always relied on shipping its own VM. In addition to fundamental changes in the VM itself that have taken place in recent years, I believe that the Scratch version of the VM also includes a couple of plugins (add-in functionality) that are required for Scratch but not included in Squeak.
I think I see one source of confusion here. I just read the link at http://info.scratch.mit.edu/Source_Code and find the following
statement:
"Since Scratch simply uses a stock Squeak virtual machine, we are not distributing the Squeak virtual machine source code. Both the source code and pre-compiled binaries for the Squeak virtual machine are available at www.squeakvm.org."
Unfortunately, I don't think that this is true, and hopefully someone can make a correction to that web page.
The squeakvm.org code repository along with VMMaker at
source.squeak.org
does contain fully versioned copies of the Squeak VM, presumably
including
all or most of the code in the current Scratch VM (except for the
Scratch
specific plugins). So it would probably be possible to build a working Scratch VM using code from the standard Squeak VM repositories. But I fear this would take some work on the part of someone who knows Scratch well enough to find and build the right versions and put the pieces together.
Dave
Well, actually, Scratch (based on Squeak 2.8 IIRC) worked fine up to the
3.10 VM. E.g. OLPC's Scratch package does not bundle a VM but uses the stock Fedora Squeak VM (3.10).
For a very long time we managed to keep VMs being able to run all Squeak
images back to 1.x. The promise was new VM + old image = fine, but new images may require new VMs. Upgrading the VM should not unnecessarily break old images.
So what did we change in 4.x VMs that it does not work anymore?
I am not certain, but we will need to figure it if it is causing problems now for Scratch and Debian. I probably will not be able to look into this myself until next week though.
I know that I can run a Squeak 3.6 image on the most up-to-date SqueakVM, but going back further in time gets problematic, and it definitely is not possible to run a 1.x image on current VMs.
My recollection (not necessarily correct) is that the mechanism for delivering events from the VM to the image was changed in a fundamental way. There would probably also be some issues related to the special objects array and primitive number assignments.
I think there was some discussion of this on the vm-dev list a couple of years ago, and some related emails with John Maloney at the time. I don't have a link right now, but I'll see if I can find it.
Dave
If these changes are really unavoidable and incompatible, then we should
indeed advise distros to package both 3.x and 4.x VMs.
- Bert -
Well, thanks to our trusty Mantis bug tracker, it looks like we have a record of the Scratch related issues. The VM compatibility issues are documented here:
Mantis 0007454: Removal of obsolete prim support for closures is a problem for Scratch images http://bugs.squeak.org/view.php?id=7454
And the Scratch plugins and licensing are here:
Mantis 0007654: Add Scratch plugins CameraPlugin and ScratchPlugin to standard VMs http://bugs.squeak.org/view.php?id=7654
It seems that the VM compatibility issues were introduced quite intentionally as part of the move to support proper block closures in Squeak. As of 2011, the thought was that the proper way forward was to do the corresponding updates in the Scratch image to incorporate full block closures, which would be an improvement for Scratch and would also remove the VM compatibility concerns.
This would still be the right thing to do, although I gather from the Scratch web site that further development of the Scratch image may no longer be a priority.
Amos, can you please clarify if further development of the Scratch image is planned? If yes, then updating the Scratch image may be the right thing to do. If no, then we can take a harder look at coming up with a way to produce a backward compatible VM. If neither of these is feasible, then it will probably be necessary to arrange a separate source distribution for a Scratch VM.
Thanks, Dave
On Wed, May 09, 2012 at 09:09:26PM -0400, David T. Lewis wrote:
Well, I for one would not put "John Maloney on the team" in the same sentence with "unfortunately" ;-)
In any case, to bracket the problem a bit I tried running a Scratch image on the 3.7-7 Unix VM from http://squeakvm.org/unix/ and it seems to work fine. This VM was built from source in March 2005. Source code for that VM is at http://squeakvm.org/unix/release/Squeak-3.7-7.src.tar.gz (which may not include the the Scratch plugins, but is otherwise a workable source code base).
Current Squeak VMs do not support a Scratch image, so the differences are due to changes in the VM / image interface that took place since 2005.
Time permitting I will try next week to identify the major changes that that have taken place since 2005 that would affect Scratch support. In principle it should be possible to produce a VM that will run Scratch along with current Squeak/Pharo/etc although I'm afraid that this will be a difficult task in practice, so a separate source distribution for a VM to run Scratch may be a more pragmatic approach.
With respect to patching the image, this is an extremely easy thing to do, but a difficult thing to manage. Unless there is active development of the Scratch image itself, it is probable better to focus effort on providing a VM that runs the Scratch image as currently distributed. But that is an idea that you might want to run by John for a second opinion if you can.
Dave
On Wed, May 09, 2012 at 01:48:19PM -0400, Amos Blanton wrote:
Thanks to everyone who is looking into this issue.
Unfortunately, the only current member of the Scratch Team with Squeak experience is John Maloney, and he is tied up working on Scratch 2.0 at the moment. I'm told it will not be possible to divert his time towards looking deeply into these issues - although he probably can respond to general questions.
So - if bundling the VM seems like the best solution, that would be fine. We do bundle the VM in the Mac / Windows version of Scratch. The latest official Debian package also bundles an older VM (not included in the current GPL'd Scratch source tarball, but still available here: http://www.assembla.com/spaces/scratchonlinux/trac_subversion_tool ). However, we were once told that this may run afoul of packaging guidelines, and that we should try to avoid it if possible. Perhaps those on thread with more knowledge of packaging guidelines for Debian can say more?
In addition - if someone would like to look into making changes to the Scratch.image to improve compatibility with later VMs, that would be great. I'm not sure how patching works with Squeak images, and if it would be possible for packagers to patch the current source. But if that's difficult, we could update the source package based on recommendations / changes from the OSS / Squeak community.
On Tue, May 8, 2012 at 10:52 AM, David T. Lewis lewis@mail.msen.com wrote:
On Tue, May 08, 2012 at 02:39:45PM +0200, Bert Freudenberg wrote:
On 08.05.2012, at 14:21, David T. Lewis wrote:
On Tue, May 08, 2012 at 10:44:26AM +0100, Alex Bradbury wrote:
David and Igor: thanks for your responses. Comments below.
On 7 May 2012 00:28, David T. Lewis lewis@mail.msen.com wrote: > If I remember correctly, Scratch derives from a Squeak 3.6 image and
uses
> a VM of similar vintage. Since the days of Squeak 3.6, there have
been
> a number of fundamental changes in the Squeak VM. I recall some
discussion
> of this perhaps a couple of years ago, and I think that the
prevailing
> view was that it made more sense to bring the Scratch image up to
date
> with respect to the VM changes, as opposed to trying to make a single > VM that could be backward compatible all the way to Squeak 3.6.
Though the current squeak image did work flawlessly in squeak-vm 4.0? So the upstream view is that this is not officially supported and the correct solution if it no longer works on 4.4 is to just use an older vm?
> From a license point of view, is there any reason that the Scratch VM > and source code could not be distributed on Debian? If this is
acceptable,
> then I expect that the fastest way to achieve a working distribution
would
> involve arranging for Scratch to run using a VM at e.g.
/usr/bin/scratch,
> and Squeak to run with /usr/bin/squeak or /usr/bin/cog.
It does seem that the Ubuntu packages bundles its own scratch_squeak_vm. Either that, or supporting the installation of squeak-vm 4.0 alongside 4.4 might be good options. For my purposes with the Raspberry Pi, I'm probably just going to install and pin the squeeze squeak-vm (4.0) until a better solution comes along.
Alex
On 8 May 2012 10:44, Alex Bradbury asb@asbradbury.org wrote:
> Though the current squeak image did work flawlessly in squeak-vm 4.0? > So the upstream view is that this is not officially supported and the > correct solution if it no longer works on 4.4 is to just use an older > vm?
squeak image -> scratch image
Alex
As far as I know, Scratch images have not worked on standard Squeak VMs for a number of years, and Scratch has always relied on shipping its own VM. In addition to fundamental changes in the VM itself that have taken place in recent years, I believe that the Scratch version of the VM also includes a couple of plugins (add-in functionality) that are required for Scratch but not included in Squeak.
I think I see one source of confusion here. I just read the link at http://info.scratch.mit.edu/Source_Code and find the following
statement:
"Since Scratch simply uses a stock Squeak virtual machine, we are not distributing the Squeak virtual machine source code. Both the source code and pre-compiled binaries for the Squeak virtual machine are available at www.squeakvm.org."
Unfortunately, I don't think that this is true, and hopefully someone can make a correction to that web page.
The squeakvm.org code repository along with VMMaker at
source.squeak.org
does contain fully versioned copies of the Squeak VM, presumably
including
all or most of the code in the current Scratch VM (except for the
Scratch
specific plugins). So it would probably be possible to build a working Scratch VM using code from the standard Squeak VM repositories. But I fear this would take some work on the part of someone who knows Scratch well enough to find and build the right versions and put the pieces together.
Dave
Well, actually, Scratch (based on Squeak 2.8 IIRC) worked fine up to the
3.10 VM. E.g. OLPC's Scratch package does not bundle a VM but uses the stock Fedora Squeak VM (3.10).
For a very long time we managed to keep VMs being able to run all Squeak
images back to 1.x. The promise was new VM + old image = fine, but new images may require new VMs. Upgrading the VM should not unnecessarily break old images.
So what did we change in 4.x VMs that it does not work anymore?
I am not certain, but we will need to figure it if it is causing problems now for Scratch and Debian. I probably will not be able to look into this myself until next week though.
I know that I can run a Squeak 3.6 image on the most up-to-date SqueakVM, but going back further in time gets problematic, and it definitely is not possible to run a 1.x image on current VMs.
My recollection (not necessarily correct) is that the mechanism for delivering events from the VM to the image was changed in a fundamental way. There would probably also be some issues related to the special objects array and primitive number assignments.
I think there was some discussion of this on the vm-dev list a couple of years ago, and some related emails with John Maloney at the time. I don't have a link right now, but I'll see if I can find it.
Dave
If these changes are really unavoidable and incompatible, then we should
indeed advise distros to package both 3.x and 4.x VMs.
- Bert -
vm-dev@lists.squeakfoundation.org
-
David T. Lewis