Bert and others have raised the question of how best to package the Cog VM and traditional interpreter VM for the next round of VM releases in December. Cog is a significant advance, and the Squeak board would like this to be fully supported for the Squeak 4.2 release; clearly it is a priority for Pharo as well. Meanwhile we have a responsibility for Etoys, Scratch, OLPC etc to provide VMs that do not break those systems.
So - how best to do this?
For unix VMs, Bert has suggested this:
Maybe we need a "squeak-interp" and "squeak-cog" binary, with "squeak" symlinking to either one (using the alternatives system) or "squeak" being a script choosing the right VM based on the image version. But we should present a coherent story to the packagers. As well as to Squeak users.
Personally I am inclined to think it would be best for this next release not to automate the selection of VM, but instead have users make an explicit selection. I say this for two reasons:
1) From a marketing point of view, it is good if users can directly experience the improvement from Cog. So the message might be to first run in the standard way (squeak-interp) to get a very cool system that is quite fast, then activate turbo mode (squeak-cog) to get a system that is very noticeably 3X faster.
2) From a technical and support point of view, there may be various cases where it will be necessary to say "sorry, you will need to run squeak-interp if you want to do that." When that happens, it would be best if we do not need to explain e.g. how to edit a shell script to force it to run squeak-interp.
The above is just my $.02 to get the discussion started. What do others think?
Dave
I'm of the mind that Apple takes at the moment. Try the action and then deal with the failure case. I'd just ship with the Cog VM turned on. If people have problems with it they can pursue how to turn it off. Otherwise you'll not get people to migrate.
On 2010-11-07, at 12:27 PM, David T. Lewis wrote:
Bert and others have raised the question of how best to package the Cog VM and traditional interpreter VM for the next round of VM releases in December. Cog is a significant advance, and the Squeak board would like this to be fully supported for the Squeak 4.2 release; clearly it is a priority for Pharo as well. Meanwhile we have a responsibility for Etoys, Scratch, OLPC etc to provide VMs that do not break those systems.
So - how best to do this?
For unix VMs, Bert has suggested this:
Maybe we need a "squeak-interp" and "squeak-cog" binary, with "squeak" symlinking to either one (using the alternatives system) or "squeak" being a script choosing the right VM based on the image version. But we should present a coherent story to the packagers. As well as to Squeak users.
Personally I am inclined to think it would be best for this next release not to automate the selection of VM, but instead have users make an explicit selection. I say this for two reasons:
- From a marketing point of view, it is good if users can directly experience
the improvement from Cog. So the message might be to first run in the standard way (squeak-interp) to get a very cool system that is quite fast, then activate turbo mode (squeak-cog) to get a system that is very noticeably 3X faster.
- From a technical and support point of view, there may be various cases
where it will be necessary to say "sorry, you will need to run squeak-interp if you want to do that." When that happens, it would be best if we do not need to explain e.g. how to edit a shell script to force it to run squeak-interp.
The above is just my $.02 to get the discussion started. What do others think?
Dave
-- =========================================================================== John M. McIntosh johnmci@smalltalkconsulting.com Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
On 7 November 2010 23:22, John M McIntosh johnmci@smalltalkconsulting.com wrote:
I'm of the mind that Apple takes at the moment. Try the action and then deal with the failure case. I'd just ship with the Cog VM turned on. If people have problems with it they can pursue how to turn it off. Otherwise you'll not get people to migrate.
+1 If people like to use older VM(s), they can always download them.
On 2010-11-07, at 12:27 PM, David T. Lewis wrote:
On Mon, Nov 08, 2010 at 04:03:13AM +0200, Igor Stasenko wrote:
On 7 November 2010 23:22, John M McIntosh johnmci@smalltalkconsulting.com wrote:
I'm of the mind that Apple takes at the moment. Try the action and then deal with the failure case. ?? I'd just ship with the Cog VM turned on. ??If people have problems with it they can pursue how to turn it off. ?? Otherwise you'll not get people to migrate.
+1 If people like to use older VM(s), they can always download them.
To be clear, the intent is to provide both Cog and the interpreter VM. The question is how best to package this such that we deliver the benefits of Cog, but also ensure that the system still works when a child opens an older Etoys image after their teacher has installed the new VM. In the near term at least, we also want to have a straightforward way to answer the question "I tried to do XXX and it does not work, what should I do?" in cases where XXX does not yet work on Cog.
Dave
On 11/8/2010 3:35 AM, David T. Lewis wrote:
To be clear, the intent is to provide both Cog and the interpreter VM.
Agree. I think the first step is to unify the support code. I.e., when it's all said and done we should have a structure roughly along the lines of:
platforms/ Cross/ Unix/ Mac/ Win/ src-interp/ src-jit/
etc. and the *only* difference should be in the src-interp vs. src-jit directories. In which case we can decide what to build based on the selecting the appropriate interp/jit source directory.
Cheers, - Andreas
The question is how best to package this such that we deliver the benefits of Cog, but also ensure that the system still works when a child opens an older Etoys image after their teacher has installed the new VM. In the near term at least, we also want to have a straightforward way to answer the question "I tried to do XXX and it does not work, what should I do?" in cases where XXX does not yet work on Cog.
Dave
On Mon, Nov 08, 2010 at 11:43:50AM -0800, Andreas Raab wrote:
On 11/8/2010 3:35 AM, David T. Lewis wrote:
To be clear, the intent is to provide both Cog and the interpreter VM.
Agree. I think the first step is to unify the support code. I.e., when it's all said and done we should have a structure roughly along the lines of:
platforms/ Cross/ Unix/ Mac/ Win/ src-interp/ src-jit/
etc. and the *only* difference should be in the src-interp vs. src-jit directories. In which case we can decide what to build based on the selecting the appropriate interp/jit source directory.
To check my understanding, you are suggesting that the src-interp and src-jit directories would be for the generated sources, as distinct from the platform support code under platforms/... and that the structure of the platforms/... tree would remain unchanged. Is that right?
Dave
On 11/8/2010 5:39 PM, David T. Lewis wrote:
On Mon, Nov 08, 2010 at 11:43:50AM -0800, Andreas Raab wrote:
On 11/8/2010 3:35 AM, David T. Lewis wrote:
To be clear, the intent is to provide both Cog and the interpreter VM.
Agree. I think the first step is to unify the support code. I.e., when it's all said and done we should have a structure roughly along the lines of:
platforms/ Cross/ Unix/ Mac/ Win/ src-interp/ src-jit/
etc. and the *only* difference should be in the src-interp vs. src-jit directories. In which case we can decide what to build based on the selecting the appropriate interp/jit source directory.
To check my understanding, you are suggesting that the src-interp and src-jit directories would be for the generated sources, as distinct from the platform support code under platforms/... and that the structure of the platforms/... tree would remain unchanged. Is that right?
Exactly. Most importantly the platforms support code would be *identical* across the variants and thus eliminate further proliferation of various bits of C code.
Cheers, - Andreas
On 09.11.2010, at 03:44, Andreas Raab wrote:
On 11/8/2010 5:39 PM, David T. Lewis wrote:
On Mon, Nov 08, 2010 at 11:43:50AM -0800, Andreas Raab wrote:
On 11/8/2010 3:35 AM, David T. Lewis wrote:
To be clear, the intent is to provide both Cog and the interpreter VM.
Agree. I think the first step is to unify the support code. I.e., when it's all said and done we should have a structure roughly along the lines of:
platforms/ Cross/ Unix/ Mac/ Win/ src-interp/ src-jit/
etc. and the *only* difference should be in the src-interp vs. src-jit directories. In which case we can decide what to build based on the selecting the appropriate interp/jit source directory.
To check my understanding, you are suggesting that the src-interp and src-jit directories would be for the generated sources, as distinct from the platform support code under platforms/... and that the structure of the platforms/... tree would remain unchanged. Is that right?
Exactly. Most importantly the platforms support code would be *identical* across the variants and thus eliminate further proliferation of various bits of C code.
Cheers,
- Andreas
I'd love to have this structure. Making the platform names more uniform is good. Though I'd retain the space in "Mac OS" because many files in that subtree contain spaces, and it's a visible reminder of that.
Shouldn't we have a single folder for generated plugins? Also, I always felt that "src" was misleading. So how about this:
platforms/ Cross/ Unix/ Mac OS/ Win/ iOS/ generated/ interp/ jit/ stack/ plugins/
However, one reason to put the generated sources inside the platform folders was because often the platform code lagged behind, and only the single maintainer could fix it. A better solution to that would be opening up VM development. As I mentioned before I'd like us to move to a DCVS. If we intend to change the repository structure this would be an excellent time.
- Bert -
On Tuesday 09 Nov 2010 3:55:53 pm Bert Freudenberg wrote:
Shouldn't we have a single folder for generated plugins?
IMHO, plugins should be capable of being built anywhere, even outside the VM sources. Plugins, by definition, use arch-specific or os-specific features and the 'upper half' could be squirted into platform directory as necessary.
Also, I always felt that "src" was misleading. So how about this:
platforms/ Cross/ Unix/ Mac OS/ Win/ iOS/
I prefer <arch>/<platform>/<os> structure (e.g. x86/pc/Win32, WinCE, Win64, linux, Mac OS, arm/davinci/linux,..). It is much easier to re-use code and development tools this way.
generated/ interp/ jit/ stack/ plugins/
I would vote to skip generated/ and Cross/ directories and leave all these file at top level. Apart from C code, there could also be auto-generated configuration files, Makefiles etc. Why treat C code specially?
However, one reason to put the generated sources inside the platform folders was because often the platform code lagged behind, and only the single maintainer could fix it. A better solution to that would be opening up VM development.
+1
As I mentioned before I'd like us to move to a DCVS. If we intend to change the repository structure this would be an excellent time.
A distributed version manager like git would be useful only for hand-coded files. What about blobs like images, changesets and sources?
Subbu
On Tue, Nov 9, 2010 at 8:56 AM, K. K. Subramaniam kksubbu.ml@gmail.com wrote:
A distributed version manager like git would be useful only for hand-coded files. What about blobs like images, changesets and sources?
How do you figure? Git or Mercurial can handle binary files as well as subversion.
Colin
On 09.11.2010, at 18:00, Colin Putney wrote:
On Tue, Nov 9, 2010 at 8:56 AM, K. K. Subramaniam kksubbu.ml@gmail.com wrote:
A distributed version manager like git would be useful only for hand-coded files. What about blobs like images, changesets and sources?
How do you figure? Git or Mercurial can handle binary files as well as subversion.
Colin
We're talking about the VM source tree. So far there are no images in there.
- Bert -
On Tuesday 09 Nov 2010 10:30:17 pm Colin Putney wrote:
On Tue, Nov 9, 2010 at 8:56 AM, K. K. Subramaniam kksubbu.ml@gmail.com
wrote:
A distributed version manager like git would be useful only for hand-coded files. What about blobs like images, changesets and sources?
How do you figure? Git or Mercurial can handle binary files as well as subversion.
Yes. but the history gets quite big with image blobs.
Bert's reply is ingenious ;-). Generated files are intermediate files, not true source files. True "sources" are actually Slang code in VMMaker image and these should be in version control.
When bootstrapping VM on a new platform, a sub-set of these intermediate files along with hand-coded source files are compiled generate a proto-vm. The proto- vm runs VMMaker image to translate all Slang code into C so that the regular VM and plugins can now be built and the proto-vm can be discarded.
The VMMaker image is a critical part of bootstrap and should be put under version control.
Subbu
On 11/9/2010 10:10 AM, K. K. Subramaniam wrote:
Bert's reply is ingenious ;-). Generated files are intermediate files, not true source files. True "sources" are actually Slang code in VMMaker image and these should be in version control.
The generated files are true source files. If you've ever had the need to ensure that you're building the same VMs across various platforms you'll quickly find that you're moving generated files around, not images.
When bootstrapping VM on a new platform, a sub-set of these intermediate files along with hand-coded source files are compiled generate a proto-vm. The proto- vm runs VMMaker image to translate all Slang code into C so that the regular VM and plugins can now be built and the proto-vm can be discarded.
The VMMaker image is a critical part of bootstrap and should be put under version control.
No and no. When bootstrapping a new VM on a new platform you move the generated files and start from those. The VMMaker image is an afterthought, only used for generation of the sources and not involved in the bootstrap of a new platform at all.
Cheers, - Andreas
On Tuesday 09 Nov 2010 11:50:42 pm Andreas Raab wrote:
On 11/9/2010 10:10 AM, K. K. Subramaniam wrote:
Bert's reply is ingenious ;-). Generated files are intermediate files, not true source files. True "sources" are actually Slang code in VMMaker image and these should be in version control.
The generated files are true source files. If you've ever had the need to ensure that you're building the same VMs across various platforms you'll quickly find that you're moving generated files around, not images.
I meant "source" in the sense of being able to copyright, apply patches and track version history, not just in the sense of using them to compile into object code.
While it is possible to bypass VMMaker and maintain sources manually, it is not how Squeak is being maintained today. AFAIK, If there is a bug in interp.c or in any of the internal plugin code that causes build to fail, patches go into VMMaker, not into generated source. The generated code contains a reference to VMMaker version for applying patches.
Of course, once the patched code is regenerated, VMMaker is not required for subsequent builds. Then, what you say applies.
Subbu
On Wed, Nov 10, 2010 at 11:33:28PM +0530, K. K. Subramaniam wrote:
On Tuesday 09 Nov 2010 11:50:42 pm Andreas Raab wrote:
On 11/9/2010 10:10 AM, K. K. Subramaniam wrote:
Bert's reply is ingenious ;-). Generated files are intermediate files, not true source files. True "sources" are actually Slang code in VMMaker image and these should be in version control.
The generated files are true source files. If you've ever had the need to ensure that you're building the same VMs across various platforms you'll quickly find that you're moving generated files around, not images.
I meant "source" in the sense of being able to copyright, apply patches and track version history, not just in the sense of using them to compile into object code.
While it is possible to ???bypass VMMaker and maintain sources manually, it is not how Squeak is being maintained today. AFAIK, If there is a bug in interp.c or in any of the internal plugin code that causes build to fail, patches go into VMMaker, not into generated source. The generated code contains a reference to VMMaker version for applying patches.
Of course, once the patched code is regenerated, VMMaker is not required for subsequent builds. Then, what you say applies.
I think we are mixing words here. From the point of view of development, Smalltalk is the source. From the point of view of rebuilding the VM in a reliable and reproducable manner, the generated C files are the source. Both are valid and important uses. It is important for people to be aware of the distinction so that they do not mistakenly "fix" a problem by modifying generated C sources, but aside from that there is no conflict.
The various suggestions for keeping generated sources outside of the ./platforms tree are, I presume, intended to reduce this kind of confusion.
Dave
On 11/9/2010 2:25 AM, Bert Freudenberg wrote:
Shouldn't we have a single folder for generated plugins? Also, I always felt that "src" was misleading. So how about this:
platforms/ Cross/ Unix/ Mac OS/ Win/ iOS/ generated/ interp/ jit/ stack/ plugins/
However, one reason to put the generated sources inside the platform folders was because often the platform code lagged behind, and only the single maintainer could fix it. A better solution to that would be opening up VM development. As I mentioned before I'd like us to move to a DCVS. If we intend to change the repository structure this would be an excellent time.
I don't think these issues are related at all. A particular release will still bundle a certain version of the platforms tree with a certain version of the generated code. However, there is no requirement for the generated code to live inside the platforms directory. To make it one notch more complicated, our internal structure for VM building actually puts the various toolchains side-by-side with platforms and src/generated, i.e.,
cygwinbuild/ "<- Windows Cygwin build" macbuild/ "<- Mac X-Code build" mgwbuild/ "<- Windows MingW build" mscbuild/ "<- Windows Visual C++ build" unixbuild/ "<- Unix gcc build" platforms/ src/
This has some advantages for us since we can check in and build VMs from the same (checked-in) src/ and platforms/ tree in the various environments, but I'm not sure we need this for the Squeak VM in general.
Cheers, - Andreas
On 9 November 2010 12:25, Bert Freudenberg bert@freudenbergs.de wrote:
I'd love to have this structure. Making the platform names more uniform is good. Though I'd retain the space in "Mac OS" because many files in that subtree contain spaces, and it's a visible reminder of that.
Shouldn't we have a single folder for generated plugins? Also, I always felt that "src" was misleading. So how about this:
platforms/ Cross/ Unix/ Mac OS/ Win/ iOS/ generated/ interp/ jit/ stack/ plugins/
i like this separation. Ookay, Andreas insists that generated files should be version controled. No problem. But existing files layout makes it hard for newcomers to distinguish generated sources from hand-written ones, which often leads to problems "i lost, i can't compile it ". So a clear separation of generated sources from hand-written ones is important improvement.
However, one reason to put the generated sources inside the platform folders was because often the platform code lagged behind, and only the single maintainer could fix it. A better solution to that would be opening up VM development. As I mentioned before I'd like us to move to a DCVS. If we intend to change the repository structure this would be an excellent time.
We talked about it before. It would be good to migrate on github (or something like this), so we could have more open development process. Maybe its time to make it happen? :)
- Bert -
On 11/9/2010 12:55 PM, Igor Stasenko wrote:
On 9 November 2010 12:25, Bert Freudenbergbert@freudenbergs.de wrote:
I'd love to have this structure. Making the platform names more uniform is good. Though I'd retain the space in "Mac OS" because many files in that subtree contain spaces, and it's a visible reminder of that.
Shouldn't we have a single folder for generated plugins? Also, I always felt that "src" was misleading. So how about this:
platforms/ Cross/ Unix/ Mac OS/ Win/ iOS/ generated/ interp/ jit/ stack/ plugins/
i like this separation. Ookay, Andreas insists that generated files should be version controled. No problem.
I'm not insisting. I'm just saying that we have them in version control so that we can reproduce our production VMs faithfully on all platforms.
But existing files layout makes it hard for newcomers to distinguish generated sources from hand-written ones, which often leads to problems "i lost, i can't compile it ". So a clear separation of generated sources from hand-written ones is important improvement.
I'm missing something. The current structure allows for the above. If you'd like to check out for example http://squeakvm.org/win32/release/SqueakVM-Win32-4.1.1-src.zip you will see that the structure is -guess what- almost identical:
platforms/ Cross/ Win32/ winbuild/ src/
The generated sources are inside winbuild (for historical reasons) but they are *very* clearly separated from the hand-written stuff. I'm not sure why everyone is so excited about a layout that we already support?
Cheers, - Andreas
On 9 November 2010 23:17, Andreas Raab andreas.raab@gmx.de wrote:
On 11/9/2010 12:55 PM, Igor Stasenko wrote:
On 9 November 2010 12:25, Bert Freudenbergbert@freudenbergs.de wrote:
I'd love to have this structure. Making the platform names more uniform is good. Though I'd retain the space in "Mac OS" because many files in that subtree contain spaces, and it's a visible reminder of that.
Shouldn't we have a single folder for generated plugins? Also, I always felt that "src" was misleading. So how about this:
platforms/ Cross/ Unix/ Mac OS/ Win/ iOS/ generated/ interp/ jit/ stack/ plugins/
i like this separation. Ookay, Andreas insists that generated files should be version controled. No problem.
I'm not insisting. I'm just saying that we have them in version control so that we can reproduce our production VMs faithfully on all platforms.
But existing files layout makes it hard for newcomers to distinguish generated sources from hand-written ones, which often leads to problems "i lost, i can't compile it ". So a clear separation of generated sources from hand-written ones is important improvement.
I'm missing something. The current structure allows for the above. If you'd like to check out for example http://squeakvm.org/win32/release/SqueakVM-Win32-4.1.1-src.zip you will see that the structure is -guess what- almost identical:
platforms/ Cross/ Win32/ winbuild/ src/
Huh? Are we talking about same things?
Take a look at our official SVN tree:
http://squeakvm.org/svn/squeak/trunk/
where winbuild?
Take a look where generated sources located for unix:
http://squeakvm.org/svn/squeak/trunk/platforms/unix/src/
The generated sources are inside winbuild (for historical reasons) but they are *very* clearly separated from the hand-written stuff. I'm not sure why everyone is so excited about a layout that we already support?
Yes. Except that if you check out from svn, there is no winbuild. If you mean that you packaging a VM source (in zip archive) , and there it contains winbuild dir, this is different. We are talking about version-controlled repository with good layout from very starting. No?
Cheers, - Andreas
On 11/9/2010 1:42 PM, Igor Stasenko wrote:
I'm missing something. The current structure allows for the above. If you'd like to check out for example http://squeakvm.org/win32/release/SqueakVM-Win32-4.1.1-src.zip you will see that the structure is -guess what- almost identical:
platforms/ Cross/ Win32/ winbuild/ src/
Huh? Are we talking about same things?
Clearly we aren't, otherwise I wouldn't be so confused :-)
Take a look at our official SVN tree:
http://squeakvm.org/svn/squeak/trunk/
where winbuild?
In platforms/win32/build. This is the proto-location and one can either build things in there or not. I choose to keep things separate but it's a choice I am making for my build setups. Others can make other choices.
The generated sources are inside winbuild (for historical reasons) but they are *very* clearly separated from the hand-written stuff. I'm not sure why everyone is so excited about a layout that we already support?
Yes. Except that if you check out from svn, there is no winbuild. If you mean that you packaging a VM source (in zip archive) , and there it contains winbuild dir, this is different. We are talking about version-controlled repository with good layout from very starting. No?
I'm not sure what we are talking about. There appears to be an immense enthusiasm about the fact that - Lo and behold! - VMMaker can generate sources in different locations. What an amazing new insight :-)
Is this simply about checking the generated sources into SVN? Or the build setups? If so, let's just do that and get over it. It's a trivial little thing to do and really doesn't warrant this lengthy discussion. If not, then I'm still completely unclear what this discussion is supposedly about.
Cheers, - Andreas
On 10 November 2010 00:40, Andreas Raab andreas.raab@gmx.de wrote:
On 11/9/2010 1:42 PM, Igor Stasenko wrote:
I'm missing something. The current structure allows for the above. If you'd like to check out for example http://squeakvm.org/win32/release/SqueakVM-Win32-4.1.1-src.zip you will see that the structure is -guess what- almost identical:
platforms/ Cross/ Win32/ winbuild/ src/
Huh? Are we talking about same things?
Clearly we aren't, otherwise I wouldn't be so confused :-)
Take a look at our official SVN tree:
http://squeakvm.org/svn/squeak/trunk/
where winbuild?
In platforms/win32/build. This is the proto-location and one can either build things in there or not. I choose to keep things separate but it's a choice I am making for my build setups. Others can make other choices.
The generated sources are inside winbuild (for historical reasons) but they are *very* clearly separated from the hand-written stuff. I'm not sure why everyone is so excited about a layout that we already support?
Yes. Except that if you check out from svn, there is no winbuild. If you mean that you packaging a VM source (in zip archive) , and there it contains winbuild dir, this is different. We are talking about version-controlled repository with good layout from very starting. No?
I'm not sure what we are talking about. There appears to be an immense enthusiasm about the fact that - Lo and behold! - VMMaker can generate sources in different locations. What an amazing new insight :-)
hehe.. actually i were always using separate dir from very starting. But on unixes its somewhat messy. For instance, i prefer to delete the src subdir to know clearly that during build i using right location of generated sources, and not interfering with ones which come by default from SVN (platforms/unix/src).
Since there is no point to keep multiple copies of generated sources for each platform, i think they should be put outside plarforms dir, into a separate (as Bert suggested) 'generated' dir. So, then all platforms could use these files as a default location for generated sources, while of course we're free to use different one.
Is this simply about checking the generated sources into SVN? Or the build setups? If so, let's just do that and get over it. It's a trivial little thing to do and really doesn't warrant this lengthy discussion. If not, then I'm still completely unclear what this discussion is supposedly about.
I'd prefer to have everything we need to be stored in SVN (err.. github ;) ), so i just checking out, cd into some subdir, do make and voila. 'one click' for geek :)
Cheers, - Andreas
On 11/9/2010 4:25 PM, Igor Stasenko wrote:
Since there is no point to keep multiple copies of generated sources for each platform, i think they should be put outside plarforms dir, into a separate (as Bert suggested) 'generated' dir. So, then all platforms could use these files as a default location for generated sources, while of course we're free to use different one.
Fine. I couldn't care less. If people want to check some version of the generated sources into SVN, that's absolutely fine with me.
Is this simply about checking the generated sources into SVN? Or the build setups? If so, let's just do that and get over it. It's a trivial little thing to do and really doesn't warrant this lengthy discussion. If not, then I'm still completely unclear what this discussion is supposedly about.
I'd prefer to have everything we need to be stored in SVN (err.. github ;) ), so i just checking out, cd into some subdir, do make and voila. 'one click' for geek :)
You're not going to change the need for file releases. The idea that at any given point you can just check out SVN head and get a functioning build on all platforms has never been true in the past, and won't be true in the future. Different people have different amounts of time for working on the various aspects so head is expected to be a bit problematic at any point in time. That's why we have file releases which provide *exactly* what you are asking for. Download the file release from http://squeakvm.org/win32/release/SqueakVM-Win32-4.1.1-src.zip, extract it, cd into the winbuild directory, type "make" and out pops a shiny new VM. Whether the winbuild directory is SVN-backed or not, matters little in this context.
Cheers, - Andreas
On 11/7/2010 12:27 PM, David T. Lewis wrote:
The above is just my $.02 to get the discussion started. What do others think?
I don't really have any skin in the game since this is a Unix specific issue (i.e., the "squeakvm" dependency) that doesn't really apply on Mac or Windows. That said, I'll add my $.02 by saying that in our Linux production systems we do exactly what you were proposing, i.e., we have fallback VMs installed side-by-side with the latest and symlink the desired VM. That makes it very easy to switch the default *and* allows you to be explicit where you want to (i.e., by executing from the linked-to location).
Cheers, - Andreas
On Sun, Nov 07, 2010 at 07:29:56PM -0800, Andreas Raab wrote:
On 11/7/2010 12:27 PM, David T. Lewis wrote:
The above is just my $.02 to get the discussion started. What do others think?
I don't really have any skin in the game since this is a Unix specific issue (i.e., the "squeakvm" dependency) that doesn't really apply on Mac or Windows. That said, I'll add my $.02 by saying that in our Linux production systems we do exactly what you were proposing, i.e., we have fallback VMs installed side-by-side with the latest and symlink the desired VM. That makes it very easy to switch the default *and* allows you to be explicit where you want to (i.e., by executing from the linked-to location).
Well that is Bert's suggestion exactly, and it is consistent with John's point of view as well, so it sounds like this is the right thing to do.
Dave
On 08.11.2010, at 04:55, David T. Lewis wrote:
On Sun, Nov 07, 2010 at 07:29:56PM -0800, Andreas Raab wrote:
On 11/7/2010 12:27 PM, David T. Lewis wrote:
The above is just my $.02 to get the discussion started. What do others think?
I don't really have any skin in the game since this is a Unix specific issue (i.e., the "squeakvm" dependency) that doesn't really apply on Mac or Windows. That said, I'll add my $.02 by saying that in our Linux production systems we do exactly what you were proposing, i.e., we have fallback VMs installed side-by-side with the latest and symlink the desired VM. That makes it very easy to switch the default *and* allows you to be explicit where you want to (i.e., by executing from the linked-to location).
Well that is Bert's suggestion exactly, and it is consistent with John's point of view as well, so it sounds like this is the right thing to do.
Dave
So we would ship two VMs. One is the old interpreter. The other is cog, or the stack interpreter, depending on the machine's architecture.
The two new ones should be in one source archive, IMHO. At build time either cog or stack VM would be selected depending on the machine's architecture. What should the name be for those binaries? Just use "squeakvm-cog" even if it's the Stack VM because it can run the same images?
Would we drop development on the old interpreter? If plugins were compatible between it and the newer VMs (are they?) it should not be much effort to keep it alive.
What about 64 bit images? We don't need extra sources for those VMs, since it's just a build flag. But we need a naming scheme for these too.
- Bert -
On Mon, Nov 08, 2010 at 12:31:35PM +0100, Bert Freudenberg wrote:
On 08.11.2010, at 04:55, David T. Lewis wrote:
On Sun, Nov 07, 2010 at 07:29:56PM -0800, Andreas Raab wrote:
On 11/7/2010 12:27 PM, David T. Lewis wrote:
The above is just my $.02 to get the discussion started. What do others think?
I don't really have any skin in the game since this is a Unix specific issue (i.e., the "squeakvm" dependency) that doesn't really apply on Mac or Windows. That said, I'll add my $.02 by saying that in our Linux production systems we do exactly what you were proposing, i.e., we have fallback VMs installed side-by-side with the latest and symlink the desired VM. That makes it very easy to switch the default *and* allows you to be explicit where you want to (i.e., by executing from the linked-to location).
Well that is Bert's suggestion exactly, and it is consistent with John's point of view as well, so it sounds like this is the right thing to do.
Dave
So we would ship two VMs. One is the old interpreter. The other is cog, or the stack interpreter, depending on the machine's architecture.
The two new ones should be in one source archive, IMHO.
Yes it should be exactly as you say, but I think we should be cautious about setting expectations. It may take a while to get this accomplished, and in the December time frame the important thing is to deliver solid working Cog and interpreter VMs. If we can merge all the code bases etc that is great, but let's not make it a precondition for the December VM and Squeak 4.2 release cycle. We have all seen what happens when a release team gets caught up in unrealistic development objectives, so let's not go there ;)
At build time either cog or stack VM would be selected depending on the machine's architecture. What should the name be for those binaries? Just use "squeakvm-cog" even if it's the Stack VM because it can run the same images?
Someone recently submitted a magic file with good names (Subbu? I can't find the link). I think the names were squeak-cog, squeak-stack, squeak-interp or somthing similar. In any case, "interp" can refer to the traditional interpreter, "stack" can refer to the portable stack interpreter positioned as a replacement for "interp", and "cog" can refer to the high performance VM.
Would we drop development on the old interpreter? If plugins were compatible between it and the newer VMs (are they?) it should not be much effort to keep it alive.
It's no real trouble to keep it alive and I intend to do so.
The plugins should definitely be identical. The C code generator should also be the same (I'm currently fumbling my ameteurish way through adoption of Eliot's enhancements into VMMaker trunk, but it's obvious that the end result should be the same). Hopefully we will get the old interpreter merged with Eliot's version (more work than I might have expected) but even if we can't do that short term, maintaining the old interpreter is easy and I'm quite happy to continue doing so as needed. Likewise for VMMakerTool, some folks may not need to use it but it is little or no work to maintain it, and I use it myself on a regular basis so keeping it alive is no problem.
What about 64 bit images? We don't need extra sources for those VMs, since it's just a build flag. But we need a naming scheme for these too.
This is something that is only in the trunk VMMaker. I am assuming that it would be good to merge this with the Cog VMMaker, although it is a fair amount of work with no direct benefit to the Cog VM so I should defer to Eliot on this. The changes are straightforward, but a lot of code gets touched and this is an example of something that may take some time and effort to merge.
Dave
On Tuesday 09 Nov 2010 7:00:51 am David T. Lewis wrote:
Someone recently submitted a magic file with good names (Subbu? I can't find the link).
you mean http://lists.squeakfoundation.org/pipermail/vm-dev/2010-October/005619.html
The archive doesn't seem to contain the attachment, so I am attaching it again.
Subbu
On Tue, Nov 09, 2010 at 10:50:31PM +0530, K. K. Subramaniam wrote:
On Tuesday 09 Nov 2010 7:00:51 am David T. Lewis wrote:
Someone recently submitted a magic file with good names (Subbu? I can't find the link).
you mean http://lists.squeakfoundation.org/pipermail/vm-dev/2010-October/005619.html
The archive doesn't seem to contain the attachment, so I am attaching it again.
Subbu
0 leshort 6502 Squeak Image Classic 0 leshort 6504 Squeak Image Standard 0 leshort 6505 Squeak Image with Closure and Float reordering
Ah yes, that's it, thanks!
The image that you run on an Intel box might have been saved on an older Mac with reversed byte ordering, and we may as well throw in the theoretically possible 64-bit variations for completeness, so the magic file entries could be extended like this:
0 lelong 6502 Squeak Image Classic 0 lelong 6504 Squeak Image Standard 0 lelong 6505 Squeak Image with Closure and Float reordering 0 belong 6502 Squeak Image Classic 0 belong 6504 Squeak Image Standard 0 belong 6505 Squeak Image with Closure and Float reordering 0 lelong 68000 Squeak Image 64-bit Classic 0 lelong 68002 Squeak Image 64-bit Standard 0 lelong 68003 Squeak Image 64-bit with Closure and Float reordering 4 belong 68000 Squeak Image 64-bit Classic 4 belong 68002 Squeak Image 64-bit Standard 4 belong 68003 Squeak Image 64-bit with Closure and Float reordering
This works with the original 64-bit image (saved on a Mac) as well as a newer 64-bit image saved on Intel, so I think the rules should be right.
Note, the format number is saved in the first 8 bytes (not 4) for a 64-bit image. A recent Linux supports "lequad" and "bequad", but this is not portable, hence the "belong" with 4-byte offsets in the last three lines.
Dave
On Monday 08 Nov 2010 5:01:35 pm Bert Freudenberg wrote:
The two new ones should be in one source archive, IMHO. At build time either cog or stack VM would be selected depending on the machine's architecture. What should the name be for those binaries? Just use "squeakvm-cog" even if it's the Stack VM because it can run the same images?
For some time, we need to build both VMs and select it at run-time in the launcher script. This shouldn't be a big issue on Linux. See
http://lists.squeakfoundation.org/pipermail/vm-dev/2010-October/005619.html
Subbu
On 09.11.2010, at 18:13, K. K. Subramaniam wrote:
On Monday 08 Nov 2010 5:01:35 pm Bert Freudenberg wrote:
The two new ones should be in one source archive, IMHO. At build time either cog or stack VM would be selected depending on the machine's architecture. What should the name be for those binaries? Just use "squeakvm-cog" even if it's the Stack VM because it can run the same images?
For some time, we need to build both VMs and select it at run-time in the launcher script. This shouldn't be a big issue on Linux. See
http://lists.squeakfoundation.org/pipermail/vm-dev/2010-October/005619.html
That is agreed upon. There will be an interpreter, plus a VM capable of running a cog image.
I was asking how to name that cog/stack vm binary. I don't see a reason to have both cog and stack VMs on one architecture officially. On x86 it would be cog, on everything else stack VM for now. The question is, should that difference be reflected in the binary name? Is it desirable to have both cog and stack VMs installed next to each other? Would people be confused if there is no "cog" binary on their architecture?
- Bert -
On Tuesday 09 Nov 2010 10:50:50 pm Bert Freudenberg wrote:
was asking how to name that cog/stack vm binary. I don't see a reason to have both cog and stack VMs on one architecture officially.
What if Cog VM chokes on existing image files? There has to be fallback. x86 environments are very diverse. It is hard to foresee vm/image combinations.
On x86 it would be cog, on everything else stack VM for now. The question is, should that difference be reflected in the binary name? Is it desirable to have both cog and stack VMs installed next to each other? Would people be confused if there is no "cog" binary on their architecture?
I feel the difference should be reflected in the name, just like in qt3 and qt4.
Subbu
On 09.11.2010, at 19:25, K. K. Subramaniam wrote:
On Tuesday 09 Nov 2010 10:50:50 pm Bert Freudenberg wrote:
was asking how to name that cog/stack vm binary. I don't see a reason to have both cog and stack VMs on one architecture officially.
What if Cog VM chokes on existing image files? There has to be fallback. x86 environments are very diverse. It is hard to foresee vm/image combinations.
Sigh. You're reading selectively ;)
We *are* going to have an interpreter for existing images. This has nothing to do with the stack/cog vm.
On x86 it would be cog, on everything else stack VM for now. The question is, should that difference be reflected in the binary name? Is it desirable to have both cog and stack VMs installed next to each other? Would people be confused if there is no "cog" binary on their architecture?
I feel the difference should be reflected in the name, just like in qt3 and qt4.
Subbu
That is certainly one option. "squeak4" would be the interpreter and "squeak5" would be stack/cog. John's Mac VMs already use that scheme.
- Bert -
Just thinking...
It would be really fine if Cog allowed forks as defined in OSProcess/CommandShell (like forkHeadlessSqueakAndDoThenQuit: )
CdAB
On Tue, Nov 9, 2010 at 11:54 AM, Bert Freudenberg bert@freudenbergs.de wrote:
Sigh. You're reading selectively ;)
We *are* going to have an interpreter for existing images. This has nothing to do with the stack/cog vm.
To put it another way, there may well be need to have both Cog and Stack VMs installed alongside each other, even on x86. It seems superfluous, but why make that scenario difficult? I say the previous suggestion makes the most sense:
squeakvm-cog squeakvm-stack squeakvm-interp.
If we're planning to maintain these three variations going forward, we shouldn't use version numbers to differentiate between them. Someday we'll be releasing version 6.0 of the interpreter, and it makes no sense to have that binary be called squeak4.
Colin
On Tue, Nov 09, 2010 at 08:54:09PM +0100, Bert Freudenberg wrote:
On 09.11.2010, at 19:25, K. K. Subramaniam wrote:
On x86 it would be cog, on everything else stack VM for now. The question is, should that difference be reflected in the binary name? Is it desirable to have both cog and stack VMs installed next to each other? Would people be confused if there is no "cog" binary on their architecture?
I feel the difference should be reflected in the name, just like in qt3 and qt4.
That is certainly one option. "squeak4" would be the interpreter and "squeak5" would be stack/cog. John's Mac VMs already use that scheme.
I notice that that command name "turbo" seems to be unclaimed on most unix boxes. We could name the cog version /usr/local/bin/turbo, so that when you want your squeak.image to go faster you would just enter:
$ turbo squeak
;-)
Dave
vm-dev@lists.squeakfoundation.org
-
Andreas Raab -
Bert Freudenberg -
Casimiro de Almeida Barreto -
Colin Putney -
David T. Lewis -
Igor Stasenko -
John M McIntosh -
K. K. Subramaniam