Mark Guzdial wrote:

I have been reading the "Optimizing Squeak" thread with great interest, though not always with great understanding :-) Playing with different VM architectures and alternative bytecodes sounds like enormous fun and a really powerful student project.

How does one actually move to a new bytecode set? What is the path by which you say "Okay, recompile all the code in the system using my new compiler in order to create a new image for my new VM"? The Digitalk people had to do this at least once, didn't they? Or do new bytecode sets typically retain some backward compatibility while bootstrapping?

I don't think that Digitalk did this, since their image was not a descendant of the Smalltalk-80 image. Probably the most reasonable way of doing it would be to make a bytecode set converter which could convert old bytecodes into new bytecodes, and integrate that with the SystemTracer to write out an image where all CompiledMethods consist of new bytecodes. Of course, you would need to update the pc inst vars in all Contexts to match the new position of their respective bytecodes... ObjectShare did an image conversion like this when they moved from the old bytecode set (which Squeak still uses) to the HPS bytecodes which are used in the VisualWorks products until today. The HPS bytecodes have been designed for JIT compilation, and the ranges of various opcode fields have been adjusted to better match the actual use in Smalltalk images. However, I don't know whether they used something like the SystemTracer or an external image converter tool. I vaguely remember talking with Peter Deutsch about the topic, but don't know anymore what his answer was...

Hans-Martin

I have been reading the "Optimizing Squeak" thread with great interest, though not always with great understanding :-) Playing with different VM architectures and alternative bytecodes sounds like enormous fun and a really powerful student project.

How does one actually move to a new bytecode set? What is the path by which you say "Okay, recompile all the code in the system using my new compiler in order to create a new image for my new VM"? The Digitalk people had to do this at least once, didn't they? Or do new bytecode sets typically retain some backward compatibility while bootstrapping?

Mark -

There are a couple of ways to do this. If there is room for the new bytecode(s) with the old, you build a new VM that supports the new, change the compiler to compile it, and recompile all methods in the system. I did this when making the extensions so Squeak could address 256 literals in a method and 256 named instvars in an object. If this frees up the old bytecode(s), then you can remove the code for the old bytecode from the VM after the recompilation.

For more radical changes, we use the SystemTracer. This writes a clone of a running (!) image onto the disk. Since it is just writing a file it can make arbitrary transformations on the way to disk. This is how we changed Squeak from an object-table to the current direct-pointer object memory format. More recently Tim Rowledge's change to the format of compiled methods was done this way. Having written the new image, it's a small matter of rewriting and debugging the VM until can run the new image.

- Dan