Overview of the OpenSmalltalk VM and its build process - Vm-dev

17 Oct 2017

2017-10-17 16:00 GMT+02:00 Clément Bera <bera.clement@gmail.com
<mailto:bera.clement@gmail.com>>:

    Hi Raffaello,

    Answers inlined...

    On Tue, Oct 17, 2017 at 3:07 PM, Raffaello Giulietti
    <raffaello.giulietti@lifeware.ch
    <mailto:raffaello.giulietti@lifeware.ch>> wrote:

        Hi,

        I'm interested in understanding the overall OpenSmalltalk VM
        structure and design and its build process.

        I read [1] and [2] to get a high level picture of the vm and
        its interesting strategy to development (Slang, VMMaker, etc.)

        Apart from these, are there other useful
        documents/sites/tutorials on the vm development process in
        general and about the underlying ideas in the build process?
        I'm aware that vm construction is a rather special topic but
        any useful reference or hint is welcome.

        I'm already familiar with the interesting [3] and [4]
        websites and their authors papers, talks and slides, but
        these do not seem to help much in clarifying the development
        process itself.

        In particular, I'm targeting Pharo 64 bit/Win 64 bit
        (x64-86). I hope to be able someday to actively participate
        in bringing Cog/Spur/Sista to a more mature status on this
        platform.

        The questions I would like to find an answer to are of the
        following kind:
        * Where should I start?

    A good start would be to load the Squeak VMMaker dev image (See
    here: http://www.mirandabanda.org/cogblog/build-image/
    <http://www.mirandabanda.org/cogblog/build-image/>) and read the
    different opened workspaces. In addition, ... 

    *Compilation toolchain*

    To check you have everything installed to compile, you can go to
    the one of the build folder (One of the windows build folder if
    you're on windows, for example build.Win64x64, in most case we
    don't recommend cross compilation), read the HowToBuild file,
    then go to one of the inner folder and use the mvm script to
    compile a VM from pre-generated C sources.

See below though, on windows, we target mingw so it's kind of
cross-compilation anyway.

    If it does not work, check on the openSmalltalk-VM repo how the
    VM is built using Travis CI and set-up your machine similarly to
    the Travis slaves.

    *Changes in Slang (GC/Interpreter/JIT/...)*

    In general when you want to change something in the GC /
    Interpreter the process is as follow:

    1) Make your changes work with the StackVM (interpreter-only)
    simulator
    2) Make your changes work with the compiled StackVM
    (interpreter-only) 
    3) Make your changes work with the CogVM simulator
    4) Make your changes work with the compiled CogVM

    In general when you want to change something in the JIT the
    process is as follow:

    1) Make your changes generate correct in-image machine code
    2) Make your changes work with the CogVM simulator
    3) Make your changes work with the compiled CogVM

    So a good place to start is to have the simulators working. You
    could use this to help:
    https://clementbera.wordpress.com/2016/05/30/simulating-the-cog-vm/
    <https://clementbera.wordpress.com/2016/05/30/simulating-the-cog-vm/>

    Once the simulator works with your changes, you can generate C
    sources with something like "VMMaker generateConfiguration"

    *Changes in Platform code*
    *
    *
    Go to the platform folder at the root and change the C files
    with a text editor

        * Should I use Squaek or can I use Pharo for vm development
        purposes?

    Currently I still use Squeak but I plan to migrate to Pharo. I'd
    say it's still a bit easier from Squeak due to package
    compatibility making it hard to commit from Pharo without
    uncommitting something else, some scripts are also present by
    default in the Squeak dev image, but everything should work on
    both environments. No big deal in using one of the other.

        * What are the restrictions of Slang?

    ...
    As long as the VM compiles and run, this is correct Slang code,
    else it's not.
    In general you should just think in C and write Smalltalk instead. 
    I don't know what to say. This is not dynamic at all. No
    dictionaries, no look-up, no nothing. Even arrays look like
    Smalltalk arrays but like C arrays you don't have the actual size.
    Blocks work as long as they're inlined and removed when
    performing inlining in the Slang to C compiler.

        * Where does VMMaker live?

    Half there:
    https://github.com/OpenSmalltalk/opensmalltalk-vm
    <https://github.com/OpenSmalltalk/opensmalltalk-vm>
    Accessible from git

    Half there:
    http://source.squeak.org/VMMaker/
    <http://source.squeak.org/VMMaker/>
    Accessible from Monticello

        * Which C toolchain shall I use? 

        * Which release of gcc, ld, make are known to work?
        * What about clang?

    Depends on the platform. The mvm script normally chooses for you. 

    All VMs are built from Travis CI so you can check how to set up
    your machine from the Travis set-up.

    On linux it's gcc by default. All recent gcc versions should
    work since 4.4
    On Mac, gcc is not supported anymore, so clang is used. I've
    used many different versions of clang and they all worked.

    Specific compiler versions that don't work are mentioned in the
    HowToBuild file in the platform folders (if any).

        * Which version of cygwin64 or mingw64 are OK?

    Arf. Both worked at some point, I don't know what the default
    script use right now I would start with the version used by the
    mvm script / the Travis build.

gcc is broken for win64 (or at least not satisfying our expectations
about stack management)
So we have to use clang.
For this reason, we prefer to compile thru cygwin which has prebuilt
(downlodable) clang modules.
It's much more difficult to get a working clang in a mingw
environment, so this brand is not maintained anymore.