Vm-dev August 2009

vm-dev@lists.squeakfoundation.org

27 participants
87 discussions

svn/squeak/trunk/platforms/unix/config file format problems
by Andreas Wacknitz 08 Aug '09

08 Aug '09

Hi all, With the last changes there seem to be a lot of files in DOS file format. That prevents successful configure runs as the scripts aren't executable under Solaris anymore. Regards Andreas

1 0

Fwd: [Pharo-project] Compiling a Squeak VM for Pharo on Ubuntu 8.04 LTS Server (64bit)
by Damien Cassou 08 Aug '09

08 Aug '09

---------- Forwarded message ---------- From: Dave Woodward <dave(a)futuremint.com> Date: Thu, Aug 6, 2009 at 7:10 PM Subject: [Pharo-project] Compiling a Squeak VM for Pharo on Ubuntu 8.04 LTS Server (64bit) To: pharo-project(a)lists.gforge.inria.fr I tried using the pre-compiled VM from the Pharo website to no avail. I'm not exactly sure what the problem is, but when running the pre-compiled Pharo VM, OR the pre-compiled Squeak 3.10-6 VM, they just quit with a "File not found" error. My only hunch regarding this is that they were compiled on 32bit Linuxes, and I'm using a 64bit Linux. I'm using a bare-bones install of Ubuntu server, 8.04LTS 64bit. I'm using Slicehost.com for a virtual server to run a Seaside application (they use all 64bit *NIXs), and I'm switching from vanilla Squeak 3.9 to Pharo. So, since the Squeak 3.10-6 VM didn't work either I decided to compile my own VM. This does not work out of the box on a vanilla Ubuntu Server 8.04 install, so here are the steps I took to successfully compile the VM from the 3.10-6 src files: (cd /Squeak-3.10-6/platforms/unix) Configure: ./config/configure --without-x --without-gl --without-quartz --without-npsqueak NOTE: The configure command adds 2 entries to disable the RomePlugin. It looks like it first tests Freetype (not installed so adds an entry to disable RomePlugin), and then tests Cairo (not installed, so adds *another* entry to disable RomePlugin). I believe the autoconf files need to be fixed for this, but I leave it up to the squeak-dev folks to figure that out as I have no knowledge of autoconf. Make: make squeak (this will error) To fix the damage from the note above, I had to remove the duplicate entry for RomePlugin_exports on line 22 of disabledPlugins.c. Make: make squeak (should be successful) make squeak install move /usr/local/lib/squeak/3.10-6/UUIDPlugin to UUIDPlugin.bak as it has the wrong ELF header, and I've heard has other problems too. And now /usr/local/bin/squeak should work! (note, without X you'll need to add "-vm display=none" for the VM to start). I hope this helps people! Thanks for everybody's hard work on Pharo! Dave Woodward ................................................... www.futuremint.com 317.332.6438 | dave(a)futuremint.com _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- Damien Cassou http://damiencassou.seasidehosting.st "Lambdas are relegated to relative obscurity until Java makes them popular by not having them." James Iry

3 2

rearchitecting the FFI implementation for reentrancy
by Eliot Miranda 07 Aug '09

07 Aug '09

Hi All, I'm looking at making the Squeak FFI reentrant to support nested calls and possibly threading. The current FFI has a couple of issues which render it non-reentrant. 1. The FFI plugin has a number of static state variables which convey information from the pre-call side of a call-out to the return-side of a call-out. These include ffiRetOop, ffiRetClass (oops), ffiRetSpec (IIUC a pointer into an object body, the result box for a struct result), ffiRetSpecSize (the size of the struct result box), etc. Since these variables are set-up before a call-out and used once the call-out has returned, any nested call-out that happens within a call-back from the call-out will overwrite these variables. Further, any GC that occurs during a call-back can move the struct result box object, rendering the ffiRetSpec pointer invalid, and likely resulting in heap corruption. 2. The platform support code for argument marshalling has static areas in which to marshall the outgoing arguments (which is fine since they're only used on the way out). But there are also static areas used to hold copies of string arguments. These are malloced, stored in a static array of pointers, and freed on the return side by ffiCleanup. Any nested callout will either overwrite the outer call's strings and/or prematurely free the string copies when the nested call-outdoes an ffiCleanup. Making the variables in 1. reentrant is straight-forward. All state is derived from the external call spec in an external callout method (primititive 120 method). The callback machinery is careful to restore newMethod to its state on return form a call-back. So the state can be refetched from newMethod on the return side of the call-out. This also avoids having to update ffiRetOop & ffiRetClass in any GC, and allows them to be kept as local variables of the call-out side. Making 2. reentrant is a little more involved, which is why I wanted to raise this on the list. here is a sketch of a generic solution that I want y'all to sanity-check for different architectures with which you're familiar. The basic idea is to use alloca to stack allocate all non-movable state for a particular call-out, using stack discipline to reclaim it automatically when the FFI call-out primitive returns. The alloca'ed state comprises three regions. In the quasi-diagram below the stack grows down. region 1, space for any copied strings and a temporary result for structure returns region 2, an array of pointers, one for each argument, each pointing to the start of its corresponding argument further on in the alloca'ed space region 3, the marshalled arguments, ready for call-out The call-out machinery can either make a pass over the arguments before marshalling to compute the size of the alloca'ed region, or it can guestimate, based on e.g. a precomputing of the size of any struct arguments. One question is whether marshaling can be done generically in the FFI plugin based on e.g. functions such as alignStruct, alignDouble, alignLongLong implemented in the platform to tell the generic code how to align elements greater than a word in size, or whether it would be better to put the code in the platform. I'm guessing the alignStruct approach would work fine and could be implemented with macros. An important question is whether the array-of-pointers scheme will allow platforms that pass values in registers to locate any and all arguments. Perhaps some platforms will require a separate array of pointers to floating-point values? Please speak up if you know of any issues here. Finally I'm assuming that to actually make the call once the arguments have been marshalled some assembler stub (perhaps one for each argument count up to the number of register arguments times the number of basic calling conventions, integer/pointer result, double, struct result) can be called which takes pointers to the three regions, and the function to be called, cuts back the stack to the top of the alloca'ed area, putting the return address in the right place, and tail-calls/jumps to the function. Does this sound sane? I'm sure it'll work on a number of platforms I'm familiar with (which e.g. does not include iPhone). I'm sure it won't work for x86-64 structures that partytially fit within registers, but the current implementation is broken for those anyway. I've used alloca successfully in the VW FFI but there were different implementations of marshalling for each ABI. Can you say either way whether you think this would work for particular platforms you're familiar with? TIA Eliot

5 5

Cog status & FFI directions [was rearchitecting the FFI implementation for reentrancy]
by Eliot Miranda 07 Aug '09

07 Aug '09

2009/8/6 Göran Krampe <goran(a)krampe.se> > > Hi Eliot and all! > > Eliot Miranda wrote: > >> Hi All, >> I'm looking at making the Squeak FFI reentrant to support nested calls >> and possibly threading. The current FFI has a couple of issues which >> render >> it non-reentrant. >> > > The tech stuff is over my head, but I do have three questions related to > this: > > 1. What about Alien? Shouldn't we try to move towards Alien instead of > current FFI? Or is that too much work at this point? I intend to merge Alien into the current FFI to allow the current FFI to marshal Aliens. Aliens are fine for modelling external data but the Alien FFI call-out mechanism is a little too naive for general use It works well on x86 but has issues on anything with an exotic calling convention (passes arguments in integer and/or floating-point registers). And see the next point about callbacks. > > > 2. Callbacks has been a sore point in Squeak for a long time. AFAIK there > is a patch available on www.squeakgtk.org/wiki/download.html, not sure > what it does or if it is the original patch from Andreas when wxSqueak was > being built. wxSqueak had a patched VM I recall. Perhaps that stuff is not > related. > One thing that IMO is much better about Alien is the callback mechanism which allows one effectively to pass function pointers to blocks. The current FFI's callback mechanism is weak. It simply does a process switch away form the process calling out and requires further work in the image, e.g. a process waiting on a semaphore that is signalled by external code, to then collect information for performing the callback. So adding in the Alien callback mechanism is also something I intend to do. > 3. Could we possibly ask for a status update on Cog and related activities? > We are itching for news! :) Also curious about your interest in Factor and > its lower bowels (definitely cool stuff going on there). The status is as follows. The Cog stack VM os being reviewed for release to the community. We hope to have this done soon, certainly before the end of September, but we're busy and this isn't on the critical path. Once it is released there will have to be some integration and merge activities before it is part of the standard VMs because we have effectively forked (although not a lot). The first incarnation of the Cog JIT is complete (for x86 only) and in use at Qwaq. We are gearing up for a new server release and the Cog VM is the Vm beneath it. The next client release will include it also. This VM has a naive code generator (every push or pop in the bytecode results in a push or pop in machine code) but good inline cacheing. Performance is as high as 5x the current interpreter for certain computer-language-shootout benchmarks. The naive code generator means there is poor loop performance (1 to: n do: ... style code can be 4 times slower than VisualWorks) and the object model means there is no machine code instance creation and no machine code at:put: primitive. But send performance is good and block activation almost as fast as VisualWorks. In our real-world experience we were last week able to run almost three times as many Qwaq Forums clients against a QF server running on the Cog VM than we were able to above the interpreters. So the Cog JIT is providing significant speedups in real-world use. I am (clearly) looking at FFI issues right now. In the Autumn I intend to start work on a less naive code generator, a better object model and a faster garbage collector, the three of which should raise performance levels to VisualWorks levels, i.e. a further 2x to 3x increase over the 4x - 5x already achieved for pure Smalltalk execution. I expect we'll be in a position to release some version of the Cog JIT to the community by Christmas. I'll be giving a guided tour of the current Cog JIT VM at SqueakFest LA on Monday. > > regards, Göran Best Eliot

3 4

VMMakerTool change '64 bit VM?' selection box label and balloon help
by David T. Lewis 05 Aug '09

05 Aug '09

The '64 bit VM?' selection box in VMMakerTool can be confusing for first-time VM builders. See for example: http://lists.squeakfoundation.org/pipermail/squeak-dev/2009-August/138006.h…. For this reason I have changed the label on the selection box from '64 bit VM?' to '64-bit word', and the balloon help from 'Set to build a 32 bit VM or a 64 bit VM' to 'Build (32- or 64-bit) VM to run 64-bit images. Default is false to support normal 32-bit images.'. The change is in VMMaker-dtl.129 on SqueakSource, and will be in effect for newly opened VMMakerTool windows. Dave

1 0

Resources directory in SecurityPlugin
by Bert Freudenberg 04 Aug '09

04 Aug '09

When running Etoys inside a web browser, the file sandbox is preventing access to all files outside the "Etoys" working directory (a.k.a. "untrusted user directory"). That means we cannot open help guides or switch languages or access examples or the sources file etc. Proposed solution: In the SecurityPlugin, allow read-only file to a "resource" directory (in addition to read-write access to the "untrusted user directory"). The default for this resource directory would be the image directory. I did it for Unix, you can find a diff attached to the Etoys ticket: http://tracker.squeakland.org/browse/SQ-241 Basically, the writeFlag is now passed down to isAccessiblePathName(), and in case it's read-only, it allows access to the resource dir (which is configurable but defaulting to the image path). Any chance to get the same behavior implemented on Mac and Win VMs, too? Thanks :) - Bert -

2 2

patch: fix double alignment detection on Solaris/SPARC with gcc-4.4.x
by Andrew Gaylard 01 Aug '09

01 Aug '09

On Mon, Jul 20, 2009 at 7:02 PM, David T. Lewis<lewis(a)mail.msen.com> wrote: > > On Sat, Jul 18, 2009 at 05:25:48PM +0200, Andrew Gaylard wrote: >> >> The main issue that I'm facing at the moment is that gcc-4.4.0 generates >> code that causes unaligned accesses in fetchFloatAtinto; the same code >> works in gcc-4.2.4 on SPARC. Until I or someone else (hint!) fixes this, >> I'm continuing to build with gcc-4.2.4. > > fetchFloatAtInto() is implemented in platforms/Cross/vm/sqMemoryAccess.h > in the form of cpp macros. It might be tricky to debug this, but I note > that the implementation depends on DOUBLE_WORD_ALIGNMENT, which is a macro > that would have been set when you run configure. You might playing with > this setting to see if it gives you the results you want. For example, as > a complete SWAG, it might be the case that SPARC plus gcc-4.2.4 needs this > macro defined but for some reason configure is not setting it for you. David, thanks for the tip. This message from configure showed you are right (the 'yes' should be 'no'): checking whether unaligned access to doubles is ok... yes It turns out that newer GCCs, from 4.3.x onward, are smart enough to optimise out the code used to detect strict alignment: bash-3.2$ gcc -O2 double-test.c double-test.c: In function 'main': double-test.c:6: warning: passing argument 1 of 'f' makes integer from pointer without a cast double-test.c:1: note: expected 'int' but argument is of type 'char *' bash-3.2$ ./a.out bash-3.2$ echo $? 0 But when compiling without optimisation: bash-3.2$ gcc double-test.c double-test.c: In function 'main': double-test.c:6: warning: passing argument 1 of 'f' makes integer from pointer without a cast double-test.c:1: note: expected 'int' but argument is of type 'char *' bash-3.2$ ./a.out Bus Error (core dumped) ... as expected. This patch complicates the code sufficiently (by using the results of the function) so that it triggers a bus error both with and without optimisation, as would be expected. Of course, configure needs to be regenerated after applying this patch. Running our internal benchmark reveals that it is worth it getting 4.4.1 to work: gcc-4.4.1: real 61m45.975s user 56m16.247s sys 4m57.253s gcc-4.2.4: real 67m29.260s user 61m40.666s sys 5m7.937s This means the VM is sped up by 9%. - Andrew --- Squeak-3.10-4/platforms/unix/config/acinclude.m4.orig 2008-09-02 20:49:43.000000000 +0200 +++ Squeak-3.10-4/platforms/unix/config/acinclude.m4 2009-07-30 16:16:55.828929831 +0200 @@ -230,10 +230,33 @@ AC_DEFUN([AC_C_DOUBLE_ALIGNMENT], [AC_CACHE_CHECK([whether unaligned access to doubles is ok], ac_cv_double_align, - AC_TRY_RUN([f(int i){*(double *)i=*(double *)(i+4);} - int main(){char b[[12]];f(b);return 0;}], + AC_TRY_RUN([ + +int f(void* i){ + *(double *)i=*(double *)(i+4); + return *(char*)i; +} +int main(){ + char b[[12]]; + b[[0]]=1; + b[[1]]=2; + b[[2]]=3; + b[[3]]=4; + b[[4]]=0; + b[[5]]=0; + b[[6]]=0; + b[[7]]=0; + b[[8]]=0; + b[[9]]=0; + b[[10]]=0; + b[[11]]=0; + + return f(b); +} +], + ac_cv_double_align="yes", ac_cv_double_align="no")) test "$ac_cv_double_align" = "no" && AC_DEFINE(DOUBLE_WORD_ALIGNMENT)]) AC_DEFUN([AC_C_DOUBLE_ORDER],

3 5

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev August 2009