Vm-dev March 2009

vm-dev@lists.squeakfoundation.org

21 participants
29 discussions

Re: [Pharo-project] [Vm-dev] newDelta >0 failed in sqUnixMemory -- Was Re: Incompatibility between Pharo and Seasidehosting
by Bruce O'Neel 04 Mar '09

04 Mar '09

Hi, There seem to be three solutions (in best to worst order). - someone (Ian maybe) needs to rebuild the Unix VM with a VMMaker from within the last two years. Apparently the bug has been fixed for a while. - pass a fixed memory size to the VM -memory 200m or so does the trick. - patch the existing C source code. cheers bruce On Tue, Mar 03, 2009 at 09:19:13AM +0100, Tudor Girba wrote: > Hi, > > I am not sure I got the summary of the discussion related to the > problem of deploying a Pharo-based image to seasidehosting. Is there > any conclusion? > > Cheers, > Doru > > > -- > www.tudorgirba.com > > "Problem solving efficiency grows with the abstractness level of > problem understanding." > > > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

1 0

Re: Alien and FFI integration
by Eliot Miranda 04 Mar '09

04 Mar '09

Hi Andreas, what I really think about this is that call-outs are the weakest part of Alien. The callout facilities are essentially a hack that works on IA32, could work on ARM or PowerPC at a pinch, and will fail horribly in general on x86-64 (um EMT64 & AMD64). e.g. the AMD64 calling convention splits struct contents across available registers as needed so e.g. a struct with one float, four longs and a float will get passed in fp arg reg 0 & fp arg reg 1 (for first and last fields) and in int arg regs 0 through 3 (for the second through the fifth fields). Yes, one can ask the programmer to work this out and use a low level interface but one will get rightly a rude answer to the question. But the only real answer is to keep call specs and have the image compile them into something that matches the ABI. For a while when David Simmons and I have discussed this we have agreed that the right way to do it is have an ABI compiler framework in the image whose job it is to compile code to make the callout on first use. When we've discussed it we've suggested that the ABI compiler produces some kind of RTL language that the JOT consumes and turns into proper machine code to actually make the callout. But as you noted on Monday, the ABI compiler could actually produce machine code. Alien already does this for producing the callback thunks for callback support. The only issues here are how to plumb in the machine code to the machine code the JIT generates for methods and (for me) how to avoid the ABI compiler mushrooming into a full blown native compiler with too much duplication. On Sun, Mar 1, 2009 at 11:56 PM, Andreas Raab <andreas.raab(a)qwaq.com> wrote: > Hi Eliot - > > [cc: vm-dev to see if anyone has some good ideas] I was thinking about how > one would move between FFI and Alien more freely and one thing that seemed > interesting is to have the compiler simply generate Alien call code for FFI > specs. E.g., when the compiler encounters, say: > > system: aString > <apicall: ulong 'system' (char*) module: 'libc'> > ^self externalCallFailed > > it would generate something like here: > > system: aString > AlienCall ifNil:[ > "Register this method to be flushed on shutdown" > Alien registerMethod: thisContext method. > "Lookup the call if it isn't cached" > AlienCall := Alien lookup: 'system' inLibrary: 'libc'. > "Result can use a cached Alien since it never changes" > AlienResult := Alien new: 4. > ]. > "Arg1 needs to be reloaded every time we call" > Arg1 := Alien fromString: aString. > AlienCall primFFICallResult: AlienResult with: Arg1. > Arg1 free. "or however one frees those" > "XXXX: Would need some success indication here" > success ifFalse:[^AlienResult asUnsignedLong]. > > "Non-primitive code follows here" > ^self externalCallFailed "aka: (thisContext sender method literalAt: 1) > reportLastError" > > The "globals" in the above (AlienCall, AlienResult, AlienArg1) are all > method literals to cache the information (have you ever considered adding a > <static: foo> directive to the compiler to mark temp foo as a temp stored in > the literal frame?). > > There are two advantages and a major disadvantage to this approach. The > good news is that this would allow one to continue to use existing FFI specs > with Alien (i.e., ease migration). Plus, some optimizations (like caching > lookups and args) can be done by the compiler now and don't require people > to write code themselves. > > The obvious disadvantage is that you need to change the code to adjust for > changes in the ABI. Which sucks. I really wish Alien could take a descriptor > and deal with the ABI details itself. I wonder if it's feasible to have > subclasses of Alien function calls that implement the different ABIs and > pass enough information along that the ABI dependent parts could be done > there? > > Cheers, > - Andreas > >

1 0

urgent info required on Slang's shift treatment...
by Eliot Miranda 03 Mar '09

03 Mar '09

Hi All, I'm being bitten by Slang's treatment of bitShift: & >>. In both cases (generateBitShift:on:indent: & generateShiftRight:on:indent:) Slang generates an unsigned shift by explicitly casting the shifted expression to usqInt. I can understand the benefit of having an unsigned shift. But there are times when one really needs a signed shift. Further, the Smalltalk versions of both bitShift: and >> are signed shifts. Dare I change e.g. generateShiftRight:on:indent: to leave the expression alone and generate either a signed or an unsigned shift based on the variable's declaration? Or must I live with a maddening cCode: '(signed)' inSmalltalk: [] carbuncle? E.

6 17

Re: [Vm-dev] Re: urgent info required on Slang's shift treatment...
by Eliot Miranda 03 Mar '09

03 Mar '09

On Tue, Mar 3, 2009 at 1:09 PM, Ken G. Brown <kbrown(a)mac.com> wrote: > > At 9:39 PM +0100 3/3/09, Bert Freudenberg apparently wrote: > >On 03.03.2009, at 21:36, Eliot Miranda wrote: > > > >>Come on original Slang developers. 'Fess up. I need to know what your > thought process was in making shifts unsigned so that I can change it > without too much hard work. Pretty please. > > > > > >I don't think you will reach them on this list. > > > >- Bert - > > Perhaps this could shed some light: > Extending the Squeak Virtual Machine > Andrew C. Greenberg > < > http://www.iam.unibe.ch/~ducasse/FreeBooks/CollectiveNBlueBook/greenberg.pdf > > No, but thanks for the pointer. "Andrew C. Greenberg" <werdna(a)mucow.com> (valid in May 2007) > > Ken G. Brown >

1 0

Push/pop considered harmful
by Andreas Raab 03 Mar '09

03 Mar '09

Folks - Ever since we started to use the Stack VM we had a series of subtle crashes, many of which we could trace to stack imbalances (i.e., mismatching numbers of push/pops in primitives). Since the Stack VM is much more affected by this we have been looking to fix these issues once and for all. One thing that occurred to us is that practically all primitives do the same: They pop the number of arguments and they push an optional return value. There are many ways in which this can get wrong, sometimes it's a subtle early return, sometimes the fact that the primitive has actually failed before the prim returns etc. One thing that all of the uses (in plugins) have in common that all the plugin *really* needs to do, is to provide a return value and leave the push/pop to be done by the VM. In this case the stack invariants wouldn't even be exposed to the plugin. What we are doing right now is along those lines - basically we are replacing push/pop in the interpreter proxy by variants that don't actually do push and pop. Rather than that, pop is ignored (it only remembers how many times you popped so we can track inconsistencies) and push remembers the return value. This should definitely be replaced at some point by a proper #primitiveReturnValue: call. I was wondering what people think about the possibility of automatically rewriting plugins to fix those uses? It should be reasonably easy to find all the uses of "interpreterProxy pop: foo" and just remove them and replace "interpreterProxy push[Float|Int]:" with "interpreterProxy primitiveReturn[Float|Int]:". Cheers, - Andreas

5 13

Alien and FFI integration
by Andreas Raab 02 Mar '09

02 Mar '09

[Sorry for posting this again; I used the wrong email address the first time] Hi Eliot - [cc: vm-dev to see if anyone has some good ideas] I was thinking about how one would move between FFI and Alien more freely and one thing that seemed interesting is to have the compiler simply generate Alien call code for FFI specs. E.g., when the compiler encounters, say: system: aString <apicall: ulong 'system' (char*) module: 'libc'> ^self externalCallFailed it would generate something like here: system: aString AlienCall ifNil:[ "Register this method to be flushed on shutdown" Alien registerMethod: thisContext method. "Lookup the call if it isn't cached" AlienCall := Alien lookup: 'system' inLibrary: 'libc'. "Result can use a cached Alien since it never changes" AlienResult := Alien new: 4. ]. "Arg1 needs to be reloaded every time we call" Arg1 := Alien fromString: aString. AlienCall primFFICallResult: AlienResult with: Arg1. Arg1 free. "or however one frees those" "XXXX: Would need some success indication here" success ifFalse:[^AlienResult asUnsignedLong]. "Non-primitive code follows here" ^self externalCallFailed "aka: (thisContext sender method literalAt: 1) reportLastError" The "globals" in the above (AlienCall, AlienResult, AlienArg1) are all method literals to cache the information (have you ever considered adding a <static: foo> directive to the compiler to mark temp foo as a temp stored in the literal frame?). There are two advantages and a major disadvantage to this approach. The good news is that this would allow one to continue to use existing FFI specs with Alien (i.e., ease migration). Plus, some optimizations (like caching lookups and args) can be done by the compiler now and don't require people to write code themselves. The obvious disadvantage is that you need to change the code to adjust for changes in the ABI. Which sucks. I really wish Alien could take a descriptor and deal with the ABI details itself. I wonder if it's feasible to have subclasses of Alien function calls that implement the different ABIs and pass enough information along that the ABI dependent parts could be done there? Cheers, - Andreas

1 0

[Fwd: Alien and FFI integration]
by Andreas Raab 02 Mar '09

02 Mar '09

Hi Eliot - [cc: vm-dev to see if anyone has some good ideas] I was thinking about how one would move between FFI and Alien more freely and one thing that seemed interesting is to have the compiler simply generate Alien call code for FFI specs. E.g., when the compiler encounters, say: system: aString <apicall: ulong 'system' (char*) module: 'libc'> ^self externalCallFailed it would generate something like here: system: aString AlienCall ifNil:[ "Register this method to be flushed on shutdown" Alien registerMethod: thisContext method. "Lookup the call if it isn't cached" AlienCall := Alien lookup: 'system' inLibrary: 'libc'. "Result can use a cached Alien since it never changes" AlienResult := Alien new: 4. ]. "Arg1 needs to be reloaded every time we call" Arg1 := Alien fromString: aString. AlienCall primFFICallResult: AlienResult with: Arg1. Arg1 free. "or however one frees those" "XXXX: Would need some success indication here" success ifFalse:[^AlienResult asUnsignedLong]. "Non-primitive code follows here" ^self externalCallFailed "aka: (thisContext sender method literalAt: 1) reportLastError" The "globals" in the above (AlienCall, AlienResult, AlienArg1) are all method literals to cache the information (have you ever considered adding a <static: foo> directive to the compiler to mark temp foo as a temp stored in the literal frame?). There are two advantages and a major disadvantage to this approach. The good news is that this would allow one to continue to use existing FFI specs with Alien (i.e., ease migration). Plus, some optimizations (like caching lookups and args) can be done by the compiler now and don't require people to write code themselves. The obvious disadvantage is that you need to change the code to adjust for changes in the ABI. Which sucks. I really wish Alien could take a descriptor and deal with the ABI details itself. I wonder if it's feasible to have subclasses of Alien function calls that implement the different ABIs and pass enough information along that the ABI dependent parts could be done there? Cheers, - Andreas

1 0

Problem building VMMaker-dtl.116 on Windows
by bryce＠kampjes.demon.co.uk 01 Mar '09

01 Mar '09

The method Interpreter>>primitiveBeCursor was modified with the signature: "eem 2/18/2009 09:01" to call ioSetCursorARGB which doesn't exist in the latest Windows source files. I worked around it by reverting primitiveBeCursor to the version "di 7/4/2004 08:41". Bryce

5 5

Virtualization effects on the Squeak VM?
by Andreas Raab 01 Mar '09

01 Mar '09

Hi - In some of my recent load testing I noticed some interesting behavior when running Squeak (specifically the Unix VM) in a virtualized environment. I don't know if anyone else does that but I'd be curious in sharing information on that. First the setting: The measures I've taken so far was running RHEL 5.2 on VMware Player and Server hosted under Windows and compare this to a bare metal server (ESX[i] is on the list for next week). BTW, for those of you raising their eyebrows about running under Windows this setup is purely driven by customer demand so we're just trying to provide people with realistic numbers about what they can expect. So far the most interesting artifacts were the following: 1) Time millisecondClockValue and its use of gettimeofday. It turns out that in our settings, Time millisecondClockValue takes about 10% total execution time (we use it for measuring throughput so it does get called *a lot*). We will work around this by adding a low-res clock that gets updated without calling gettimeofday but it was interesting to notice how much time goes into that. 2) Socket write calls. My recent comments about socket write behavior turned out to be a virtualization effect. Running the benchmark both virtualized and non-virtualized shows a difference of 3-4 in how much time is spent in socket writes (30% virtualized vs. 8% bare metal). Keep in mind that this is running under load so we're making roughly 50k socket write requests per second with a total bandwidth of 10-20 Mbps. Has anyone seen similar effects and can confirm them? Are there other gotchas that people have observed when running virtualized servers? I suspect that the socket write issue is really the result of VMware interfacing with Windows. If anyone has seen similar effects and can share some insights, I'd appreciate it. Cheers, - Andreas

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev March 2009