Vm-dev March 2011

vm-dev@lists.squeakfoundation.org

43 participants
133 discussions

testing MT and reentrant
by Esteban Lorenzano 25 Mar '11

25 Mar '11

Hi, I'm trying to build latest oscog... and, well, I wonder how can I test the MT FFI? Or the reentrant FFI? Thanks, Esteban

11 25

Input semaphore signaling on win32
by Igor Stasenko 25 Mar '11

25 Mar '11

I integrated fix. So all images on all platforms could drop event polling logic (of course they should use proper VMs) on next release. And i made a little change comparing to my initial commit, which not puts event to event buffer unless there's a semaphore registered to handle them. See http://gitorious.org/cogvm/blessed/commit/d52a7e171fc6e5051bd4ba7b751bda1b0… for details -- Best regards, Igor Stasenko AKA sig.

2 1

Re: FFI syntax
by Torsten Bergmann 25 Mar '11

25 Mar '11

>Glad to hear it. I hate the attitude that the only way to make progress >is to break all code in existence. It's a lame and lazy attitude, >unworthy of engineers who understand what they're doing. It always depends on how fast you want to make progress and if you want to carry existing code with you/move it over to something new, correct? ;) To me two main questions remain: 1. why is it still hard to interface with the outside world from Squeak (is it a problem of current FFI implementation, the lack of callbacks, lack of pinned objects, ...) I doubt that it is a problem of Syntax, although I liked the way it was done in Smalltalk/MT and Smallscript: Example ST/MT: WINAPI MessageBox: NULL with: result asString with: 'Can you do this in another language' with: MB_YESNO | MB_ICONHAND Callback methods are in special method categories. Example Smallscript/S#: User32::MessageBox(NULL,'Hello','World',0) 2. how easy could a FFI (referring to a general framework - not the current implementation we have) be made so that people could easily interface with external stuff Especially if there is an answer to the second question one can think about making it compatible with existing stuff... I've used FFI a few times (for instance to bind libharu for PDF generation, see HPDF on squeaksource) and the current syntax was no problem. But the lack of callbacks and others ... holds us back to provide good bindings to C/C++, COM, Java, ... Bye T. -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de

3 2

VM Maker: VMMaker-dtl.222.mcz
by squeak-dev-noreply＠lists.squeakfoundation.org 25 Mar '11

25 Mar '11

Dave Lewis uploaded a new version of VMMaker to project VM Maker: http://www.squeaksource.com/VMMaker/VMMaker-dtl.222.mcz ==================== Summary ==================== Name: VMMaker-dtl.222 Author: dtl Time: 19 March 2011, 11:09:09 am UUID: 809d5eb5-6f73-4e6a-a595-428ed4536dc0 Ancestors: VMMaker-dtl.221 Fix MiscPrimitivePluginTest to properly test behavior of #primitiveFindSubstring and verify presence of a fix in TMethod>>argConversionExprFor:stackIndex: that performs type checking on string arguments.

9 22

Re: [Pharo-users] Get "Invalid utf8 input detected" error for a filename
by Igor Stasenko 25 Mar '11

25 Mar '11

I think this is VM-related issue. On 24 March 2011 15:31, Juraj Kubelka <juraj.kubelka(a)gmail.com> wrote: > For now I am able to fix it by following code: > #+BEGIN_SRC > Locale current. "a Locale(en)" > Latin2Environment class compile: 'fileNameConverterClass > ^ISO88592TextConverter'. > Locale switchToID: (LocaleID isoLanguage: 'cs'). > LanguageEnvironment clearDefault. > LanguageEnvironment defaultFileNameConverter. "an ISO88592TextConverter" > #+END_SRC > It is not perfect, directories with Czech characters is not possible to > browse. But I do not mind for now. It is anyway strange, because my > operating system environment is cs_CZ.UTF-8. And files are encoded in UTF-8. > But FileDirectory>>primLookupEntryIn:index: returns file names in a one byte > encoding. It is not valid Latin1 or Latin2. > If anyone knows better solution, please let me know. > Thanks in advance. > Juraj > On Wed, Mar 23, 2011 at 2:10 PM, Juraj Kubelka <juraj.kubelka(a)gmail.com> > wrote: >> >> Hi, >> If I try to execute: >> (FileDirectory on: '/home/jura') directoryNames >> it says "Invalid utf8 input detected" >> (UTF8TextConverter>>errorMalformedInput), because of a directory named >> 'Veřejné' ('ř' character is the wrong one). In the file system it is the >> right. FileDirectory>>primLookupEntryIn:index: returns a wrong ByteString >> ('Ve?ejné'). >> It was tested on Pharo 1.1.1 and Moose 4.3 where I am not able to use a >> Moose Panel tool because of the problem. >> Am I able to solve this problem somehow? Set-up something... >> Thank you in advance, >> Juraj > -- Best regards, Igor Stasenko AKA sig.

6 13

Re: [Pharo-project] test crashing the cog vm
by Eliot Miranda 24 Mar '11

24 Mar '11

On Thu, Mar 24, 2011 at 11:42 AM, Igor Stasenko <siguctua(a)gmail.com> wrote: > On 24 March 2011 19:34, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > > > > On Tue, Mar 22, 2011 at 5:34 AM, Stéphane Ducasse > > <stephane.ducasse(a)inria.fr> wrote: > >> > >> But why we could not have a byecode validator at the image level that > >> first make sure that byte code are in sync with the format of the > objects. > > > > Because it can be compromised. An in-image verifier is subject to > attack, > > and could be disabled by an attack that got past the in-image verifier > > before it got a chance to run. An in-VM verifier is not possible to > > side-step because it is the only way to execute code. So an in-VM > verifier > > can be secure but an in-image one can't and so is pointless. > > > For real hacker there's nothing impossible :) > True, but it can be much harder. How would you attempt to hack past the fact that the interpreter and JIT code, including the verifier, is in read-only memory? If this is the only route to create executable code, and it always verifies the bytecode before it executes then it is secure right? [there are issues; one doesn't want to verify a method each time it is activated in the interpreter, but e.g. a bit in the header saying "this method has been verified" might be easy to compromise. I'm not sure it is though, because one big advantage of the current funky method format (half literals, half btes) is that there's only one primitive to construct a method and it has to have a valid header, and there's only one way to modify the header, objectAt:put:. So as far as I can see the VM can in fact preserve the integrty a header flag bit that says "this method has been verified". Am I right or wrong?] > > Right now its not possible to split image to layered onion (like > operating system does, where you have kernel level, > and user level), but i think (at least in theory) such composition > could be implemented, except that sure thing > we don't have resources to invest in this direction. > > It is actually nice field for research (hello guys from academy :) > > -- > Best regards, > Igor Stasenko AKA sig. > >

3 5

Re: FFI syntax (was: Re: [Vm-dev] Re: testing MT and reentrant)
by Igor Stasenko 24 Mar '11

24 Mar '11

On 24 March 2011 20:58, Igor Stasenko <siguctua(a)gmail.com> wrote: > On 24 March 2011 20:56, Igor Stasenko <siguctua(a)gmail.com> wrote: >> On 24 March 2011 20:47, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >>> >>>> >>>> You can always put a string literal inside literal array.. it will >>>> cost you just extra quotes and extra parsing effort.. >>> >>> Yes, but that would be a failure. If yu ever need a string literal I'd >>> argue you're better off just using a string literal. But I think you could >>> be OK. I think your hack could work for everything. I just want a proof. >> >> it won't parse only following declarations: >> >> #( int foo( int arg[] ) ) >> > but we could modify compiler to treat [ and ] inside a literal array as > #'[' > and > #']' > > :) > oops, sorry, my bad.. it actually works... #( int foo ( int arg[] ) ) -> #(#int #foo #(#int #arg #'[' #']')) #( int foo ( int arg[10] ) ) -> #(#int #foo #(#int #arg #'[' 10 #']')) There is a bug in parser, which fails to parse literal arrays if you don't put space between function name and its arguments. #( int foo( int arg[] ) ) - fails #( int foo ( int arg[] ) ) - works >> which of course could be turned into just: >> >> #( int foo( int * arg ) ) >> > > > -- > Best regards, > Igor Stasenko AKA sig. > -- Best regards, Igor Stasenko AKA sig.

1 0

FFI syntax (was: Re: [Vm-dev] Re: testing MT and reentrant)
by Igor Stasenko 24 Mar '11

24 Mar '11

On 24 March 2011 20:10, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > On Thu, Mar 24, 2011 at 12:02 PM, Igor Stasenko <siguctua(a)gmail.com> wrote: >> >> On 24 March 2011 19:43, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >> > >> > >> > >> > 2011/3/24 Levente Uzonyi <leves(a)elte.hu> >> >> >> >> >> >> On Thu, 24 Mar 2011, Eliot Miranda wrote: >> >> >> >> (pine can't quote your mail, sorry) >> > >> > :) No problem. >> > >> >> >> >> "I have some image-level changes that I'll try and commit asap, or at >> >> least give you as some inbox packages >> >> - parsing FFI method annotations to discover the #threaded keyword that >> >> sets the "go ahead and thread this call" bit in an FFI spec >> >> - an additional inst var in ExternalFunction (IIRC) that allows the VM >> >> to record how much stack space to reserve when marshalling an FFI call >> >> - the new callback marshalling machinery that provides >> >> platform-independence >> >> >> >> I just need to fid the time to push the code to you. >> >> >> >> Now on testing I use >> >> a) an image with a "listener" that reads and writes from/to stdin/out >> >> while allowing one to interact with the image >> >> b) a native Mac OS file dialog that uses a threaded call to invoke >> >> without blocking the image and threaded callbacks to determine which files >> >> to >> >> show (filter callback) and what the accept action is. I've of course >> >> lost my workspace containing this and so have to ferret out the doits from >> >> my >> >> changes file (I feel such a fool!). Alas this will need work as it >> >> used an extension to a Teleplace native file dialog plugin. But posting it >> >> to >> >> FFI will be good; it will test the FFI further. >> >> c) a threaded version of the ODBC connect that hasn't really been >> >> tested >> >> >> >> etting you and others to start pounding on these would be fantastic. >> >> Just a matter of finding time. f you're happy for me to push inbox >> >> packages, >> >> changesets, workspace contents then I can provide something quickly. >> >> If you want me to commit canges that I've made sure are backward compatible >> >> and don't break trunk I'm going ot be much slower. >> >> >> >> best regards (albeit feeling a little frazzled :-) ), >> >> Eliot" >> >> >> >> Isn't it the best time to migrate the syntax of FFI calls to Pragmas? I >> >> know it takes some time to implement the pragma support, but IIRC Lukas did >> >> that a few years ago, so dusting it off and adding support for threaded >> >> calls (which I didn't see yet) shouldn't be that hard. >> > >> +1000. >> >> > SPunds good. Got any URLs? Lukas, got any code? >> > >> >> >> >> Both formats could be supported for a while, then we could leave the >> >> current format behind and simplify the parsers/compilers, etc. >> > >> +1 >> >> > Indeed. In fact, nothing is set in stone here. I desperately want an >> > accurate C signature, not the quick hack we have now (a good thing we have a >> > quick hack, but its a long way short of copy/pasting C into the pragma). >> > The interpreted marshalling scheme Andreas did (ffi specs etc) can serve >> > for the moment. The main thing before getting rid of that is a design for >> > e.g. a NativeBoost based marshalling code generator that is cross-platform, >> > cross-ABI and works with the interpreted VMs. I think this is therefore a >> > good way off, and there are people like Esteban who need to get going now, >> > so we can't let the perfect be the enemy of the good and need to provide >> > something functional asap, and that may mean living with the current ffi >> > annotation for a few months. >> >> In NativeBoost i made a separate class for parsing a C function >> declaration. It supports two forms of declaration: >> - named function, i.e. >> int foo (int bar, float zork) >> and anonymous one: >> int (int bar, float zork) >> >> as input, parser takes a literal array. >> >> So its quite easy to integrate it into compiler/pragma parser, we only >> need to find out what format will work for us. >> >> so, it could be something like: >> >> <cdecl: #( int foo (int bar , float baz )) module: #myModule> >> >> or: >> <ffi: #( cdecl int foo ... ) module: ... > > > This is such a fascinating hack. e.g. > #(int (*compar)(const void *, const void *)) > #(#int #(#* #compar) #(#const #void #*, #const #void #*)) > #(int (struct { field : 8; fence : 8; gate: 16; } arg)) > #(#int #(#struct #'{' #field #':' 8 #';' #fence #':' 8 #';' #gate: 16 > #';' #'}' #arg)) > Can anyone prove that Igor's approach works for all C signatures? After i did it, i were able to copy-paste functions declarations straightly from windoze documentation, without making any changes! :) >What about other languages? You can always put a string literal inside literal array.. it will cost you just extra quotes and extra parsing effort.. > I find this is like really dirty sex, repellent and irresistible at one and > the same time. :))) >> >> The class for parsing function prototype can be found in SqS/NativeBoost >> in class NBFnSpecParser >> and tests are in: >> NBFnSpecParserTest in NativeBoost-Test package. >> (some of them are NB-specific) but its easy to rip out extra functionality >> :) >> >> Also, there is a wiki page explaining this format: >> http://code.google.com/p/nativeboost/wiki/NBFnSpecParser >> >> So, if we agree with plan, i could port this class and adopt it for >> compiler (and yes , by taking Lukas code as well) >> >> -- >> Best regards, >> Igor Stasenko AKA sig. > > -- Best regards, Igor Stasenko AKA sig.

1 0

playing with threaded ffi as external plugin...
by Esteban Lorenzano 24 Mar '11

24 Mar '11

Hi, I found that ThreadedIA32FFIPlugin wasn't being loaded at all... so I started debugging until I saw some details (btw... xcode4 debugger is even worst than before): I changed this declaration: #ifdef SQUEAK_BUILTIN_PLUGIN #undef EXPORT // was #undef EXPORT(returnType) but screws NorCroft cc #define EXPORT(returnType) static returnType #endif with: #ifndef SQUEAK_BUILTIN_PLUGIN ... #endif also I changed: #ifdef SQUEAK_BUILTIN_PLUGIN extern #endif with: #ifndef SQUEAK_BUILTIN_PLUGIN extern #endif and then everything start working. I still does not know how to test the threads or callbacks for FFI, but a simple "MacOSShell new system: 'ls'." was working fine :) but here is my concern: the change I made looks like a heavy change... what I did is correct? should I change VMMaker code to generate plugins with my change? cheers, Esteban

3 4

Cog FreeBSD patches
by David Graham 24 Mar '11

24 Mar '11

Hello, I've noticed a few emails about running Cog on FreeBSD on some of the other mailing lists and was wondering if it would be possible to get a few patches integrated into the main repository? Sorry if I'm not following protocol for patch submission or style, I couldn't find any guidelines. I'd be happy to resubmit if this isn't the place. With the following patches, Cog r2372 will compile and run on FreeBSD (and PCBSD) 8.2 i386. I've been able to use Squeak, Pharo, and Seaside images. Here is the configure line: ../../platforms/unix/config/configure CFLAGS="-g -O2 -msse2 -DNDEBUG -DITIMER_HEARTBEAT=1 -DNO_VM_PROFILE=1 -DCOGMTVM=0 -DNOEXECINFO -DDEBUGVM=0" LIBS=-lpthread --without-npsqueak Here is the diff. Index: platforms/unix/plugins/SqueakFFIPrims/ffi-config =================================================================== --- platforms/unix/plugins/SqueakFFIPrims/ffi-config (revision 2372) +++ platforms/unix/plugins/SqueakFFIPrims/ffi-config (working copy) @@ -39,6 +39,7 @@ case ${abi} in linux) abi=sysv;; + freebsd) abi=sysv;; darwin*) abi=darwin;; *) abi=libffi; lib="-lffi";; esac Index: platforms/unix/vm/sqUnixMain.c =================================================================== --- platforms/unix/vm/sqUnixMain.c (revision 2372) +++ platforms/unix/vm/sqUnixMain.c (working copy) @@ -60,6 +60,9 @@ # include <execinfo.h> # define BACKTRACE_DEPTH 64 #endif +#if __FreeBSD__ +# include <sys/ucontext.h> +#endif #if defined(__alpha__) && defined(__osf__) # include <sys/sysinfo.h> @@ -777,6 +780,9 @@ # elif __linux__ && __i386__ void *fp = (void *)(uap ? uap->uc_mcontext.gregs[REG_EBP]: 0); void *sp = (void *)(uap ? uap->uc_mcontext.gregs[REG_ESP]: 0); +# elif __FreeBSD__ && __i386__ + void *fp = (void *)(uap ? uap->uc_mcontext.mc_ebp: 0); + void *sp = (void *)(uap ? uap->uc_mcontext.mc_esp: 0); # else # error need to implement extracting pc from a ucontext_t on this system # endif Index: platforms/Cross/plugins/SqueakFFIPrims/sqFFIPlugin.c =================================================================== --- platforms/Cross/plugins/SqueakFFIPrims/sqFFIPlugin.c (revision 2372) +++ platforms/Cross/plugins/SqueakFFIPrims/sqFFIPlugin.c (working copy) @@ -10,7 +10,7 @@ *****************************************************************************/ #include <stdio.h> -#if !WIN32 +#if !WIN32 && !__FreeBSD__ # include <alloca.h> #endif #include <string.h>

8 40

← Newer
1
2
3
4
5
6
7
...
14
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev March 2011