Who is the maintainer of the FFI-Package for Squeak and/or Pharo ? I especially interested in the FFI-Unix examples and the state on working FFI on linux for current squeak release (AFAIK it does not work with "squeak all in one" 4.5)
regards Nicolai
On Sun, Sep 28, 2014 at 11:19:30AM +0200, Nicolai Hess wrote:
Who is the maintainer of the FFI-Package for Squeak and/or Pharo ? I especially interested in the FFI-Unix examples and the state on working FFI on linux for current squeak release (AFAIK it does not work with "squeak all in one" 4.5)
The FFI package, including the FFI-Unix examples that you mention, is at http://source.squeak.org/FFI.
The primary author was Andreas Raab. Eliot Miranda is the most recent contributor, and he is probably the most knowledgeable in this area. Currently, the Squeak core developers (those with commit rights to Squeak trunk) have commit rights to the FFI project, and can make updates as needed.
The FFI examples for Unix work with the old FFI plugin, which is still present in the interpreter VM. The active development of Squeak FFI is taking place in Cog, and I'm not sure if the old FFI examples work with the newer plugins (which would be present in the Squeak all-in-one). The FFI plugin in the interpreter VM (i.e. trunk VMMaker) needs to be updated, and presumably the FFI examples may need some updates as well.
Dave
On Sun, Sep 28, 2014 at 09:04:59AM -0400, David T. Lewis wrote:
On Sun, Sep 28, 2014 at 11:19:30AM +0200, Nicolai Hess wrote:
Who is the maintainer of the FFI-Package for Squeak and/or Pharo ? I especially interested in the FFI-Unix examples and the state on working FFI on linux for current squeak release (AFAIK it does not work with "squeak all in one" 4.5)
The FFI package, including the FFI-Unix examples that you mention, is at http://source.squeak.org/FFI.
The primary author was Andreas Raab. Eliot Miranda is the most recent contributor, and he is probably the most knowledgeable in this area. Currently, the Squeak core developers (those with commit rights to Squeak trunk) have commit rights to the FFI project, and can make updates as needed.
The FFI examples for Unix work with the old FFI plugin, which is still present in the interpreter VM. The active development of Squeak FFI is taking place in Cog, and I'm not sure if the old FFI examples work with the newer plugins (which would be present in the Squeak all-in-one). The FFI plugin in the interpreter VM (i.e. trunk VMMaker) needs to be updated, and presumably the FFI examples may need some updates as well.
A small but important correction: For the old plugin (currently in the interpreter VM), FFI works only for a 32-bit VM. It also works for a 64-bit VM, but only if the patches in http://bugs.squeak.org/view.php?id=7237 have been applied (in other words, if you compiled it yourself and included those patches).
This is mainly of historical interest, although it is possible that some of the 64-bit patches will prove helpful for the newer FFI plugin. But the emphasis now should be on Eliot's more recent FFI work.
Dave
Hi David,
On Sep 28, 2014, at 12:44 PM, "David T. Lewis" lewis@mail.msen.com wrote:
On Sun, Sep 28, 2014 at 09:04:59AM -0400, David T. Lewis wrote:
On Sun, Sep 28, 2014 at 11:19:30AM +0200, Nicolai Hess wrote:
Who is the maintainer of the FFI-Package for Squeak and/or Pharo ? I especially interested in the FFI-Unix examples and the state on working FFI on linux for current squeak release (AFAIK it does not work with "squeak all in one" 4.5)
The FFI package, including the FFI-Unix examples that you mention, is at http://source.squeak.org/FFI.
The primary author was Andreas Raab. Eliot Miranda is the most recent contributor, and he is probably the most knowledgeable in this area. Currently, the Squeak core developers (those with commit rights to Squeak trunk) have commit rights to the FFI project, and can make updates as needed.
The FFI examples for Unix work with the old FFI plugin, which is still present in the interpreter VM. The active development of Squeak FFI is taking place in Cog, and I'm not sure if the old FFI examples work with the newer plugins (which would be present in the Squeak all-in-one). The FFI plugin in the interpreter VM (i.e. trunk VMMaker) needs to be updated, and presumably the FFI examples may need some updates as well.
A small but important correction: For the old plugin (currently in the interpreter VM), FFI works only for a 32-bit VM. It also works for a 64-bit VM, but only if the patches in http://bugs.squeak.org/view.php?id=7237 have been applied (in other words, if you compiled it yourself and included those patches).
Good point. It should be easier to adapt the ThreadedFFIPlugin to 64 bits because there's no assembler involved.
But this gets to the crux of why we need to rearchitect, because the x86-64/IA64 ABI is extremely complex to use interpretively and much easier to approach with a special-purpose compiler. Some steps are being taken in this direction in the Pharo community.
In fact there's more possibility of getting x86-64/IA64 working with the Squeak FFI than with eg VisualWorks because Andreas' type signature scheme (ExternalType et al) is richer than VW's. but it's work; much of the marshaling code in the plugin would need rewriting.
We're just starting the Spur 64-but VM and do will tackle this soon enough (some time mid next year?). Whether by that time it'll make more sense to extend the existing ThreadedFFIPlugin or go with the new architecture is too early to tell.
HTH
This is mainly of historical interest, although it is possible that some of the 64-bit patches will prove helpful for the newer FFI plugin. But the emphasis now should be on Eliot's more recent FFI work.
+1
Dave
Cheers, Eliot
Hi,
I have recently made some experiments with pharo and FFI, and I have ran into problems with callbacks. I am using linux 32 bits. First question : when calling a dynamic library is it necessary that it exports all symbols ? What if symbols are noted « U » (undefined) ? 2) On one of my callbacks got the message « callback failed to own the VM » . Have you any idea of where it comes from ? 3) If the callback is called in a different thread, is it a problem ?
4) On Pharo I have an error on the following : is it incorrect ? It is a 64 bits problem ? Using FFI Linux examples.
testGetOurWindowLocation "self new testGetOurWindowLocation"
| display ourWindow aParent anX anY aWidth aHeight aBorderWidth aDepth | display := X11Display XOpenDisplay: nil. ourWindow := display ourWindow. (Alien lookup: 'XGetGeometry' inLibrary: 'X11') primFFICallResult: nil withArguments: {display. ourWindow. (aParent := Alien new: 4). (anX := Alien new: 4). (anY := Alien new: 4). (aWidth := Alien new: 4). (aHeight := Alien new: 4). (aBorderWidth := Alien new: 4). (aDepth := Alien new: 4)}. (anX unsignedLongAt: 1) inspect
Annick Le 28 sept. 2014 à 21:44, David T. Lewis lewis@mail.msen.com a écrit :
On Sun, Sep 28, 2014 at 09:04:59AM -0400, David T. Lewis wrote:
On Sun, Sep 28, 2014 at 11:19:30AM +0200, Nicolai Hess wrote:
Who is the maintainer of the FFI-Package for Squeak and/or Pharo ? I especially interested in the FFI-Unix examples and the state on working FFI on linux for current squeak release (AFAIK it does not work with "squeak all in one" 4.5)
The FFI package, including the FFI-Unix examples that you mention, is at http://source.squeak.org/FFI.
The primary author was Andreas Raab. Eliot Miranda is the most recent contributor, and he is probably the most knowledgeable in this area. Currently, the Squeak core developers (those with commit rights to Squeak trunk) have commit rights to the FFI project, and can make updates as needed.
The FFI examples for Unix work with the old FFI plugin, which is still present in the interpreter VM. The active development of Squeak FFI is taking place in Cog, and I'm not sure if the old FFI examples work with the newer plugins (which would be present in the Squeak all-in-one). The FFI plugin in the interpreter VM (i.e. trunk VMMaker) needs to be updated, and presumably the FFI examples may need some updates as well.
A small but important correction: For the old plugin (currently in the interpreter VM), FFI works only for a 32-bit VM. It also works for a 64-bit VM, but only if the patches in http://bugs.squeak.org/view.php?id=7237 have been applied (in other words, if you compiled it yourself and included those patches).
This is mainly of historical interest, although it is possible that some of the 64-bit patches will prove helpful for the newer FFI plugin. But the emphasis now should be on Eliot's more recent FFI work.
Dave
Hi Annick,
On Mon, Sep 29, 2014 at 5:30 AM, Annick Fron list@afceurope.com wrote:
Hi,
I have recently made some experiments with pharo and FFI, and I have ran into problems with callbacks. I am using linux 32 bits. First question : when calling a dynamic library is it necessary that it exports all symbols ? What if symbols are noted « U » (undefined) ?
If you have undefined symbols in an external library, they need to be resoled by the VM or other libraries loaded in the system. If any are not resolved the library will fail to load.
- On one of my callbacks got the message « callback failed to own the VM
» . Have you any idea of where it comes from ?
Yes. This means you're trying to callback from a different thread than the VM thread. hat isn't supported yet.
- If the callback is called in a different thread, is it a problem ?
Yes.
4) On Pharo I have an error on the following : is it incorrect ? It is a 64
bits problem ? Using FFI Linux examples.
testGetOurWindowLocation "self new testGetOurWindowLocation"
| display ourWindow aParent anX anY aWidth aHeight aBorderWidth aDepth | display := X11Display XOpenDisplay: nil. ourWindow := display ourWindow. (Alien lookup: 'XGetGeometry' inLibrary: 'X11') primFFICallResult: nil withArguments: {display. ourWindow. (aParent := Alien new: 4). (anX := Alien new: 4). (anY := Alien new: 4). (aWidth := Alien new: 4). (aHeight := Alien new: 4). (aBorderWidth := Alien new: 4). (aDepth := Alien new: 4)}. (anX unsignedLongAt: 1) inspect
What's the error you get?
Annick
Le 28 sept. 2014 à 21:44, David T. Lewis lewis@mail.msen.com a écrit :
On Sun, Sep 28, 2014 at 09:04:59AM -0400, David T. Lewis wrote:
On Sun, Sep 28, 2014 at 11:19:30AM +0200, Nicolai Hess wrote:
Who is the maintainer of the FFI-Package for Squeak and/or Pharo ? I especially interested in the FFI-Unix examples and the state on working FFI on linux for current squeak release (AFAIK it does not work with "squeak all in one" 4.5)
The FFI package, including the FFI-Unix examples that you mention, is at http://source.squeak.org/FFI.
The primary author was Andreas Raab. Eliot Miranda is the most recent contributor, and he is probably the most knowledgeable in this area. Currently, the Squeak core developers (those with commit rights to
Squeak
trunk) have commit rights to the FFI project, and can make updates as
needed.
The FFI examples for Unix work with the old FFI plugin, which is still present in the interpreter VM. The active development of Squeak FFI is taking place in Cog, and I'm not sure if the old FFI examples work with the newer plugins (which would be present in the Squeak all-in-one). The FFI plugin in the interpreter VM (i.e. trunk VMMaker) needs to be updated, and presumably the FFI examples may need some updates as well.
A small but important correction: For the old plugin (currently in the interpreter VM), FFI works only for a 32-bit VM. It also works for a
64-bit
VM, but only if the patches in http://bugs.squeak.org/view.php?id=7237
have
been applied (in other words, if you compiled it yourself and included those patches).
This is mainly of historical interest, although it is possible that some of the 64-bit patches will prove helpful for the newer FFI plugin. But the emphasis now should be on Eliot's more recent FFI work.
Dave
Hi Eliot,
The error I get is « bad argument » on the primitive call.
Another question : why Alien newC: has dataSize 8 bits ? is it for 64 bits architectures ??
Le 30 sept. 2014 à 20:38, Eliot Miranda eliot.miranda@gmail.com a écrit :
Hi Annick,
On Mon, Sep 29, 2014 at 5:30 AM, Annick Fron list@afceurope.com wrote: Hi,
I have recently made some experiments with pharo and FFI, and I have ran into problems with callbacks. I am using linux 32 bits. First question : when calling a dynamic library is it necessary that it exports all symbols ? What if symbols are noted « U » (undefined) ?
If you have undefined symbols in an external library, they need to be resoled by the VM or other libraries loaded in the system. If any are not resolved the library will fail to load.
- On one of my callbacks got the message « callback failed to own the VM » . Have you any idea of where it comes from ?
Yes. This means you're trying to callback from a different thread than the VM thread. hat isn't supported yet.
- If the callback is called in a different thread, is it a problem ?
Yes.
- On Pharo I have an error on the following : is it incorrect ? It is a 64 bits problem ?
Using FFI Linux examples.
testGetOurWindowLocation "self new testGetOurWindowLocation"
| display ourWindow aParent anX anY aWidth aHeight aBorderWidth aDepth | display := X11Display XOpenDisplay: nil. ourWindow := display ourWindow. (Alien lookup: 'XGetGeometry' inLibrary: 'X11') primFFICallResult: nil withArguments: {display. ourWindow. (aParent := Alien new: 4). (anX := Alien new: 4). (anY := Alien new: 4). (aWidth := Alien new: 4). (aHeight := Alien new: 4). (aBorderWidth := Alien new: 4). (aDepth := Alien new: 4)}. (anX unsignedLongAt: 1) inspect
What's the error you get?
Annick Le 28 sept. 2014 à 21:44, David T. Lewis lewis@mail.msen.com a écrit :
On Sun, Sep 28, 2014 at 09:04:59AM -0400, David T. Lewis wrote:
On Sun, Sep 28, 2014 at 11:19:30AM +0200, Nicolai Hess wrote:
Who is the maintainer of the FFI-Package for Squeak and/or Pharo ? I especially interested in the FFI-Unix examples and the state on working FFI on linux for current squeak release (AFAIK it does not work with "squeak all in one" 4.5)
The FFI package, including the FFI-Unix examples that you mention, is at http://source.squeak.org/FFI.
The primary author was Andreas Raab. Eliot Miranda is the most recent contributor, and he is probably the most knowledgeable in this area. Currently, the Squeak core developers (those with commit rights to Squeak trunk) have commit rights to the FFI project, and can make updates as needed.
The FFI examples for Unix work with the old FFI plugin, which is still present in the interpreter VM. The active development of Squeak FFI is taking place in Cog, and I'm not sure if the old FFI examples work with the newer plugins (which would be present in the Squeak all-in-one). The FFI plugin in the interpreter VM (i.e. trunk VMMaker) needs to be updated, and presumably the FFI examples may need some updates as well.
A small but important correction: For the old plugin (currently in the interpreter VM), FFI works only for a 32-bit VM. It also works for a 64-bit VM, but only if the patches in http://bugs.squeak.org/view.php?id=7237 have been applied (in other words, if you compiled it yourself and included those patches).
This is mainly of historical interest, although it is possible that some of the 64-bit patches will prove helpful for the newer FFI plugin. But the emphasis now should be on Eliot's more recent FFI work.
Dave
-- best, Eliot
Hi Nicolai,
On Sep 28, 2014, at 2:19 AM, Nicolai Hess nicolaihess@web.de wrote:
Who is the maintainer of the FFI-Package for Squeak and/or Pharo ?
I guess I'm the de facto maintainer because I'm the one who has changed things most recently, reimplementing the FFI plugin via the ThreadedFFIPlugin.
I especially interested in the FFI-Unix examples and the state on working FFI on linux for current squeak release (AFAIK it does not work with "squeak all in one" 4.5)
Can you give me a reproducible case? I'd this with a Cog,Stack or Interpreter VM? Which specific things fail and how?
regards Nicolai
Eliot (phone)
Thank you David, Thank you Eliot
2014-09-28 21:24 GMT+02:00 Eliot Miranda eliot.miranda@gmail.com:
Hi Nicolai,
On Sep 28, 2014, at 2:19 AM, Nicolai Hess nicolaihess@web.de wrote:
Who is the maintainer of the FFI-Package for Squeak and/or Pharo ?
I guess I'm the de facto maintainer because I'm the one who has changed things most recently, reimplementing the FFI plugin via the ThreadedFFIPlugin.
I especially interested in the FFI-Unix examples and the state on working FFI on linux for current squeak release (AFAIK it does not work with "squeak all in one" 4.5)
Can you give me a reproducible case? I'd this with a Cog,Stack or Interpreter VM? Which specific things fail and how?
Image ----- /home/nicolai/Downloads/Squeak-4.5-All-in-One.app/Contents/Resources/Squeak4.5-13680.image Squeak4.5 latest update: #13680 Current Change Set: Unnamed1 Image format 6505 (32 bit)
Virtual Machine --------------- /home/nicolai/Downloads/Squeak-4.5-All-in-One.app/Contents/Linux-i686/lib/squeak/4.0-2776/squeak Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.331] Unix built on Aug 22 2013 10:35:37 Compiler: 4.1.2 20080704 (Red Hat 4.1.2-48) platform sources revision VM: r2776 http://www.squeakvm.org/svn/squeak/branches/Cog Plugins: r2545 http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins CoInterpreter VMMaker.oscog-eem.331 uuid: 37d2e4b0-2f37-4e2d-8313-c63637785e59 Aug 22 2013 StackToRegisterMappingCogit VMMaker.oscog-eem.333 uuid: 84da9cb8-7f30-4cb7-b4fb-239a11f63b54 Aug 22 2013
Load Name: FFI-Pools-eem.3 Name: FFI-Kernel-eem.26 Name: FFI-Tests-djm.7
Run FFI-Test -> 32 run, 24 passes, 0 expected failures, 0 failures, 8 errors, 0 unexpected passes
Load Name: FFI-Unix-mtf.4
run X11Display class>>#coloredRectangles -> Error: Could not coerse arguments
Load newer cogvm Virtual Machine --------------- /media/Programs/squeak/Squeak-4.5-All-in-One.app/coglinux/lib/squeak/4.0-2987/squeak Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.746] Unix built on Jun 11 2014 14:59:18 Compiler: 4.1.2 20080704 (Red Hat 4.1.2-48) platform sources revision VM: r2987 http://www.squeakvm.org/svn/squeak/branches/Cog Date: 2014-06-11 14:26:16 -0700 Plugins: r2954 http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins CoInterpreter VMMaker.oscog-eem.746 uuid: cc4961d3-e629-4e28-b308-88eab314a8c9 Jun 11 2014 StackToRegisterMappingCogit VMMaker.oscog-eem.766 uuid: 3045c341-cfbf-494c-a7b1-fb29c26a7340 Jun 11 2014
Run FFI-Test -> 32 run, 32 passes, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes
The error in FFI-Unix is strange. The last package comment from Name: FFI-Unix-mtf.4 Author: mtf is "made the XLib examples actually work (Window is an unsigned int handle, not a struct)"
That the window is an int handle and not a struct explains why this version replaces the parent class of XDrawable from ExternalStructur to Object. But all calls with an XDrawable still have this object as parameter ( and not its instance var "xid"). I don't understand how this ever worked (or was there some kind of automatic argument conversion like: transform an object with an instvar -> a int, if the function expects an int handle?)
Sure we can replace all XDrawable arguments in X11 calls from "aXDrawable" to "aXDrawable xid". But I only wonder, did this ever work and if Author "mtf" did the last change, can I contact him or is he still maintaining the FF-Unix examples?
regards Nicolai
Eliot (phone)
2014-09-28 22:04 GMT+02:00 Nicolai Hess nicolaihess@web.de:
Thank you David, Thank you Eliot
2014-09-28 21:24 GMT+02:00 Eliot Miranda eliot.miranda@gmail.com:
Hi Nicolai,
On Sep 28, 2014, at 2:19 AM, Nicolai Hess nicolaihess@web.de wrote:
Who is the maintainer of the FFI-Package for Squeak and/or Pharo ?
I guess I'm the de facto maintainer because I'm the one who has changed things most recently, reimplementing the FFI plugin via the ThreadedFFIPlugin.
I especially interested in the FFI-Unix examples and the state on working FFI on linux for current squeak release (AFAIK it does not work with "squeak all in one" 4.5)
Can you give me a reproducible case? I'd this with a Cog,Stack or Interpreter VM? Which specific things fail and how?
Image
/home/nicolai/Downloads/Squeak-4.5-All-in-One.app/Contents/Resources/Squeak4.5-13680.image Squeak4.5 latest update: #13680 Current Change Set: Unnamed1 Image format 6505 (32 bit)
Virtual Machine
/home/nicolai/Downloads/Squeak-4.5-All-in-One.app/Contents/Linux-i686/lib/squeak/4.0-2776/squeak Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.331] Unix built on Aug 22 2013 10:35:37 Compiler: 4.1.2 20080704 (Red Hat 4.1.2-48) platform sources revision VM: r2776 http://www.squeakvm.org/svn/squeak/branches/Cog Plugins: r2545 http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins CoInterpreter VMMaker.oscog-eem.331 uuid: 37d2e4b0-2f37-4e2d-8313-c63637785e59 Aug 22 2013 StackToRegisterMappingCogit VMMaker.oscog-eem.333 uuid: 84da9cb8-7f30-4cb7-b4fb-239a11f63b54 Aug 22 2013
Load Name: FFI-Pools-eem.3 Name: FFI-Kernel-eem.26 Name: FFI-Tests-djm.7
Run FFI-Test -> 32 run, 24 passes, 0 expected failures, 0 failures, 8 errors, 0 unexpected passes
Load Name: FFI-Unix-mtf.4
run X11Display class>>#coloredRectangles -> Error: Could not coerse arguments
Load newer cogvm Virtual Machine
/media/Programs/squeak/Squeak-4.5-All-in-One.app/coglinux/lib/squeak/4.0-2987/squeak Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.746] Unix built on Jun 11 2014 14:59:18 Compiler: 4.1.2 20080704 (Red Hat 4.1.2-48) platform sources revision VM: r2987 http://www.squeakvm.org/svn/squeak/branches/Cog Date: 2014-06-11 14:26:16 -0700 Plugins: r2954 http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins CoInterpreter VMMaker.oscog-eem.746 uuid: cc4961d3-e629-4e28-b308-88eab314a8c9 Jun 11 2014 StackToRegisterMappingCogit VMMaker.oscog-eem.766 uuid: 3045c341-cfbf-494c-a7b1-fb29c26a7340 Jun 11 2014
Run FFI-Test -> 32 run, 32 passes, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes
The error in FFI-Unix is strange. The last package comment from Name: FFI-Unix-mtf.4 Author: mtf is "made the XLib examples actually work (Window is an unsigned int handle, not a struct)"
That the window is an int handle and not a struct explains why this version replaces the parent class of XDrawable from ExternalStructur to Object. But all calls with an XDrawable still have this object as parameter ( and not its instance var "xid"). I don't understand how this ever worked (or was there some kind of automatic argument conversion like: transform an object with an instvar -> a int, if the function expects an int handle?)
Sure we can replace all XDrawable arguments in X11 calls from "aXDrawable" to "aXDrawable xid". But I only wonder, did this ever work and if Author "mtf" did the last change, can I contact him or is he still maintaining the FF-Unix examples?
I would like to raise this topic again.
Nothing has changed, the linux example still don't work and I can not get information about who maintains the unix-examples.
From my analysis (see above), the fix may be simple (in another discussion
around that time some one else asked how to make that working, and confirmed it works with that fix). But I am confused because the latest update for the examples had the comment "made the XLib examlpes actually work". So maybe my usage or version of FFI is wrong.
The last time I asked, someone said this works for him with the current stable vm, but it was actually the latest cog vm he used.
But now, a year later, with the current 4.6 cog and 5.0 spur vm, some FFI unit tests still failing so maybe I am using the wrong FFI package (FFI-Kernel-EstebanLorenzano.27) ?
(BTW, there is a typo in ExternalAddress class >> loadSymbol: moduleSymbol module: module <primitive: 'primitiveLoadSymbolFromModule' module: 'SqueakFFIPrims'> ^ self primitiveFail it should call ^ self primitiveFailed
regards Nicolai
Eliot (phone)
squeak-dev@lists.squeakfoundation.org
-
Annick Fron -
David T. Lewis -
Eliot Miranda -
Nicolai Hess