I got pulled off to do some $ work and haven't got back to this but I would like to get back to it.
Has there been any progress on this in the publicly available 64bit VM?
I see there is a completely separate FFI plugin for 32 vs 64 and worry 64 is still not quite ready for real work.
On Dec 3, 2017, at 7:18 AM, Ben Coman btc@openinworld.com wrote:
On 27 November 2017 at 00:24, Todd Blanchard <tblanchard@mac.com mailto:tblanchard@mac.com> wrote:
i'm getting the idea that we should probably write a test suite/library for FFI
I noticed these...
- https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/pl... https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/plugins/SqueakFFIPrims/ffi-test-main.c
- https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/Cross/p... https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/Cross/plugins/SqueakFFIPrims/sqFFITestFuncs.c
- https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/Cross/p... https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/Cross/plugins/IA32ABI/AlienSUnitTestProcedures.c
These look like c-code test frames for FFI. Could these be built by the OpenSmalltalk CI to be normally shipped with the VM so that Image-side CI can test against them?
And also some general info...
- https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/pl... https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/plugins/SqueakFFIPrims/00README
On Nov 24, 2017, at 12:54 AM, Ben Coman <btc@openinworld.com mailto:btc@openinworld.com> wrote:
On 24 November 2017 at 13:16, Ben Coman <btc@openinworld.com mailto:btc@openinworld.com> wrote:
On 22 November 2017 at 21:59, Ben Coman <btc@openinworld.com mailto:btc@openinworld.com> wrote:
On 22 November 2017 at 13:38, Todd Blanchard <tblanchard@mac.com mailto:tblanchard@mac.com> wrote:
I've been trying to track this down for a couple weeks now.
I have concluded that structs passed by value to functions on the 64 bit VM are not properly populated. The struct's memory is all zero'd.
I found this while trying to work with LibClang and found that functions that fetched code locations from code ranges always returned invalid zero'd locations. After spending some time with lldb I have traced the problem into the native code and found that the argument is not correct.
I've carved out the wee bit of clang to reproduce this in a tiny library.
The gist of it is below and the entire file is included. Basically the struct passed to the function clang_getRangeStart is zero'd memory regardless of the data I send from the image side.
My last analysis discovered something interesting about strings defined inside shared libraries being handled differently, but only later realised I had chased the wrong rabbit down the hole.
I've now investigated the premise you actually poses, and I agree, that structs being zero is some cases. The attached zipfile containing libstruct.c a few comparison cases - three "good" layouts that work fine and one "bad" that mostly gets zeros but sometimes other weird numbers. There is a Makefile with three main targets:
make layout - statically compiles libstruct.c and runs produced a.out to display structure layouts
make run - downloads Pharo, starts it loading LibStruct.st http://libstruct.st/ , then you manually run LibStruct>>>LibStructTest>>#testStructs and observe structure values on console
make debug - starts LLDB to run Pharo with breakpoints pre-configured for when you run #testStructs. Note Pharo will freeze and you need to move to LLDB. Try these commands... frame variable call print_struct(&GoodStruct1_fmt, &aStruct) continue
That works on Ubuntu 16.04 64 bit. You will need to tune it for OSX. clang and lldb are required.
================================ The offsets configured in the class variables of all ExternalStructs correctly matches that reported by the C code test frame results here...
$ make layout clang -g libstruct.c ./a.out
GoodStruct1: uint32_t:int1: 01 4 uint32_t:int2: 05 4 GoodStruct2: uint32_t:int1: 01 4 uint32_t:int2: 05 4 void*:ptr_data: 09 8 GoodStruct3: void*:ptr_data: 01 8 uint32_t:int1: 09 4 uint32_t:int2: 13 4 BadStruct: void*:ptr_data1: 01 8 void*:ptr_data2: 09 8 uint32_t:int1: 17 4 uint32_t:int2: 21 4
================================
$ make run clang -g -o libstruct.so -shared -fPIC libstruct.c getpharo/pharo-vm/lib/pharo/5.0-201707201942/pharo getpharo/Pharo.image ../LibStruct.st http://libstruct.st/
Now manually browse to and run LibStruct>>>LibStructTest>>#testStructs LibStructTest>>testStructs |good1Struct good2Struct good3Struct badStruct |
good1Struct := GoodStruct1 new int1: 2; int2: 3. good2Struct := GoodStruct2 new int1: 2; int2: 3; ptr_force_int: 4. good3Struct := GoodStruct3 new int1: 2; int2: 3; ptr_force_int: 4. badStruct := BadStruct new int1: 2; int2: 3; ptr_force_int1: 4; ptr_force_int1: 5.
self assert: (LibStruct tryGood1: good1Struct) equals: 6. self assert: (LibStruct tryGood2: good2Struct) equals: 6. self assert: (LibStruct tryGood3: good3Struct) equals: 6. "Problem exposed in next line" self assert: (LibStruct tryBad: badStruct) equals: 6.
which on console produces...
GoodStruct1: uint32_t:int1: 01 4 = 02 00 00 00 uint32_t:int2: 05 4 = 03 00 00 00 GoodStruct2: uint32_t:int1: 01 4 = 02 00 00 00 uint32_t:int2: 05 4 = 03 00 00 00 void*:ptr_data: 09 8 = 04 00 00 00 00 00 00 00 GoodStruct3: void*:ptr_data: 01 8 = 04 00 00 00 00 00 00 00 uint32_t:int1: 09 4 = 02 00 00 00 uint32_t:int2: 13 4 = 03 00 00 00 BadStruct: void*:ptr_data1: 01 8 = 00 00 00 00 00 00 00 00 void*:ptr_data2: 09 8 = 00 00 00 00 00 00 00 00 uint32_t:int1: 17 4 = 29 03 03 00 uint32_t:int2: 21 4 = 00 00 00 00
Comparing GoodStruct3 and BadStruct, it seems one pointer is handled fine, but not two. The first time this is run after Image boots seems like BadStruct gets some random data.
================================ In same image, subsequent runs of LibStruct>>>LibStructTest>>testStructs give only all zeros for BadStruct, as Todd observed.
GoodStruct1: uint32_t:int1: 01 4 = 02 00 00 00 uint32_t:int2: 05 4 = 03 00 00 00 GoodStruct2: uint32_t:int1: 01 4 = 02 00 00 00 uint32_t:int2: 05 4 = 03 00 00 00 void*:ptr_data: 09 8 = 00 00 00 00 00 00 00 00 GoodStruct3: void*:ptr_data: 01 8 = 00 00 00 00 00 00 00 00 uint32_t:int1: 09 4 = 02 00 00 00 uint32_t:int2: 13 4 = 03 00 00 00 BadStruct: void*:ptr_data1: 01 8 = 00 00 00 00 00 00 00 00 void*:ptr_data2: 09 8 = 00 00 00 00 00 00 00 00 uint32_t:int1: 17 4 = 00 00 00 00 uint32_t:int2: 21 4 = 00 00 00 00
cheers -ben
<FFI64StructTest.zip>