I got pulled off to do some $ work and haven't got back to this but I would like to get back to it.

Has there been any progress on this in the publicly available 64bit VM?

I see there is a completely separate FFI plugin for 32 vs 64 and worry 64 is still not quite ready for real work.

On Dec 3, 2017, at 7:18 AM, Ben Coman <btc@openinworld.com> wrote:



On 27 November 2017 at 00:24, Todd Blanchard <tblanchard@mac.com> wrote:
 
i'm getting the idea that we should probably write a test suite/library for FFI

I noticed these...

These look like c-code test frames for FFI.  Could these be built by the OpenSmalltalk CI to be normally shipped with the VM so that Image-side CI can test against them?

And also some general info...

 

On Nov 24, 2017, at 12:54 AM, Ben Coman <btc@openinworld.com> wrote:


On 24 November 2017 at 13:16, Ben Coman <btc@openinworld.com> wrote:


On 22 November 2017 at 21:59, Ben Coman <btc@openinworld.com> wrote:


On 22 November 2017 at 13:38, Todd Blanchard <tblanchard@mac.com> wrote:
 
I've been trying to track this down for a couple weeks now.

I have concluded that structs passed by value to functions on the 64 bit VM are not properly populated.  The struct's memory is all zero'd.

I found this while trying to work with LibClang and found that functions that fetched code locations from code ranges always returned invalid zero'd locations.  After spending some time with lldb I have traced the problem into the native code and found that the argument is not correct.

I've carved out the wee bit of clang to reproduce this in a tiny library.

The gist of it is below and the entire file is included.  Basically the struct passed to the function clang_getRangeStart is zero'd memory regardless of the data I send from the image side.

My last analysis discovered something interesting about strings defined inside shared libraries being handled differently, but only later realised I had chased the wrong rabbit down the hole.  
 
I've now investigated the premise you actually poses, and I agree, that structs being zero is some cases.  
The attached zipfile containing  libstruct.c  a few comparison cases - three "good" layouts that work fine and one "bad" that mostly gets zeros but sometimes other weird numbers.  There is a Makefile with three main targets:
1. make layout - statically compiles  libstruct.c  and runs produced  a.out  to display structure layouts 

2. make run - downloads Pharo, starts it loading  LibStruct.st  , then you manually run LibStruct>>>LibStructTest>>#testStructs and observe structure values on console

3. make debug - starts LLDB to run Pharo with breakpoints pre-configured for when you run #testStructs.  
   Note Pharo will freeze and you need to move to LLDB.  Try these commands... 
      frame variable
      call print_struct(&GoodStruct1_fmt, &aStruct)
      continue

That works on Ubuntu 16.04 64 bit. 
You will need to tune it for OSX.
clang and lldb are required.

================================
The offsets configured in the class variables of all ExternalStructs 
correctly matches that reported by the C code test frame results here...

$ make layout        
clang -g libstruct.c
./a.out

GoodStruct1:
uint32_t:int1:    01 4 
uint32_t:int2:    05 4 
GoodStruct2:
uint32_t:int1:    01 4 
uint32_t:int2:    05 4 
void*:ptr_data:    09 8 
GoodStruct3:
void*:ptr_data:    01 8 
uint32_t:int1:    09 4 
uint32_t:int2:    13 4 
BadStruct:
void*:ptr_data1:    01 8 
void*:ptr_data2:    09 8 
uint32_t:int1:    17 4 
uint32_t:int2:    21 4 

================================

$ make run   
clang -g -o libstruct.so -shared -fPIC libstruct.c
getpharo/pharo-vm/lib/pharo/5.0-201707201942/pharo getpharo/Pharo.image ../LibStruct.st

Now manually browse to and run  LibStruct>>>LibStructTest>>#testStructs
LibStructTest>>testStructs
|good1Struct good2Struct good3Struct badStruct |
good1Struct    := GoodStruct1    new int1: 2; int2: 3.
good2Struct    := GoodStruct2    new int1: 2; int2: 3; ptr_force_int: 4.
good3Struct    := GoodStruct3    new int1: 2; int2: 3; ptr_force_int: 4.
badStruct      := BadStruct      new int1: 2; int2: 3; ptr_force_int1: 4; ptr_force_int1: 5.
self assert: (LibStruct tryGood1: good1Struct)       equals: 6.
self assert: (LibStruct tryGood2: good2Struct)       equals: 6.
self assert: (LibStruct tryGood3: good3Struct)       equals: 6.
"Problem exposed in next line"
self assert: (LibStruct tryBad: badStruct)           equals: 6.

which on console produces...

GoodStruct1:
uint32_t:int1:    01 4 = 02 00 00 00
uint32_t:int2:    05 4 = 03 00 00 00
GoodStruct2:
uint32_t:int1:    01 4 = 02 00 00 00
uint32_t:int2:    05 4 = 03 00 00 00
void*:ptr_data:    09 8 = 04 00 00 00 00 00 00 00
GoodStruct3:
void*:ptr_data:    01 8 = 04 00 00 00 00 00 00 00
uint32_t:int1:    09 4 = 02 00 00 00
uint32_t:int2:    13 4 = 03 00 00 00
BadStruct:
void*:ptr_data1:    01 8 = 00 00 00 00 00 00 00 00
void*:ptr_data2:    09 8 = 00 00 00 00 00 00 00 00
uint32_t:int1:    17 4 = 29 03 03 00
uint32_t:int2:    21 4 = 00 00 00 00

Comparing GoodStruct3 and BadStruct, it seems one pointer is handled fine, but not two.
The first time this is run after Image boots seems like BadStruct gets some random data.

================================
In same image, subsequent runs of  LibStruct>>>LibStructTest>>testStructs     
give only all zeros for BadStruct, as Todd observed.

GoodStruct1:
uint32_t:int1:    01 4 = 02 00 00 00
uint32_t:int2:    05 4 = 03 00 00 00
GoodStruct2:
uint32_t:int1:    01 4 = 02 00 00 00
uint32_t:int2:    05 4 = 03 00 00 00
void*:ptr_data:    09 8 = 00 00 00 00 00 00 00 00
GoodStruct3:
void*:ptr_data:    01 8 = 00 00 00 00 00 00 00 00
uint32_t:int1:    09 4 = 02 00 00 00
uint32_t:int2:    13 4 = 03 00 00 00
BadStruct:
void*:ptr_data1:    01 8 = 00 00 00 00 00 00 00 00
void*:ptr_data2:    09 8 = 00 00 00 00 00 00 00 00
uint32_t:int1:    17 4 = 00 00 00 00
uint32_t:int2:    21 4 = 00 00 00 00


cheers -ben

<FFI64StructTest.zip>