Hi Todd,

On Sun, Nov 26, 2017 at 8:24 AM, Todd Blanchard <tblanchard@mac.com> wrote:

 

i'm getting the idea that we should probably write a test suite/library for FFI

See the package FFI-Tests at http://source.squeak.org/FFI and the file sqFFITestFuncs.c

On Nov 24, 2017, at 12:54 AM, Ben Coman <btc@openinworld.com> wrote:

On 24 November 2017 at 13:16, Ben Coman <btc@openinworld.com> wrote:

On 22 November 2017 at 21:59, Ben Coman <btc@openinworld.com> wrote:

On 22 November 2017 at 13:38, Todd Blanchard <tblanchard@mac.com> wrote:

 

I've been trying to track this down for a couple weeks now.

I have concluded that structs passed by value to functions on the 64 bit VM are not properly populated.  The struct's memory is all zero'd.

I found this while trying to work with LibClang and found that functions that fetched code locations from code ranges always returned invalid zero'd locations.  After spending some time with lldb I have traced the problem into the native code and found that the argument is not correct.

I've carved out the wee bit of clang to reproduce this in a tiny library.

The gist of it is below and the entire file is included.  Basically the struct passed to the function clang_getRangeStart is zero'd memory regardless of the data I send from the image side.

The build command I used on sierra is clang -shared -undefined dynamic_lookup -o microclang.dylib microclang.c

On Ubuntu 16.04 I used...

$ clang -shared -fPIC -o libmicroclang.so microclang.c

$ clang test.c -L. -l microclang

   test.c:6:53: error: no member named 'begin_int_data' in 'CXSourceLocation'

   if(clang_getRangeStart(clang_getArbitraryRange()).begin_int_data == 0) 

I presume you meant...

    if(clang_getRangeStart(clang_getArbitraryRange()).int_data == 0) 

so correcting and continuing...

$ clang test.c -L. -l microclang

$ LD_LIBRARY_PATH=. ./a.out

That failed

So I'm not sure how to proceed.  

I was expecting that would work while Pharo failed.

Now interestingly...

$ clang test.c microlang.c

$ ./a.out

That worked

So it seems a similar problem exists outside our FFI. 

cheers -ben

P.S. I refactored you code to extract a header file (attached)

The issue is still beyond my ken, but I've made some progress towards isolating/understanding the issue.

Attached zip exploded here for easy reference...

___microlang.h___

typedef unsigned  uintptr_t;

typedef struct {

  const void *ptr_data[2];

} CXSourceRange_;

CXSourceRange_ clang_getArbitraryRange_();

int clang_getRangeEnd_(CXSourceRange_ range);

___microclang.c___

#include "microclang.h"

const char* libraryString = "library_pointer";

CXSourceRange_ clang_getArbitraryRange_()

{       CXSourceRange_ range = {0};

        range.ptr_data[0] = (void*)libraryString;

        return range;

}

int clang_getRangeEnd_(CXSourceRange_ range)

{       // Special decoding for CXSourceLocations for CXLoadedDiagnostics.

        if ((uintptr_t)range.ptr_data[0] & 0x1)

        {       return 0;       }

        else

        {       return 1;       }

}

___test.c___

#include <stdio.h>

#include "microclang.h"

const char* localString =  "local_pointer";

void test( CXSourceRange_ range, char *note )

{       int result = clang_getRangeEnd_(range);

        if(result == 0)

        {       printf("That failed (%s)\n", note); }

        else

        {       printf("That worked (%s)\n", note); }

}

int main()

{       CXSourceRange_ range1 = clang_getArbitraryRange_();

        test(range1, "library string");

        CXSourceRange_ range2 = {0};

        range2.ptr_data[0] = (void*)localString;

        test(range2, "local string");

}

___Makefile___

default: clean static shared

clean:

        rm -f *so *App

        @echo

shared:

        clang -g -o libmicroclang.so -shared -fPIC microclang.c

        clang -g -o sharedApp test.c -L. -lmicroclang

        LD_LIBRARY_PATH=. ./sharedApp

        @echo

static:

        clang -g -o staticApp test.c microclang.c

        ./staticApp

        @echo

Now running...

$ make > report

gives...

___report___

rm -f *so *App

clang -g -o staticApp test.c microclang.c

./staticApp

That worked (library string)

That worked (local string)

clang -g -o libmicroclang.so -shared -fPIC microclang.c

clang -g -o sharedApp test.c -L. -lmicroclang

LD_LIBRARY_PATH=. ./sharedApp

That failed (library string)

That worked (local string)

Further simplification dealing *only* with strings (see attached sharedLibString.zip)

(also attached is clang3.zip as a step along the way)

___microclang.c___

typedef unsigned  uintptr_t;

const char* myLibraryString = "library_pointer";

const char * lib_getLibraryString()

{       return myLibraryString;

}

int lib_testString( const char *aString )

{       unsigned test = (uintptr_t)aString & 0x1;

        printf("\n  test=%d, aString-->%d\n", test, (uintptr_t)aString);

        if (test)

        {       return 0;       }

        else

        {       return 1;       }

}

___test.c___

#include <stdio.h>

int lib_testString( const char *aString );

const char *lib_getLibraryString();

const char *localString =  "local_pointer";

void test( int result, char *note )

{       if(result == 0)

        {       printf("That failed (%s)\n", note); }

        else 

        {       printf("That worked (%s)\n", note); }

}

int main()

{       const char * libraryString = lib_getLibraryString();

        test(lib_testString(libraryString), "library string");

        test(lib_testString(localString), "local string");

}

$ make > report

___report___

rm -f *so *App

clang -g -o staticApp test.c microclang.c

./staticApp

  test=0, aString-->4196150

  That worked (library string)

  test=0, aString-->4196068

  That worked (local string)

clang -g -o libmicroclang.so -shared -fPIC microclang.c

clang -g -o sharedApp test.c -L. -lmicroclang

LD_LIBRARY_PATH=. ./sharedApp

  test=1, aString-->-792512599

  That failed (library string)

  test=0, aString-->4196484

  That worked (local string)

cheers -ben

<sharedLibString.zip><clang3.zip>

--

_,,,^..^,,,_

best, Eliot