Hi,
Sorry if this sounds ignorant but I am not sure how to do this with Opensmalltalk VM. How do I get a working VMMaker loaded into a Pharo image these days? It was quite simple in the pharo-vm days but now I am not even sure where to start looking?
I want to debug a FFI problem I see with Ben's clang-c bindings. I get a callback from C with three arguments and one of them is of type CXCursor (it needs to be passed on the stack):
struct CXCursor { int enum void *ptr[3]; };
It's represented as FFIExternalStructure subclass: #CXCursor. In the case of the callback the handle is an instance of FFIExternalStructureReferenceHandle with an Alien embedded into it.
I am trying to understand if the bug is on handling the callback when the CXCursor is created or when trying to push it onto the stack when making a call.
The memory on the stack is turned into an Alien. And Alien>>#referenceStructAt:length: doesn't return a ByteArray but the FFIExternalStructureReferenceHandle with the Alien in it.
At the same time I think FFIPlugin>>#ffiPushStructureContentsOf: doesn't seem to deal with FFIExternalStructureReferencehandle (or handle being an Alien).
Or do I chase it from the wrong end?
cheers holger
On Wed, 3 Apr 2019 at 22:54, Holger Freyther holger@freyther.de wrote:
Hi,
Sorry if this sounds ignorant but I am not sure how to do this with Opensmalltalk VM. How do I get a working VMMaker loaded into a Pharo image these days? It was quite simple in the pharo-vm days but now I am not even sure where to start looking?
You haven't said what you've tried. This looks updated recently... http://wiki.squeak.org/squeak/vmmaker
btw, I see an experimental git repo for VMMaker ...
https://github.com/feenkcom/opensmalltalk-vm/tree/pullrequest4/vmmaker/small...
but this is not what the production VMs are built from.
I want to debug a FFI problem I see with Ben's clang-c bindings. I get a
callback from C with three arguments and one of them is of type CXCursor (it needs to be passed on the stack):
You are referring this page then...?
https://blog.openinworld.com/2016/09/pharo-libclang-ffi-part-4-ast-walking-w...
Which part of it are you up to?
struct CXCursor {
int enum void *ptr[3];};
It's represented as FFIExternalStructure subclass: #CXCursor.
For more context for interested readers, CxCursor definition is shown here... https://blog.openinworld.com/2016/09/pharo-libclang-ffi-part-3-loading-an-as...
In the case of the callback the handle is an instance of
FFIExternalStructureReferenceHandle with an Alien embedded into it.
Wow, those posts were back in 2016. How time flies. Its vague, but I don't remember ever seeing class FFIExternalStructureReferenceHandle or Alien anywhere. I'll have to freshly re-follow the posts myself to review.
I am trying to understand if the bug is on handling the callback when the CXCursor is created or when trying to push it onto the stack when making a call.
The memory on the stack is turned into an Alien. And Alien>>#referenceStructAt:length: doesn't return a ByteArray but the FFIExternalStructureReferenceHandle with the Alien in it.
At the same time I think FFIPlugin>>#ffiPushStructureContentsOf: doesn't seem to deal with FFIExternalStructureReferencehandle (or handle being an Alien).
I'm not familiar with that part of it. Hopefully someone else can chime in. btw, did you do the whole series of post individually, or bulk load the code from... http://smalltalkhub.com/#!/~BenComan/LibclangPractice
cheers -ben
Hi Holger,
On Wed, Apr 3, 2019 at 7:36 AM Holger Freyther holger@freyther.de wrote:
Hi,
Sorry if this sounds ignorant but I am not sure how to do this with Opensmalltalk VM. How do I get a working VMMaker loaded into a Pharo image these days? It was quite simple in the pharo-vm days but now I am not even sure where to start looking?
I want to debug a FFI problem I see with Ben's clang-c bindings. I get a callback from C with three arguments and one of them is of type CXCursor (it needs to be passed on the stack):
struct CXCursor { int enum void *ptr[3]; };
It's represented as FFIExternalStructure subclass: #CXCursor. In the case of the callback the handle is an instance of FFIExternalStructureReferenceHandle with an Alien embedded into it.
What is the C signature of the callback function, and what is the source of the marshaling method in the relevant Callback subclass's signatures protocol? Also, what's the callout's signature and what is the Smalltalk code for the call?
I am trying to understand if the bug is on handling the callback when the
CXCursor is created or when trying to push it onto the stack when making a call.
The memory on the stack is turned into an Alien. And Alien>>#referenceStructAt:length: doesn't return a ByteArray but the FFIExternalStructureReferenceHandle with the Alien in it.
At the same time I think FFIPlugin>>#ffiPushStructureContentsOf: doesn't seem to deal with FFIExternalStructureReferencehandle (or handle being an Alien).
Or do I chase it from the wrong end?
Without the code I can't help, hence my questions above.
_,,,^..^,,,_ best, Eliot
On 3. Apr 2019, at 23:50, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Holger,
It's represented as FFIExternalStructure subclass: #CXCursor. In the case of the callback the handle is an instance of FFIExternalStructureReferenceHandle with an Alien embedded into it.
What is the C signature of the callback function, and what is the source of the marshaling method in the relevant Callback subclass's signatures protocol? Also, what's the callout's signature and what is the Smalltalk code for the call?
C declaration:
typedef enum CXChildVisitResult(* CXCursorVisitor) (CXCursor cursor, CXCursor parent, CXClientData client_data);
and passed to the c function below.
unsigned clang_visitChildren (CXCursor parent, CXCursorVisitor visitor, CXClientData client_data );
Smalltalk Source:
acceptCallbackFn := FFICallback signature: #( CXChildVisitResult ( CXCursor cursor, CXCursor parent, CXString client_data))
CXChildVisitResult is an enum, CXCursor/CXString are FFIExternalStructure subclasses.
Smalltalk code:
Libclang clang_visitChildren__parentCursor: rootCursor visitorCallback: acceptCallbackFn clientData: rootClientData.
#clang_visitChildren__parentCursor: parent
visitorCallback: visitor clientData: client_data ^ self ffiCall: #( uint clang_visitChildren( CXCursor parent, CXCursorVisitor visitor, CXClientData client_data))
Or do I chase it from the wrong end?
Without the code I can't help, hence my questions above.
The full code is in Ben's LibclangPractice (http://smalltalkhub.com/#!/~BenComan/LibclangPractice). I have set a breakpoint in LibclangTest>>#visitChildrenCallbackReturning: to look at the cursor variable and call it's methods. I think my Pharo PR (https://github.com/pharo-project/pharo/pull/3136/files#diff-8f4b31166c3a817d...) is fixing the callback handling in Unified-FFI.
I think I have stumbled into three separate bugs and I just noticed that they are Pharo specific. Unified-FFI doesn't seem to exist for Squeak.
1st) an Alien ending inside a handle of a FFIExternalStructure sub-instance. I have made a PR for it and I think it is reasonable to turn the Alien into an ExternalAddress early.
2nd) Pushing a FFIExternalStructure sub-instance with a FFIExternalStructureReferenceHandle in it to the stack doesn't work. Squeak+ThreadedFFIPlugin simply don't know what a FFIExternalStructureReferenceHandle valueOOP is.
3rd) Returning a struct from the callback doesn't to work either. That's a UnifiedFFI bug as well:
In FFICallbackParameterTests>>#testPassingStructureInTheStack | param | callback := FFICallback signature: #(int (FFITestStructureSmallIntFloatStructure a)) block: [ :a | self assert: a x equals: 2.0. self assert: a y equals: 3. ..
change the "int" to "FFITestStructureSmallIntFloatStructure" and the first assertion will fail. The value of "a x" becomes 3. I have not debugged this.
I think only 2nd) is relevant to this list. Support for FFIExternalStructureReferenceHandle will add one level of indirection.
Hi Holger,
On Thu, Apr 4, 2019 at 10:04 AM Holger Freyther holger@freyther.de wrote:
On 3. Apr 2019, at 23:50, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Holger,
It's represented as FFIExternalStructure subclass: #CXCursor. In the
case of the callback the handle is an instance of FFIExternalStructureReferenceHandle with an Alien embedded into it.
What is the C signature of the callback function, and what is the source
of the marshaling method in the relevant Callback subclass's signatures protocol?
Also, what's the callout's signature and what is the Smalltalk code for
the call?
C declaration:
typedef enum CXChildVisitResult(* CXCursorVisitor) (CXCursor cursor, CXCursor parent, CXClientData client_data);
and passed to the c function below.
unsigned clang_visitChildren (CXCursor parent, CXCursorVisitor visitor, CXClientData client_data );
Smalltalk Source:
acceptCallbackFn := FFICallback signature: #( CXChildVisitResult ( CXCursorcursor, CXCursor parent, CXString client_data))
somewhere in the Callback hierarchy there should me a marshaling method for the platform you're on that matches that signature. Fo example, in Squeak if I'm calling, say, sort, then there's a callback such as
Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign].
to marshal the callback's incoming arguments each platform needs a suitable marshaling method that the Callback machinery matches to the signature. Here they are from Squeak:
Callback methods for signatures voidstarvoidstarRetint: callbackContext regs: regsAlien <signature: #(int (*)(const void *, const void *))> self subclassResponsibility
CallbackForARM32 methods for signatures voidstarvoidstarRetint: callbackContext regs: regsAlien <signature: #(int (*)(const void *, const void *))> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongAt: 5)))
CallbackForWin64X64 methods for signatures voidstarvoidstarRetint: callbackContext regs: regsAlien <signature: #(int (*)(const void *, const void *))> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongLongAt: 9)))
CallbackForX64 methods for signatures voidstarvoidstarRetint: callbackContext regs: regsAlien <signature: #(int (*)(const void *, const void *))> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongLongAt: 9)))
What are is method for your platform?
CXChildVisitResult is an enum, CXCursor/CXString are FFIExternalStructure
subclasses.
Smalltalk code:
Libclang clang_visitChildren__parentCursor: rootCursor visitorCallback: acceptCallbackFn clientData: rootClientData.#clang_visitChildren__parentCursor: parent
visitorCallback: visitor clientData: client_data ^ self ffiCall: #( uint clang_visitChildren( CXCursorparent,
CXCursorVisitor visitor,
CXClientData client_data))
Or do I chase it from the wrong end?
Without the code I can't help, hence my questions above.
The full code is in Ben's LibclangPractice ( http://smalltalkhub.com/#!/~BenComan/LibclangPractice). I have set a breakpoint in LibclangTest>>#visitChildrenCallbackReturning: to look at the cursor variable and call it's methods. I think my Pharo PR ( https://github.com/pharo-project/pharo/pull/3136/files#diff-8f4b31166c3a817d...) is fixing the callback handling in Unified-FFI.
I think I have stumbled into three separate bugs and I just noticed that they are Pharo specific. Unified-FFI doesn't seem to exist for Squeak.
1st) an Alien ending inside a handle of a FFIExternalStructure sub-instance. I have made a PR for it and I think it is reasonable to turn the Alien into an ExternalAddress early.
2nd) Pushing a FFIExternalStructure sub-instance with a FFIExternalStructureReferenceHandle in it to the stack doesn't work. Squeak+ThreadedFFIPlugin simply don't know what a FFIExternalStructureReferenceHandle valueOOP is.
3rd) Returning a struct from the callback doesn't to work either. That's a UnifiedFFI bug as well:
In FFICallbackParameterTests>>#testPassingStructureInTheStack
| param | callback := FFICallback signature: #(int (FFITestStructureSmallIntFloatStructurea)) block: [ :a | self assert: a x equals: 2.0. self assert: a y equals: 3. ..
change the "int" to "FFITestStructureSmallIntFloatStructure" and the first assertion will fail. The value of "a x" becomes 3. I have not debugged this.
I think only 2nd) is relevant to this list. Support for FFIExternalStructureReferenceHandle will add one level of indirection.
On Thu, Apr 4, 2019 at 11:43 AM Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Holger,
On Thu, Apr 4, 2019 at 10:04 AM Holger Freyther holger@freyther.de wrote:
On 3. Apr 2019, at 23:50, Eliot Miranda eliot.miranda@gmail.com
wrote:
Hi Holger,
It's represented as FFIExternalStructure subclass: #CXCursor. In the
case of the callback the handle is an instance of FFIExternalStructureReferenceHandle with an Alien embedded into it.
What is the C signature of the callback function, and what is the
source of the marshaling method in the relevant Callback subclass's signatures protocol?
Also, what's the callout's signature and what is the Smalltalk code for
the call?
C declaration:
typedef enum CXChildVisitResult(* CXCursorVisitor) (CXCursor cursor, CXCursor parent, CXClientData client_data);
and passed to the c function below.
unsigned clang_visitChildren (CXCursor parent, CXCursorVisitor visitor, CXClientData client_data );
Smalltalk Source:
acceptCallbackFn := FFICallback signature: #( CXChildVisitResult ( CXCursorcursor, CXCursor parent, CXString client_data))
somewhere in the Callback hierarchy there should me a marshaling method for the platform you're on that matches that signature. Fo example, in Squeak if I'm calling, say, sort, then there's a callback such as
Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign].
to marshal the callback's incoming arguments each platform needs a suitable marshaling method that the Callback machinery matches to the signature. Here they are from Squeak:
Callback methods for signatures voidstarvoidstarRetint: callbackContext regs: regsAlien <signature: #(int (*)(const void *, const void *))> self subclassResponsibility
CallbackForARM32 methods for signatures voidstarvoidstarRetint: callbackContext regs: regsAlien <signature: #(int (*)(const void *, const void *))> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongAt: 5)))
CallbackForWin64X64 methods for signatures voidstarvoidstarRetint: callbackContext regs: regsAlien <signature: #(int (*)(const void *, const void *))> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongLongAt: 9)))
CallbackForX64 methods for signatures voidstarvoidstarRetint: callbackContext regs: regsAlien <signature: #(int (*)(const void *, const void *))> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongLongAt: 9)))
Missed one (finger trouble)
CallbackForIA32 methods for signatures voidstarvoidstarRetint: callbackContext sp: spAlien <signature: #(int (*)(const void *, const void *))> ^callbackContext wordResult: (block value: (Alien forPointer: (spAlien unsignedLongAt: 1)) value: (Alien forPointer: (spAlien unsignedLongAt: 5)))
What are is method for your platform?
CXChildVisitResult is an enum, CXCursor/CXString are FFIExternalStructure
subclasses.
Smalltalk code:
Libclang clang_visitChildren__parentCursor: rootCursor visitorCallback: acceptCallbackFn clientData: rootClientData.#clang_visitChildren__parentCursor: parent
visitorCallback: visitor clientData: client_data ^ self ffiCall: #( uint clang_visitChildren( CXCursorparent,
CXCursorVisitor visitor,
CXClientData client_data))
Or do I chase it from the wrong end?
Without the code I can't help, hence my questions above.
The full code is in Ben's LibclangPractice ( http://smalltalkhub.com/#!/~BenComan/LibclangPractice). I have set a breakpoint in LibclangTest>>#visitChildrenCallbackReturning: to look at the cursor variable and call it's methods. I think my Pharo PR ( https://github.com/pharo-project/pharo/pull/3136/files#diff-8f4b31166c3a817d...) is fixing the callback handling in Unified-FFI.
I think I have stumbled into three separate bugs and I just noticed that they are Pharo specific. Unified-FFI doesn't seem to exist for Squeak.
1st) an Alien ending inside a handle of a FFIExternalStructure sub-instance. I have made a PR for it and I think it is reasonable to turn the Alien into an ExternalAddress early.
2nd) Pushing a FFIExternalStructure sub-instance with a FFIExternalStructureReferenceHandle in it to the stack doesn't work. Squeak+ThreadedFFIPlugin simply don't know what a FFIExternalStructureReferenceHandle valueOOP is.
3rd) Returning a struct from the callback doesn't to work either. That's a UnifiedFFI bug as well:
In FFICallbackParameterTests>>#testPassingStructureInTheStack
| param | callback := FFICallback signature: #(int (FFITestStructureSmallIntFloatStructurea)) block: [ :a | self assert: a x equals: 2.0. self assert: a y equals: 3. ..
change the "int" to "FFITestStructureSmallIntFloatStructure" and the first assertion will fail. The value of "a x" becomes 3. I have not debugged this.
I think only 2nd) is relevant to this list. Support for FFIExternalStructureReferenceHandle will add one level of indirection.
-- _,,,^..^,,,_ best, Eliot
vm-dev@lists.squeakfoundation.org
-
Ben Coman -
Eliot Miranda -
Holger Freyther