The comment is mainly a note to myself, but by way of explanation, if you allocate a new object (such as a String that you intend to use within the plugin), the act of allocating that object may cause Squeak to run its garbage collector to make space for the new string object, which moves the objects in object memory, potentially making OOP references in your plugin invalid. Thus my intent was that #transientCStringFromString: should be used only if the null-terminated string buffer is going to be used while the primitive is run, and the comment is a reminder to use #pushRemappableOop: and #popRemappableOop to protect against the garbage collector moving OOPs in the plugin.
Setting that issue aside for the moment, what you are probably trying to do here is just convert the Squeak string into a (char *) that will be valid for use in the external C library. For this you can malloc a buffer for the null-terminated string (because the buffer will stay allocated after your primitive exits, even if it is being used elsewhere by an external library), or you can use a statically allocated buffer in your primitive if you know in advance the maximum length of the string.
I recall that Eliot Miranda suggested using alloca() rather than malloc(), as this avoids the need to free allocated memory later. This is probably the best approach overall if the library function runs to completion as part of your primitive (in other words, as long as the function does not start some new thread that makes use of the string after your primitive has completed and returned control to Squeak).
To summarize: If you want to pass a Smalltalk string of length 10 to a primitive, copy the 10 characters in the string into a new buffer of length 11 with the 11th byte set to zero, then pass that buffer to the C function.
Dave
On Mon, Sep 14, 2009 at 07:36:41PM -0700, Ronald Spengler wrote:
Thanks Dave! That helps a lot. This snippet scares me a little bit though: "in a section of code in which the garbage collector is guaranteed not to run"
I realize now that:
- I don't know how to guarantee that the gc won't run
- My C library will take it's sweet time running, and it's runtime is
a function of it's input, could be forever in the extreme case.
So, to be safe, on the C side of things, should I copy the string ASAP? Or does C code escape the garbage collector? Is it safe to malloc()?
Thanks again for your help, and please forgive my ignorance.
On Sun, Sep 13, 2009 at 6:56 PM, David T. Lewis lewis@mail.msen.com wrote:
On Sun, Sep 13, 2009 at 01:23:32PM -0700, Ronald Spengler wrote:
Hello everyone. I have a named primitive, and I need to send a ByteString to it, to be processed and returned by an external library. To get a string into Slang, should I send it #asByteArray, and would that let me treat the bytes as integers on the stack? I'm basically trying to get a char* on the other side.
You can use the ByteString as a parameter to the primitive, no problem. The only tricky bit is that C expects null terminated strings, so you need to copy the contents of the ByteString into a null terminated array before you can let it be used by the C library as a char *.
I'm sure there are lots of examples, but you can look at OSProcessPlugin>>cStringFromString: and OSProcess>>transientCStringFromString: for examples of how to copy the string buffer into a null terminated buffer for use in C. Look at senders of these two methods for examples of primitives that pass strings as parameters. (OSProcessPlugin is on SqueakSource if you do not have it).
Following are a couple of examples taken from OSPP. In both cases, a buffer is allocated with size one greater than the string length, and the contents of the Smalltalk string are copied into the buffer space with a trailing null terminator. The #primitiveChdir example allocates a new Smalltalk string to use for the buffer, and #primitivePutEnv uses malloc to allocate the new buffer (because in this case the buffer must be "permanently" valid after the primitive exits).
primitiveChdir ? ? ? ?"Call chdir(2) to change current working directory to the specified path string. Answer ? ? ? ?nil for success, or errno on failure."
? ? ? ?| path errno | ? ? ? ?self export: true. ? ? ? ?self var: 'path' type: 'char *'. ? ? ? ?self var: 'errno' type: 'extern int'. ? ? ? ?path := self transientCStringFromString: (interpreterProxy stackObjectValue: 0). ? ? ? ?(self chdir: path) ? ? ? ? ? ? ? ?ifTrue: [interpreterProxy pop: 2; push: interpreterProxy nilObject] ? ? ? ? ? ? ? ?ifFalse: [interpreterProxy pop: 2; pushInteger: errno].
transientCStringFromString: aString ? ? ? ?"Answer a new null-terminated C string copied from aString. ? ? ? ?The string is allocated in object memory, and will be moved ? ? ? ?without warning by the garbage collector. Any C pointer ? ? ? ?reference the the result is valid only until the garbage ? ? ? ?collector next runs. Therefore, this method should only be used ? ? ? ?within a single primitive in a section of code in which the ? ? ? ?garbage collector is guaranteed not to run. Note also that ? ? ? ?this method may itself invoke the garbage collector prior ? ? ? ?to allocating the new C string.
? ? ? ?Warning: The result of this method will be invalidated by the ? ? ? ?next garbage collection, including a GC triggered by creation ? ? ? ?of a new object within a primitive. Do not call this method ? ? ? ?twice to obtain two string pointers."
? ? ? ?| len stringPtr newString cString | ? ? ? ?self returnTypeC: 'char *'. ? ? ? ?self var: 'stringPtr' declareC: 'char *stringPtr'. ? ? ? ?self var: 'cString' declareC: 'char *cString'. ? ? ? ?len := interpreterProxy sizeOfSTArrayFromCPrimitive: (interpreterProxy arrayValueOf: aString). ? ? ? ?"Allocate space for a null terminated C string." ? ? ? ?interpreterProxy pushRemappableOop: aString. ? ? ? ?newString := interpreterProxy ? ? ? ? ? ? ? ?instantiateClass: interpreterProxy classString ? ? ? ? ? ? ? ?indexableSize: len + 1. ? ? ? ?stringPtr := interpreterProxy arrayValueOf: interpreterProxy popRemappableOop. ? ? ? ?cString := interpreterProxy arrayValueOf: newString. ? ? ? ? ? ?"Point to the actual C string." ? ? ? ?self cCode: '(char *)strncpy(cString, stringPtr, len)'. ? ? ? ? "Make a copy of the string." ? ? ? ?cString at: (len) put: 0. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "Null terminate the C string." ? ? ? ?^ cString
primitivePutEnv ? ? ? ?"Set an environment variable using a string of the form 'KEY=value'. This ? ? ? ?implementation allocates a C string using malloc to allocate from the C heap ? ? ? ?(using cStringFromString rather than transientCStringFromString). This ? ? ? ?is necessary because the C runtime library does not make a copy of the ? ? ? ?string into separately allocated environment memory."
? ? ? ?| cStringPtr keyValueString | ? ? ? ?self export: true. ? ? ? ?self var: 'cStringPtr' declareC: 'char *cStringPtr'. ? ? ? ?keyValueString := interpreterProxy stackObjectValue: 0. ? ? ? ?cStringPtr := self cStringFromString: keyValueString. ? ? ? ?((self putenv: cStringPtr) == 0) ? ? ? ?"Set environment variable." ? ? ? ? ? ? ? ?ifTrue: [interpreterProxy pop: 2; push: keyValueString] ? ? ? ? ? ? ? ?ifFalse: [^ interpreterProxy primitiveFail]
cStringFromString: aString ? ? ? ?"Answer a new null-terminated C string copied from aString. The C string ? ? ? ?is allocated from the C runtime heap. See transientCStringFromString for ? ? ? ?a version which allocates from object memory. ? ? ? ?Caution: This may invoke the garbage collector."
? ? ? ?| len sPtr cString | ? ? ? ?self returnTypeC: 'char *'. ? ? ? ?self var: 'sPtr' declareC: 'char *sPtr'. ? ? ? ?self var: 'cString' declareC: 'char *cString'. ? ? ? ?sPtr := interpreterProxy arrayValueOf: aString. ? ? ? ?len := interpreterProxy sizeOfSTArrayFromCPrimitive: sPtr. ? ? ? ?cString := self callocWrapper: len + 1 size: 1. ? ? ? ? "Space for a null terminated C string." ? ? ? ?self cCode: '(char *) strncpy (cString, sPtr, len)'. ? ?"Copy the string." ? ? ? ?^ cString
callocWrapper: count size: objectSize ? ? ? ?"Using malloc() and calloc() is something I would like to avoid, since it is ? ? ? ?likely to cause problems some time in the future if somebody redesigns ? ? ? ?object memory allocation. This wrapper just makes it easy to find senders ? ? ? ?of calloc() in my code. -dtl"
? ? ? ?self returnTypeC: 'void *'. ? ? ? ?^ self cCode: 'calloc(count, objectSize)'
Dave
-- Ron