Hello all,
I'm currently working on a TLS / SSL implementation and on windows I am planning to support the Certificate Store that is built in. This requires me to use the CryptoAPI dlls. The code is complex and it uses a number of header files and many steps. I built some C++ code that does what I'm looking for so now I would like to incorporate this into Squeak.
I'd assumed that FFI was like DDL/C connect in VW but that's not what I'm seeing. So my question is: What is the best way to get this working? Should I hack away at FFI and support the api directly, and is that even possible? Should I create DLL's to call and use them with FFI, or should I compile some sort of plugins? My preference would be to call the api directly without having to create dll's, duplicating my C++ code in Smalltalk if possible.
Some direction and comparisons of different options would be very helpful.
Thank you for your help!
Ron Teitelbaum
Well I've done all three, or is that four?
a) FFI class via platform specific instances interfacing to generic classes that a squeak user interacts with. b) FFI class/subclasses that are modified on the fly depending on platform to get module name correct and or provide overrides by platform, lots of this in Croquet. c) wrapper classes that use plugin code that lightly wraps api, support work all done in Smalltalk, I did this for the extended serial support. d) Generic classes that wrap plugin code that interfaces to platform specific code, lots of this in the VM
Issues. Generally getting folks to setup a development system, get VMMaker, understand plugins, build and test something is only done by a minority of squeakers.
In using FFI, mapping the parms/return values to the correct Smalltalk external objects will at some point drive you crazy, it does all work, but sometimes late at night you wonder. For the most part we do this in Sophie, any one can do FFI, but debugging is hard.
Using lightly wrap APIs, that provides a bit clearer, in my opinion, parameter specifications that then invoke the api. You usually can use the development system source debugger to halt and debug at the api inteface. It's a bit easy for folks to assemble features since they can code the steps in smalltalk and interface to the apis, new apis require someone to mangle plugin code and compile plugins. Technically it's very similar to FFI, but avoids FFI.
Using fully abstracted plugin with platform specific code, well that's hard, and few folks want to participate. You can of course write all the code via plugin code, I believe I do this for the mac menu bar plugin since it's platform specific, there is no platform C code, everything is done in squeak slang then compiled into the VM as the mac menu bar plugin (used in Sophie btw).
On 29-Jul-06, at 9:15 PM, Ron Teitelbaum wrote:
Hello all,
I’m currently working on a TLS / SSL implementation and on windows I am planning to support the Certificate Store that is built in. This requires me to use the CryptoAPI dlls. The code is complex and it uses a number of header files and many steps. I built some C ++ code that does what I’m looking for so now I would like to incorporate this into Squeak.
I’d assumed that FFI was like DDL/C connect in VW but that’s not what I’m seeing. So my question is: What is the best way to get this working? Should I hack away at FFI and support the api directly, and is that even possible? Should I create DLL’s to call and use them with FFI, or should I compile some sort of plugins? My preference would be to call the api directly without having to create dll’s, duplicating my C++ code in Smalltalk if possible.
Some direction and comparisons of different options would be very helpful.
Thank you for your help!
Ron Teitelbaum
-- ======================================================================== === John M. McIntosh johnmci@smalltalkconsulting.com Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===
John,
Thank you very much for your response, it appears that you support DLL's to wrap and simplify Api's. This is doable but I wonder about maintainability and prefer not to push code outside of Smalltalk if possible for that reason. Although, I'm sure that it is the easier way to go, and probably the path I will take. Also given the cryptographic nature of the code that I'm writing and the need eventually to have the code certified, adding another level of code between the MS API and Squeak leaves another layer to protect.
Thanks again for giving me more to consider,
Ron Teitelbaum
From: John M McIntosh Sent: Sunday, July 30, 2006 2:26 AM
Well I've done all three, or is that four?
a) FFI class via platform specific instances interfacing to generic classes that a squeak user interacts with. b) FFI class/subclasses that are modified on the fly depending on platform to get module name correct and or provide overrides by platform, lots of this in Croquet. c) wrapper classes that use plugin code that lightly wraps api, support work all done in Smalltalk, I did this for the extended serial support. d) Generic classes that wrap plugin code that interfaces to platform specific code, lots of this in the VM
Issues. Generally getting folks to setup a development system, get VMMaker, understand plugins, build and test something is only done by a minority of squeakers.
In using FFI, mapping the parms/return values to the correct Smalltalk external objects will at some point drive you crazy, it does all work, but sometimes late at night you wonder. For the most part we do this in Sophie, any one can do FFI, but debugging is hard.
Using lightly wrap APIs, that provides a bit clearer, in my opinion, parameter specifications that then invoke the api. You usually can use the development system source debugger to halt and debug at the api inteface. It's a bit easy for folks to assemble features since they can code the steps in smalltalk and interface to the apis, new apis require someone to mangle plugin code and compile plugins. Technically it's very similar to FFI, but avoids FFI.
Using fully abstracted plugin with platform specific code, well that's hard, and few folks want to participate. You can of course write all the code via plugin code, I believe I do this for the mac menu bar plugin since it's platform specific, there is no platform C code, everything is done in squeak slang then compiled into the VM as the mac menu bar plugin (used in Sophie btw).
On 29-Jul-06, at 9:15 PM, Ron Teitelbaum wrote:
Hello all,
I'm currently working on a TLS / SSL implementation and on windows I am planning to support the Certificate Store that is built in. This requires me to use the CryptoAPI dlls. The code is complex and it uses a number of header files and many steps. I built some C ++ code that does what I'm looking for so now I would like to incorporate this into Squeak.
I'd assumed that FFI was like DDL/C connect in VW but that's not what I'm seeing. So my question is: What is the best way to get this working? Should I hack away at FFI and support the api directly, and is that even possible? Should I create DLL's to call and use them with FFI, or should I compile some sort of plugins? My preference would be to call the api directly without having to create dll's, duplicating my C++ code in Smalltalk if possible.
Some direction and comparisons of different options would be very helpful.
Thank you for your help!
Ron Teitelbaum
--
=== John M. McIntosh johnmci@smalltalkconsulting.com Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===
Ron Teitelbaum wrote:
I’d assumed that FFI was like DDL/C connect in VW but that’s not what I’m seeing. So my question is: What is the best way to get this working?
My first question is: What is it that you're trying to get working? My second one: In which way is the FFI not like DLL/C connect in VW and why would that matter?
Cheers, - Andreas
Andreas,
I'm supporting the certificate store of windows. I will be reading and writing certificates to be used for SSL/TLS. The reason for this is to simplify some of the tasks needed to support SSL and to support what is already perceived to be a safe storage method in windows. (I am also thinking about supporting LDAP for storage instead but that is down the road, first things first)
As for the VW dll/c connect code. I found the ability to set up libs and header files and automatically generate code very useful. Looking at the FFI kernel it appears that the pieces are there for me to assemble manually, which I why I asked the question if it is possible. What concerns me most is having to dissect the 5 or 6 very large header files that are needed to support the few api calls that are made.
Are there some good examples of more complex FFI calls and structures, or easier ways to support many different required header files? I haven't downloaded the VMMaker or looked at slang, do you suggest I look at that first before deciding on which path to take?
Thanks for you help,
Ron Teitelbaum
-----Original Message----- From: squeak-dev-bounces@lists.squeakfoundation.org [mailto:squeak-dev- bounces@lists.squeakfoundation.org] On Behalf Of Andreas Raab Sent: Sunday, July 30, 2006 4:07 AM To: The general-purpose Squeak developers list Subject: Re: Suggestions for C++ or DLL code
Ron Teitelbaum wrote:
I'd assumed that FFI was like DDL/C connect in VW but that's not what I'm seeing. So my question is: What is the best way to get this working?
My first question is: What is it that you're trying to get working? My second one: In which way is the FFI not like DLL/C connect in VW and why would that matter?
Cheers,
- Andreas
Ron Teitelbaum wrote:
As for the VW dll/c connect code. I found the ability to set up libs and header files and automatically generate code very useful.
Yes, it is useful. And no, the FFI doesn't do header file parsing. I think there was some talk about using SWIG for dealing with this but I don't know if there is a Squeak backend to it yet.
Looking at the FFI kernel it appears that the pieces are there for me to assemble manually, which I why I asked the question if it is possible. What concerns me most is having to dissect the 5 or 6 very large header files that are needed to support the few api calls that are made.
If you only have a few api calls, you may just support the structures you need for those FFI calls.
Are there some good examples of more complex FFI calls and structures, or easier ways to support many different required header files? I haven't downloaded the VMMaker or looked at slang, do you suggest I look at that first before deciding on which path to take?
No, there really isn't much support for it. And none too many complex examples. Personally I only use the FFI for OpenGL and here we punt by implementing those functions that deal with complex data structures in primitives and only expose the ones with the trivial interfaces. If that is an option for you, it's a very useful one.
Cheers, - Andreas
Thank you, that is helpful.
Ron
From: Andreas Raab Sent: Monday, July 31, 2006 3:22 AM
Personally I only use the FFI for OpenGL and here we punt by implementing those functions that deal with complex data structures in primitives and only expose the ones with the trivial interfaces. If that is an option for you, it's a very useful one.
Cheers,
- Andreas
Am 31.07.2006 um 09:22 schrieb Andreas Raab:
Ron Teitelbaum wrote:
As for the VW dll/c connect code. I found the ability to set up libs and header files and automatically generate code very useful.
Yes, it is useful. And no, the FFI doesn't do header file parsing. I think there was some talk about using SWIG for dealing with this but I don't know if there is a Squeak backend to it yet.
I don't think so. But there is a generic XML backend for SWIG that looked promising last time I tried it. Should be straight-forward to generate FFI declarations from that.
- Bert -
Andreas and John,
Ok that was a lot more fun then I thought it would be. And I understand what John was saying, you pull your hair out until it actually works. When it did it was quite fun!
From: John M McIntosh In using FFI, mapping the parms/return values to the correct Smalltalk external objects will at some point drive you crazy, it does all work, but sometimes late at night you wonder.
I parsed out the parameters and used apicall and it WORKED!! FFI is pretty powerful, I really like the form shifting it is much easier the VW!! I hadn't realized that you could use structures as parameters, it's very cool.
One of the things that really made it easier is I wrote a C++ app that just prints out the final values of the parameters, it's a hack to keep from having to look everything up.
Thank you for your help getting it to work!!
From: Andreas Raab Sent: Monday, July 31, 2006 3:22 AM
If you only have a few api calls, you may just support the structures you need for those FFI calls.>
Am 30.07.2006 um 06:15 schrieb Ron Teitelbaum:
Hello all,
I’m currently working on a TLS / SSL implementation and on windows I am planning to support the Certificate Store that is built in. This requires me to use the CryptoAPI dlls.
Well ... FFI is inherently unsafe - it allows to call any C function in any library. So depending on the area of application this might be a problem, since you're giving up the relative safety of a VM-based system. For example, if the VM's sandbox is enabled, FFI calls are disallowed, so you could not use crypto in a sandboxed environment.
- Bert -
Bert,
Thank you for your comments, I wasn't planning on using the sandbox since there are a number of external connections like Glorp-PostgreSQL needed where I plan to ultimately use this new functionality. I was hoping to address this with a signed image and VM. A signed image and VM could keep any changes out of the system.
The thing that concerns me is changes to a live system that happen without needing to save the image. Plenty can happen as the system is running so the question is: how can we secure the image while it is running?
Some things that would be needed off the top of my head are:
1) Disable file in 2) Disable the compiler for accepting or performing new code, but allow it for already coded performs. (I'm sure this is much easier said then done). 3) Disable interactive debugger. 4) Enable some sort of class level MAC checking which can verify that no changes were made to code before it is executed. 5) Prevent changes to compiled methods
Plus more that I'm sure will also be needed. If we can sign the image and VM and change the VM to check the signature before staring up and find a way to protect the live image then we will have gone a long way to having a more secure development platform.
I've mentioned this before and got absolutely no comment from anyone. (Maybe the email was too long to read?)
Some input would be very much appreciated.
Ron Teitelbaum
From: Bert Freudenberg Sent: Monday, July 31, 2006 12:37 PM Am 30.07.2006 um 06:15 schrieb Ron Teitelbaum:
Hello all,
I'm currently working on a TLS / SSL implementation and on windows I am planning to support the Certificate Store that is built in. This requires me to use the CryptoAPI dlls.
Well ... FFI is inherently unsafe - it allows to call any C function in any library. So depending on the area of application this might be a problem, since you're giving up the relative safety of a VM-based system. For example, if the VM's sandbox is enabled, FFI calls are disallowed, so you could not use crypto in a sandboxed environment.
- Bert -
On 31-Jul-06, at 1:18 PM, Ron Teitelbaum wrote:
- Disable the compiler for accepting or performing new code, but
allow it for already coded performs. (I'm sure this is much easier said then done).
If you're using the same terminology that I'm used to, you don't need the compiler to execute #perform: type messages. All that does is tell the vm to pretend that the relevant message had been sent in the usual way. Similarly for blocks.
Hmm, I should probably qualify that with a "unless someone has recently massively changed the system in a way that would render them a target of the 'Ninjas of Smalltalk Doom'".
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: SFA: Seek Financial Assistance
Tim,
Shall I take this to mean that removing the compiler is possible and would be enough to prevent new code from entering the system? Or am I stuck with programming immunoglobulins that search and destroy programs as soon as any new bytes are introduced?
Ron
From: tim Rowledge Sent: Monday, July 31, 2006 4:35 PM
On 31-Jul-06, at 1:18 PM, Ron Teitelbaum wrote:
- Disable the compiler for accepting or performing new code, but
allow it for already coded performs. (I'm sure this is much easier said then done).
If you're using the same terminology that I'm used to, you don't need the compiler to execute #perform: type messages. All that does is tell the vm to pretend that the relevant message had been sent in the usual way. Similarly for blocks.
Hmm, I should probably qualify that with a "unless someone has recently massively changed the system in a way that would render them a target of the 'Ninjas of Smalltalk Doom'".
tim
tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: SFA: Seek Financial Assistance
On 31-Jul-06, at 1:43 PM, Ron Teitelbaum wrote:
Tim,
Shall I take this to mean that removing the compiler is possible
Certainly should be.
and would be enough to prevent new code from entering the system?
Ah, well since one could install compiled methods from a SmartRefStream or by loading a project (neither of which need the compiler so far as I recall), or indeed by assembling one from a simple array of bytes, I suspect it will take a bit more to be really secure.
You could for example tweak Behavior addSelector:withMethod: (and maybe others? I hope not) so that no new method can be installed in a method dictionary and also remove methods like Object withArgs:executeMethod: that will run a compiled method directly.
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: NBR: Unconditional No BRanch
Ron Teitelbaum wrote:
...
The thing that concerns me is changes to a live system that happen without needing to save the image. Plenty can happen as the system is running so the question is: how can we secure the image while it is running?
Some things that would be needed off the top of my head are:
- Disable file in
- Disable the compiler for accepting or performing new code, but allow it
for already coded performs. (I'm sure this is much easier said then done). 3) Disable interactive debugger. 4) Enable some sort of class level MAC checking which can verify that no changes were made to code before it is executed. 5) Prevent changes to compiled methods
Plus more that I'm sure will also be needed. If we can sign the image and VM and change the VM to check the signature before staring up and find a way to protect the live image then we will have gone a long way to having a more secure development platform.
I've mentioned this before and got absolutely no comment from anyone. (Maybe the email was too long to read?)
Some unstructured ideas from the top of my head: Securing a Smalltalk image is pretty difficult. The VM protects the system against a number of security holes such as buffer overflows etc. which would allow external attackers to compromise security. Protecting against internal attacks is much more difficult. If I were a computer security expert, I'd probably prefer an architecture where there is absolutely no possibility of adding new code to a running application. The Smalltalk image is *designed* to be the exact opposite of that :-) A truly secure VM/image combo would perhaps need to be done differently: The image should be split up into a read-only part containing classes and methods, and a writable part where the more mundane objects reside. Ideally, the read-only part would be protected using the hardware and OS mechanisms, so its integrity does not depend on the correctness of the VM and all its primitives. Of course, none of the development tools are supposed to be present in such an image, and even the hooks which they use should be carefully removed. There are some primitives (such as #become: and class-changing primitives) which a secure VM should either not implement or implement in a safer way. Checking the code integrity with checksums etc. is certainly a necessity. Linking to and depending on external code without checking its integrity as well is probably a no-no. Static linking is probably ok.
In any case, soothing the paranoia of security experts is going to be a difficult task. Once you know how many possibilities there are for tampering with software, you understand why...
Cheers, Hans-Martin
Hans-Martin,
Thank you for your thoughts!
I'm not sure the separation of code to isolate read only classes would be of much help since most of the problem lives in the memory model not in the stored code on disk. I would think that placing a certificate on the image file would be good enough "IF" we could prevent changes to the object structure and code in memory.
#become is a real problem since it is a very necessary part of proxy handling for almost all db interfaces I've worked with (including GLORP).
I wonder how much of this could be solved by on the fly checksums for everything. A few years ago this would have been impossible but with today performance is it really out of the question?
I agree with the need to remove the development tools.
As for the appeasing the security hacks: It's a nice challenge and I can tell you that "good enough security" is all anyone can ask for. If we set the bar too high we prevent advances in security that will do nothing but help to make a bad situation better.
Ron Teitelbaum
From: Hans-Martin Mosner Sent: Monday, July 31, 2006 4:50 PM
Ron Teitelbaum wrote:
...
The thing that concerns me is changes to a live system that happen
without
needing to save the image. Plenty can happen as the system is running so the question is: how can we secure the image while it is running?
Some things that would be needed off the top of my head are:
- Disable file in
- Disable the compiler for accepting or performing new code, but allow
it
for already coded performs. (I'm sure this is much easier said then
done).
- Disable interactive debugger.
- Enable some sort of class level MAC checking which can verify that no
changes were made to code before it is executed. 5) Prevent changes to compiled methods
Plus more that I'm sure will also be needed. If we can sign the image
and
VM and change the VM to check the signature before staring up and find a
way
to protect the live image then we will have gone a long way to having a
more
secure development platform.
I've mentioned this before and got absolutely no comment from anyone. (Maybe the email was too long to read?)
Some unstructured ideas from the top of my head: Securing a Smalltalk image is pretty difficult. The VM protects the system against a number of security holes such as buffer overflows etc. which would allow external attackers to compromise security. Protecting against internal attacks is much more difficult. If I were a computer security expert, I'd probably prefer an architecture where there is absolutely no possibility of adding new code to a running application. The Smalltalk image is *designed* to be the exact opposite of that :-) A truly secure VM/image combo would perhaps need to be done differently: The image should be split up into a read-only part containing classes and methods, and a writable part where the more mundane objects reside. Ideally, the read-only part would be protected using the hardware and OS mechanisms, so its integrity does not depend on the correctness of the VM and all its primitives. Of course, none of the development tools are supposed to be present in such an image, and even the hooks which they use should be carefully removed. There are some primitives (such as #become: and class-changing primitives) which a secure VM should either not implement or implement in a safer way. Checking the code integrity with checksums etc. is certainly a necessity. Linking to and depending on external code without checking its integrity as well is probably a no-no. Static linking is probably ok.
In any case, soothing the paranoia of security experts is going to be a difficult task. Once you know how many possibilities there are for tampering with software, you understand why...
Cheers, Hans-Martin
On 31-Jul-06, at 1:50 PM, Hans-Martin Mosner wrote:
Some unstructured ideas from the top of my head: Securing a Smalltalk image is pretty difficult. The VM protects the system against a number of security holes such as buffer overflows etc. which would allow external attackers to compromise security.
Ah, I'll note that the squeak VM really hasn't been hardened against attack, it's much less paranoid than the VW VM. In many places we might pass a ByteArray and a length, where the length is calculated from the ByteArray in Smalltalk however nothing prevents someone from making that VM call with a bogus ByteArray and length and see if something interesting will happen. Of course if the host operating system API provides some interesting side effect ,when passing correctly constructed information from our viewpoint, the VM won't prevent attack.
Really *all* VM entry point would need to be looked at in a proper audit to avoid buffer overflow issues, even perhaps accidents which generally are fatal.
Yes yes, someday I promised making a list of the entry points, however workload seems to be stalling that event. Perhaps someone would be interested? That also needs to be done in order to create a set of SUnits so we can enable some degree of cross platform testing and help people who want to build a VM on a new platform.
-- ======================================================================== === John M. McIntosh johnmci@smalltalkconsulting.com Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===
Hi--
John writes:
Yes yes, someday I promised making a list of the entry points, however workload seems to be stalling that event. Perhaps someone would be interested?
I also intend to do this eventually. I would do it now if it were paid.
-C
John M McIntosh wrote:
Ah, I'll note that the squeak VM really hasn't been hardened against attack, it's much less paranoid than the VW VM. In many places we might pass a ByteArray and a length, where the length is calculated from the ByteArray in Smalltalk however nothing prevents someone from making that VM call with a bogus ByteArray and length and see if something interesting will happen.
Which places are that?
Cheers, - Andreas
mmm, in looking it seems we check length versus data in all the places a cursory scan show, but.
What if I pass -1 as width to primitiveWrite24BmpLine
or perhaps pass something odd to primAECoerceDesc: typeCode to: result.
On 31-Jul-06, at 10:49 PM, Andreas Raab wrote:
John M McIntosh wrote:
Ah, I'll note that the squeak VM really hasn't been hardened against attack, it's much less paranoid than the VW VM. In many places we might pass a ByteArray and a length, where the length is calculated from the ByteArray in Smalltalk however nothing prevents someone from making that VM call with a bogus ByteArray and length and see if something interesting will happen.
Which places are that?
Cheers,
- Andreas
-- ======================================================================== === John M. McIntosh johnmci@smalltalkconsulting.com Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===
John M McIntosh wrote:
mmm, in looking it seems we check length versus data in all the places a cursory scan show, but.
What if I pass -1 as width to primitiveWrite24BmpLine
Yes, this would cause problems. Report and fix available at:
http://bugs.impara.de/view.php?id=4360
or perhaps pass something odd to primAECoerceDesc: typeCode to: result.
Don't know. Given that this is Applescript it should probably be treated like the FFI, e.g., not be used in any environment where we care about random access to the OS by untrusted code.
Cheers, - Andreas
On Tue, 01 Aug 2006 00:19:00 +0200, John M McIntosh wrote: ...
Really *all* VM entry point would need to be looked at in a proper audit to avoid buffer overflow issues, even perhaps accidents which generally are fatal.
If InterpreterSimulatorXyz where up-to-date and usable, whatever the VM uses from the host OS can easily be looked up. The list below is from a 3.7-5989 simulation run by a 3.7-1 VM (itself ;-) which was compiled with the Goya extension. And if it had TestCases for the full VM->OS interface, the list would've been complete.
/Klaus
---------------------
InterpreterSimulator 30 ioGetNextEvent: 1 1 by Interpreter>>#primitiveGetNextEvent InterpreterSimulator 30 ioSeconds 4 1 by Interpreter>>#primitiveSecondsClock InterpreterSimulator 30 ioMSecs 4 1 by Interpreter>>#checkForInterrupts by Interpreter>>#primitiveExternalCall by Interpreter>>#primitiveMillisecondClock InterpreterSimulator 30 ioSetInputSemaphore: 1 1 by Interpreter>>#primitiveInputSemaphore InterpreterSimulator 30 vmPathSize 4 1 by Interpreter>>#primitiveVMPath InterpreterSimulator 30 showDisplayBits:w:h:d:left:right:top:bottom: 6 1 by Interpreter>>#displayBitsOf:Left:Top:Right:Bottom: actually ioShowDisplay(...) InterpreterSimulator 30 ioMicroMSecs 4 1 by ObjectMemory>>#fullGC by ObjectMemory>>#incrementalGC InterpreterSimulator 30 ioHasDisplayDepth: 4 1 by Interpreter>>#primitiveTestDisplayDepth InterpreterSimulator 30 sqMemoryExtraBytesLeft: 0 1 by ObjectMemory>>#bytesLeft: InterpreterSimulator 30 vmPathGet:Length: 5 1 by Interpreter>>#primitiveVMPath InterpreterSimulator 30 ioLowResMSecs 4 1 by Interpreter>>#primitiveExternalCall BitBltSimulation 31 lockSurfaces 12 1 by self>>#copyBits by self>>#warpBits InterpreterSimulator 30 sqFileAtEnd: 4 2 by FilePlugin>>#primitiveFileAtEnd actually sqFileAtEnd() InterpreterSimulator 30 primitiveFileOpen 15 2 by StandardFileStream>>#primOpen:writable: InterpreterSimulator 30 sqFileClose: 4 2 by FilePlugin>>#primitiveFileClose InterpreterSimulatorLSB 30 nextLongFrom: 12 2 via Interpreter>>#readImageFromFile:HeapSize:StartingAt:
Klaus,
Thank you for the starting point.
Ron Teitelbaum
From: Klaus D. Witzel Sent: Tuesday, August 01, 2006 8:35 AM
On Tue, 01 Aug 2006 00:19:00 +0200, John M McIntosh wrote: ...
Really *all* VM entry point would need to be looked at in a proper audit to avoid buffer overflow issues, even perhaps accidents which generally are fatal.
If InterpreterSimulatorXyz where up-to-date and usable, whatever the VM uses from the host OS can easily be looked up. The list below is from a 3.7-5989 simulation run by a 3.7-1 VM (itself ;-) which was compiled with the Goya extension. And if it had TestCases for the full VM->OS interface, the list would've been complete.
/Klaus
InterpreterSimulator 30 ioGetNextEvent: 1 1 by Interpreter>>#primitiveGetNextEvent InterpreterSimulator 30 ioSeconds 4 1 by Interpreter>>#primitiveSecondsClock InterpreterSimulator 30 ioMSecs 4 1 by Interpreter>>#checkForInterrupts by Interpreter>>#primitiveExternalCall by Interpreter>>#primitiveMillisecondClock InterpreterSimulator 30 ioSetInputSemaphore: 1 1 by Interpreter>>#primitiveInputSemaphore InterpreterSimulator 30 vmPathSize 4 1 by Interpreter>>#primitiveVMPath InterpreterSimulator 30 showDisplayBits:w:h:d:left:right:top:bottom: 6 1 by Interpreter>>#displayBitsOf:Left:Top:Right:Bottom: actually ioShowDisplay(...) InterpreterSimulator 30 ioMicroMSecs 4 1 by ObjectMemory>>#fullGC by ObjectMemory>>#incrementalGC InterpreterSimulator 30 ioHasDisplayDepth: 4 1 by Interpreter>>#primitiveTestDisplayDepth InterpreterSimulator 30 sqMemoryExtraBytesLeft: 0 1 by ObjectMemory>>#bytesLeft: InterpreterSimulator 30 vmPathGet:Length: 5 1 by Interpreter>>#primitiveVMPath InterpreterSimulator 30 ioLowResMSecs 4 1 by Interpreter>>#primitiveExternalCall BitBltSimulation 31 lockSurfaces 12 1 by self>>#copyBits by self>>#warpBits InterpreterSimulator 30 sqFileAtEnd: 4 2 by FilePlugin>>#primitiveFileAtEnd actually sqFileAtEnd() InterpreterSimulator 30 primitiveFileOpen 15 2 by StandardFileStream>>#primOpen:writable: InterpreterSimulator 30 sqFileClose: 4 2 by FilePlugin>>#primitiveFileClose InterpreterSimulatorLSB 30 nextLongFrom: 12 2 via Interpreter>>#readImageFromFile:HeapSize:StartingAt:
On 1-Aug-06, at 5:34 AM, Klaus D. Witzel wrote:
On Tue, 01 Aug 2006 00:19:00 +0200, John M McIntosh wrote: ...
Really *all* VM entry point would need to be looked at in a proper audit to avoid buffer overflow issues, even perhaps accidents which generally are fatal.
If InterpreterSimulatorXyz where up-to-date and usable,
If *anybody* has time and interest available to improve the simulator, please feel free to work on it. My time for that particular corner is currently zero.
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: QVC: Question Valid Command
The main two questions are:
1) can you disable compiler ? (that means no doIt) 2) can you disable loading of compiled code and class by other means ?
If you cannot remove compiler, then you must restrict its use.
Le Lundi 31 Juillet 2006 22:18, Ron Teitelbaum a écrit :
Bert,
Thank you for your comments, I wasn't planning on using the sandbox since there are a number of external connections like Glorp-PostgreSQL needed where I plan to ultimately use this new functionality. I was hoping to address this with a signed image and VM. A signed image and VM could keep any changes out of the system.
The thing that concerns me is changes to a live system that happen without needing to save the image. Plenty can happen as the system is running so the question is: how can we secure the image while it is running?
Some things that would be needed off the top of my head are:
- Disable file in
- Disable the compiler for accepting or performing new code, but allow it
for already coded performs. (I'm sure this is much easier said then done).
A restricted compiler must not have access to Smalltalk environment but only a restricted one. It must also have access to a restricted number of messages, expecially not the system reflexive ones like allOwners allInstances basicAt: subclasses become: etc... It is safer in this case to authorize a few than forbid a lot of messages.
I did this kind of tricks in the days when ParcPlace runtime policy were really conservative. Maybe you will find more informations in commercial Smalltalk versions...
- Disable interactive debugger.
- Enable some sort of class level MAC checking which can verify that no
changes were made to code before it is executed.
It is much simpler to close the system by reducing it's interface, than trying to secure the kernel.
Anyway, what do mean before it is executed? Once image loaded does execution ever stop ? You mean at some strategic or random points of execution ?
- Prevent changes to compiled methods
You can try and lock MethodDictionary so that they do not accept to add remove or change any key/value, but for UndefinedObject doIt... (You just have to redefine few messages...). But you must make sure that there isn't a way to interpret some unbound CompiledMethod...
Without restricted compiler, I can figure some attacks on such a lock, like creating a simple dictionary, copying compiled methods from here and there and changing class to that of MethodDictionary and then replacing MethodDictionary's MethodDictionary via a become:., all this in a single doIt...
If you allow user relexive access to the Compiler, MethodDictionary or whatever you did protect, then your system is not closed enough.
Then you can play the game to protect the protections, but untill you find a way to have a protection loop, that is just adding complexity to the system, and you might well add more weak points if your design is too complex...
How to obtain a cyclic protection? That may be interesting...
Plus more that I'm sure will also be needed. If we can sign the image and VM and change the VM to check the signature before staring up and find a way to protect the live image then we will have gone a long way to having a more secure development platform.
And of course the plugins and DLL must be signed too.
I've mentioned this before and got absolutely no comment from anyone. (Maybe the email was too long to read?)
Some input would be very much appreciated.
Maybe i didn't had much value...
Nicolas
Ron Teitelbaum
From: Bert Freudenberg Sent: Monday, July 31, 2006 12:37 PM
Am 30.07.2006 um 06:15 schrieb Ron Teitelbaum:
Hello all,
I'm currently working on a TLS / SSL implementation and on windows I am planning to support the Certificate Store that is built in. This requires me to use the CryptoAPI dlls.
Well ... FFI is inherently unsafe - it allows to call any C function in any library. So depending on the area of application this might be a problem, since you're giving up the relative safety of a VM-based system. For example, if the VM's sandbox is enabled, FFI calls are disallowed, so you could not use crypto in a sandboxed environment.
- Bert -
From: nicolas cellier Sent: Monday, July 31, 2006 6:12 PM
The main two questions are:
- can you disable compiler ? (that means no doIt)
- can you disable loading of compiled code and class by other means ?
If you cannot remove compiler, then you must restrict its use.
[snip]
A restricted compiler must not have access to Smalltalk environment but only a restricted one. It must also have access to a restricted number of messages, expecially not the system reflexive ones like allOwners allInstances basicAt: subclasses become: etc... It is safer in this case to authorize a few than forbid a lot of messages.
I did this kind of tricks in the days when ParcPlace runtime policy were really conservative. Maybe you will find more informations in commercial Smalltalk versions...
I will look into it; it's a very good suggestion.
- Disable interactive debugger.
- Enable some sort of class level MAC checking which can verify that no
changes were made to code before it is executed.
It is much simpler to close the system by reducing it's interface, than trying to secure the kernel.
Anyway, what do mean before it is executed? Once image loaded does execution ever stop ? You mean at some strategic or random points of execution ?
What I mean is before each user method is executed, or each class is accessed if method execution is too resource intensive to verify the structure and the methods match some signed hash to verify that they have not changed. We could implement some time dependent verification or other caching methods to help improve performance.
- Prevent changes to compiled methods
You can try and lock MethodDictionary so that they do not accept to add remove or change any key/value, but for UndefinedObject doIt... (You just have to redefine few messages...). But you must make sure that there isn't a way to interpret some unbound CompiledMethod...
Without restricted compiler, I can figure some attacks on such a lock, like creating a simple dictionary, copying compiled methods from here and there and changing class to that of MethodDictionary and then replacing MethodDictionary's MethodDictionary via a become:., all this in a single doIt...
If you allow user relexive access to the Compiler, MethodDictionary or whatever you did protect, then your system is not closed enough.
Then you can play the game to protect the protections, but untill you find a way to have a protection loop, that is just adding complexity to the system, and you might well add more weak points if your design is too complex...
How to obtain a cyclic protection? That may be interesting...
Plus more that I'm sure will also be needed. If we can sign the image
and
VM and change the VM to check the signature before staring up and find a way to protect the live image then we will have gone a long way to
having a
more secure development platform.
And of course the plugins and DLL must be signed too.
Agreed.
Maybe i didn't had much value...
Nicolas
On the contrary your comments are very helpful and I hope you will continue to participate as more concrete suggestions are made.
Ron Teitelbaum
Le Lundi 31 Juillet 2006 18:36, Bert Freudenberg a écrit :
Am 30.07.2006 um 06:15 schrieb Ron Teitelbaum:
Hello all,
I’m currently working on a TLS / SSL implementation and on windows I am planning to support the Certificate Store that is built in. This requires me to use the CryptoAPI dlls.
Well ... FFI is inherently unsafe - it allows to call any C function in any library. So depending on the area of application this might be a problem, since you're giving up the relative safety of a VM-based system. For example, if the VM's sandbox is enabled, FFI calls are disallowed, so you could not use crypto in a sandboxed environment.
- Bert -
FFI is unsafe of course as C code is unsafe. But you have to explain why a plugin would be safer.
In the plugin case, user has to deal with pointers on data too, he has to do conversions by hands where FFI would automate part of this process and he has to be extremely carefull with object creation that might break previously collected pointers in Smalltalk space.
IMHO, plugins are harder to write than FFI calls.
So, the plugin are maybe safer because only a few well experienced and well behaved programmers dare jump the hurd, in a word because it is rarely used...
I am exagerating a bit of course. Am i?
Nicolas
nicolas cellier wrote:
FFI is unsafe of course as C code is unsafe.
No, that's only true for plugins. For the FFI it's a lot worse. Even if you have a perfectly safe function foo in bar.dll how do you prevent a malicious attacker instead to, e.g., execute "format c:" from "system.dll"? The FFI won't care.
But you have to explain why a plugin would be safer.
A plugin provides access that has been explicitly granted. You write a primitive in a plugin to provide access to that function and NO MORE. Without the FFI for example, it is not possible to divert any of the plugins to do the moral equivalent of "format c:". For plugins your statement is correct - the plugin is as safe as the code it uses and if there is a bug in the underlying C code that bug might be exploited. But beyond that, there is no authority that a plugin grants.
In the plugin case, user has to deal with pointers on data too, he has to do conversions by hands where FFI would automate part of this process and he has to be extremely carefull with object creation that might break previously collected pointers in Smalltalk space.
This is true in both situations. Unless you consider utterly trivial examples, managing complex data structures using the FFI is a nightmare - one of the reasons why I prefer to deal with them plugin-style.
So, the plugin are maybe safer because only a few well experienced and well behaved programmers dare jump the hurd, in a word because it is rarely used...
Given the amount of plugins and the amount of FFI usage I think plugin usage outnumbers FFI usage by an order of magnitude at least.
Cheers, - Andreas
Am 01.08.2006 um 07:43 schrieb Andreas Raab:
nicolas cellier wrote:
So, the plugin are maybe safer because only a few well experienced and well behaved programmers dare jump the hurd, in a word because it is rarely used...
Given the amount of plugins and the amount of FFI usage I think plugin usage outnumbers FFI usage by an order of magnitude at least.
Indeed. Just do a "Smalltalk listLoadedModules" to see which plugins you are using *right now*.
You can think of plugins as an extension of the VM, so in principle they can be made as safe as the VM itself is - as long as each primitive they provide has one specific function that can be checked for inputs and side-effects. This is not the case for the FFI primitives.
- Bert -
All,
I'll have to preface this comment with "I JUST GOT FFI WORKING" but consider the potential risk of FFI compared to any other external communication mechanisms.
"IF" the image is secured so that no changes can be made to a running image which I suppose would include calls to FFI <...> (he says not really knowing how the calls are made or interpreted) then there are benefits to direct calls.
First the implementation of the dll wrapper would not be in question only the direct calls to the OS provided calls would need to be reviewed.
Second securing the image by removing FFI is too drastic in my opinion. How will we get Crystal Reports or any other direct API to work without it?
Third support for external API's are simplified for developers by having the calls in the image not wrapped in a DLL. Everything is much more transparent.
Forth there is not much difference in terms of risk to the system between the execution of the DLL function and the FFI call if parameters are passed in. Both methods need to have proper handling for error conditions before the primitive call. I'm not convinced that error checking is better in a DLL then in Smalltalk and on the contrary if we make the error checking harder to apply (by pushing it into the DLL) it is less likely that changes will be made to prevent the error condition.
Ron Teitelbaum
From: Bert Freudenberg Sent: Tuesday, August 01, 2006 11:20 AM
Am 01.08.2006 um 07:43 schrieb Andreas Raab:
nicolas cellier wrote:
So, the plugin are maybe safer because only a few well experienced and well behaved programmers dare jump the hurd, in a word because it is rarely used...
Given the amount of plugins and the amount of FFI usage I think plugin usage outnumbers FFI usage by an order of magnitude at least.
Indeed. Just do a "Smalltalk listLoadedModules" to see which plugins you are using *right now*.
You can think of plugins as an extension of the VM, so in principle they can be made as safe as the VM itself is - as long as each primitive they provide has one specific function that can be checked for inputs and side-effects. This is not the case for the FFI primitives.
- Bert -
Totally agree
The same level of carefull precondition testing can be achieved in Smalltalk and FFI than through plugin.
The major difference is that with FFI, we cannot prevent user to bypass those protections and precondition tests.
But if the user also have access to plugin source code, you cannot prevent either... The only difference is that it is more difficult to program the plugin than FFI, and that is why it is safer in a way... Also, i am not sure if we can modify the plugin without restarting the image... Can we? If not, that's a good point for plugins and security.
If we are lazy, both FFI and plugin implementations will have security holes. But maybe, code will be corrected faster in a FFI framework.
I have no experience with Squeak plugin, but practiced both user primitives and DLLCC in VW. And the latter gave me more flexibility and rapid development than the former (except for cross machine portability for which good old dirty C preprocessor directives are somehow more efficient).
I would encourage FFI because it does open Squeak on the external world. Of course it does scratch the "do it all in Smalltalk" paradigm, but from a strategic point of view, opening Squeak is really something needed if you want to convince more people to join (think of how easy it is in python to call external code).
And sorry, the plugin framework is not simple enough. You have to learn a new language that is neither Smalltalk, nor C, you have to go through code generation for generating the wrapper function to call your external one, then you are on your own to compile the wrapper code and integrate it in the image... Rather a long development cycle for Smalltalkers. That is only my opinion, and of course, good arguments can make it change.
Nicolas
Le Mardi 01 Août 2006 18:19, Ron Teitelbaum a écrit :
All,
I'll have to preface this comment with "I JUST GOT FFI WORKING" but consider the potential risk of FFI compared to any other external communication mechanisms.
"IF" the image is secured so that no changes can be made to a running image which I suppose would include calls to FFI <...> (he says not really knowing how the calls are made or interpreted) then there are benefits to direct calls.
First the implementation of the dll wrapper would not be in question only the direct calls to the OS provided calls would need to be reviewed.
Second securing the image by removing FFI is too drastic in my opinion. How will we get Crystal Reports or any other direct API to work without it?
Third support for external API's are simplified for developers by having the calls in the image not wrapped in a DLL. Everything is much more transparent.
Forth there is not much difference in terms of risk to the system between the execution of the DLL function and the FFI call if parameters are passed in. Both methods need to have proper handling for error conditions before the primitive call. I'm not convinced that error checking is better in a DLL then in Smalltalk and on the contrary if we make the error checking harder to apply (by pushing it into the DLL) it is less likely that changes will be made to prevent the error condition.
Ron Teitelbaum
From: Bert Freudenberg Sent: Tuesday, August 01, 2006 11:20 AM
Am 01.08.2006 um 07:43 schrieb Andreas Raab:
nicolas cellier wrote:
So, the plugin are maybe safer because only a few well experienced and well behaved programmers dare jump the hurd, in a word because it is rarely used...
Given the amount of plugins and the amount of FFI usage I think plugin usage outnumbers FFI usage by an order of magnitude at least.
Indeed. Just do a "Smalltalk listLoadedModules" to see which plugins you are using *right now*.
You can think of plugins as an extension of the VM, so in principle they can be made as safe as the VM itself is - as long as each primitive they provide has one specific function that can be checked for inputs and side-effects. This is not the case for the FFI primitives.
- Bert -
Another point to consider is that a plugin run at the same speed as C code and can't be optimized dynamically by a Jitter implementation. imho enhancements on the VM will apply to clients of FFI, but will not be applicable to plugins. Ale.
----- Original Message ----- From: "nicolas cellier" ncellier@ifrance.com To: "The general-purpose Squeak developers list" squeak-dev@lists.squeakfoundation.org Sent: Tuesday, August 01, 2006 3:14 PM Subject: Re: FFI vs DLL (was Suggestions for C++ or DLL code)
Totally agree
The same level of carefull precondition testing can be achieved in Smalltalk and FFI than through plugin.
The major difference is that with FFI, we cannot prevent user to bypass those protections and precondition tests.
But if the user also have access to plugin source code, you cannot prevent either... The only difference is that it is more difficult to program the plugin than FFI, and that is why it is safer in a way... Also, i am not sure if we can modify the plugin without restarting the image... Can we? If not, that's a good point for plugins and security.
If we are lazy, both FFI and plugin implementations will have security holes. But maybe, code will be corrected faster in a FFI framework.
I have no experience with Squeak plugin, but practiced both user primitives and DLLCC in VW. And the latter gave me more flexibility and rapid development than the former (except for cross machine portability for which good old dirty C preprocessor directives are somehow more efficient).
I would encourage FFI because it does open Squeak on the external world. Of course it does scratch the "do it all in Smalltalk" paradigm, but from a strategic point of view, opening Squeak is really something needed if you want to convince more people to join (think of how easy it is in python to call external code).
And sorry, the plugin framework is not simple enough. You have to learn a new language that is neither Smalltalk, nor C, you have to go through code generation for generating the wrapper function to call your external one, then you are on your own to compile the wrapper code and integrate it in the image... Rather a long development cycle for Smalltalkers. That is only my opinion, and of course, good arguments can make it change.
Nicolas
Le Mardi 01 Août 2006 18:19, Ron Teitelbaum a écrit :
All,
I'll have to preface this comment with "I JUST GOT FFI WORKING" but consider the potential risk of FFI compared to any other external communication mechanisms.
"IF" the image is secured so that no changes can be made to a running
image
which I suppose would include calls to FFI <...> (he says not really knowing how the calls are made or interpreted) then there are benefits to direct calls.
First the implementation of the dll wrapper would not be in question only the direct calls to the OS provided calls would need to be reviewed.
Second securing the image by removing FFI is too drastic in my opinion. How will we get Crystal Reports or any other direct API to work without
it?
Third support for external API's are simplified for developers by having the calls in the image not wrapped in a DLL. Everything is much more transparent.
Forth there is not much difference in terms of risk to the system between the execution of the DLL function and the FFI call if parameters are
passed
in. Both methods need to have proper handling for error conditions before the primitive call. I'm not convinced that error checking is better in a DLL then in Smalltalk and on the contrary if we make the error checking harder to apply (by pushing it into the DLL) it is less likely that
changes
will be made to prevent the error condition.
Ron Teitelbaum
From: Bert Freudenberg Sent: Tuesday, August 01, 2006 11:20 AM
Am 01.08.2006 um 07:43 schrieb Andreas Raab:
nicolas cellier wrote:
So, the plugin are maybe safer because only a few well experienced and well behaved programmers dare jump the hurd, in a word because it is rarely used...
Given the amount of plugins and the amount of FFI usage I think plugin usage outnumbers FFI usage by an order of magnitude at least.
Indeed. Just do a "Smalltalk listLoadedModules" to see which plugins you are using *right now*.
You can think of plugins as an extension of the VM, so in principle they can be made as safe as the VM itself is - as long as each primitive they provide has one specific function that can be checked for inputs and side-effects. This is not the case for the FFI primitives.
- Bert -
squeak-dev@lists.squeakfoundation.org
-
Alejandro F. Reimondo -
Andreas Raab -
Bert Freudenberg -
Craig Latta -
Hans-Martin Mosner -
John M McIntosh -
John M McIntosh -
Klaus D. Witzel -
nicolas cellier -
Ron Teitelbaum -
tim Rowledge