Hello, i am fed up seeing the messages like "FFI can't find a library" or "VM can't find my plugin".
I think we should trash too clever intrinsic library lookup mechanism and provide a simple primitive, which attempts to load a library by given file name & path and if it fails, give an extended response, why: - library found, but can't be loaded (because it 64bit while you running 32 bit), or it can't be found and so on.
It should be up to the user to find and load the library first, and only after you obtained a valid library handle, attempt to do FFI calls.
IMO, same should be done to plugin loading mechanism. VM should give you the paths: - what is the current directory - what is the directory where VM located
then user code could attempt to load external plugin by specifying one of these paths or any other to the library which representing plugin. And only after he knows that library successfully loaded, he could attempt to use its primitives (or do FFI calls).
The approach is simple: - first you making sure that you loaded external module/plugin, then you start using it. And it is your responsibility to make sure you loaded it, not VM's. This could be done at image startup or at code entry point, or even inside a generic code which handling a primitive failure.
VM should behave like a stupid idiot: - if module with name, which specified in primitive pragma <primitive: 'foo' module: 'bar' > not registered already in internal list of VM's modules, it should fail unconditionally without even trying to load external library. Even for internal plugins: A user code can handle this error and instruct VM explicitly to load & initialize internal plugin, but not implicitly like currently VM does for internal and external plugins.
If we put a module loading mechanism at image side, it will give us much more flexibility, and all future/potential problems with it could be fixed there without touching VM code anymore.
I know that it means quite a lot of work (some named primitives even assume that they never fail). But i think it is worth it. Because if it will stay like that, we will always have issues with it, over and over again.
I am not sure if it applies to your proposal here, but just in case I want to mention one note of caution. The #primLoadModule that was discussed previously in the "Sandbox" thread will not work on a 64-bit platform, for the same reason that FFI does not work on a 64-bit platform.
Dave
On Thu, May 26, 2011 at 01:48:42PM +0200, Igor Stasenko wrote:
Hello, i am fed up seeing the messages like "FFI can't find a library" or "VM can't find my plugin".
I think we should trash too clever intrinsic library lookup mechanism and provide a simple primitive, which attempts to load a library by given file name & path and if it fails, give an extended response, why:
- library found, but can't be loaded (because it 64bit while you
running 32 bit), or it can't be found and so on.
It should be up to the user to find and load the library first, and only after you obtained a valid library handle, attempt to do FFI calls.
IMO, same should be done to plugin loading mechanism. VM should give you the paths:
- what is the current directory
- what is the directory where VM located
then user code could attempt to load external plugin by specifying one of these paths or any other to the library which representing plugin. And only after he knows that library successfully loaded, he could attempt to use its primitives (or do FFI calls).
The approach is simple:
- first you making sure that you loaded external module/plugin, then
you start using it. And it is your responsibility to make sure you loaded it, not VM's. This could be done at image startup or at code entry point, or even inside a generic code which handling a primitive failure.
VM should behave like a stupid idiot:
- if module with name, which specified in primitive pragma
<primitive: 'foo' module: 'bar' > not registered already in internal list of VM's modules, it should fail unconditionally without even trying to load external library. Even for internal plugins: A user code can handle this error and instruct VM explicitly to load & initialize internal plugin, but not implicitly like currently VM does for internal and external plugins.
If we put a module loading mechanism at image side, it will give us much more flexibility, and all future/potential problems with it could be fixed there without touching VM code anymore.
I know that it means quite a lot of work (some named primitives even assume that they never fail). But i think it is worth it. Because if it will stay like that, we will always have issues with it, over and over again.
-- Best regards, Igor Stasenko AKA sig.
On 26 May 2011 14:16, David T. Lewis lewis@mail.msen.com wrote:
I am not sure if it applies to your proposal here, but just in case I want to mention one note of caution. The #primLoadModule that was discussed previously in the "Sandbox" thread will not work on a 64-bit platform, for the same reason that FFI does not work on a 64-bit platform.
Sorry, i din't followed that issue (cant remember). The reason because of mixing 32/64bit libraries, or something else?
The primitive you mentioning reusing the same routine to load external module as VM internally for primitives do. So, it is step in right direction (explicitly instruct VM to load modules), but more work needs to be done.
Now imagine that you have a primitives:
loadInternalModule: moduleName loadExternalModule: moduleName path: pathToModuleLibrary
and by analogy, for FFI:
loadExternalLibrary: pathToExternalLibrary
and in case of failure, these prims could then give you an extended response: - module already loaded - internal module not exists - external library file not found - external library file found but cannot be loaded (not a library, mix 64/32bit mode etc) - module loaded but failed to initialize .. etc
Then you can easily diagnose the problems per each attempt to load some library. But not like today.. if you telling to load module named 'GL' , VM tries all different combinations like libGL.so GL.so libGL.dynlib multiplied on various crazy locations in file system..
Is it only me who thinks that, this looks like shooting blindly into the sky hoping there are enough ducks flying so you can hit one?
Also, if you explicitly control the module loading then you can also control which module takes priority - internal or external one.
if you know, today's VMs first attempting to load external library with given name, and only then internal plugins.
Apparently, if you wanna play games in sandbox, you'd be willing to change a priority or even do not attempt to use any external modules at all.
2011/5/26 Igor Stasenko siguctua@gmail.com
On 26 May 2011 14:16, David T. Lewis lewis@mail.msen.com wrote:
I am not sure if it applies to your proposal here, but just in case I want to mention one note of caution. The #primLoadModule that was discussed previously in the "Sandbox" thread will not work on a 64-bit platform, for the same reason that FFI does not work on a 64-bit platform.
Sorry, i din't followed that issue (cant remember). The reason because of mixing 32/64bit libraries, or something else?
The primitive you mentioning reusing the same routine to load external module as VM internally for primitives do. So, it is step in right direction (explicitly instruct VM to load modules), but more work needs to be done.
Now imagine that you have a primitives:
loadInternalModule: moduleName loadExternalModule: moduleName path: pathToModuleLibrary
and by analogy, for FFI:
loadExternalLibrary: pathToExternalLibrary
and in case of failure, these prims could then give you an extended response:
- module already loaded
- internal module not exists
- external library file not found
- external library file found but cannot be loaded (not a library,
mix 64/32bit mode etc)
- module loaded but failed to initialize
.. etc
Then you can easily diagnose the problems per each attempt to load some library. But not like today.. if you telling to load module named 'GL' , VM tries all different combinations like libGL.so GL.so libGL.dynlib multiplied on various crazy locations in file system..
Is it only me who thinks that, this looks like shooting blindly into the sky hoping there are enough ducks flying so you can hit one?
Also, if you explicitly control the module loading then you can also control which module takes priority
- internal or external one.
if you know, today's VMs first attempting to load external library with given name, and only then internal plugins.
Apparently, if you wanna play games in sandbox, you'd be willing to change a priority or even do not attempt to use any external modules at all.
+1 paths used to look for external libraries (FFI or plugins) should be simpler and documented. I am always looking into log to know why it failed. IMO, it should also be easy to share external libs between VMs. I already share some libraries but it requires to discover some strange paths.
#Luc
-- Best regards, Igor Stasenko AKA sig.
On Thu, May 26, 2011 at 02:39:52PM +0200, Igor Stasenko wrote:
On 26 May 2011 14:16, David T. Lewis lewis@mail.msen.com wrote:
I am not sure if it applies to your proposal here, but just in case I want to mention one note of caution. The #primLoadModule that was discussed previously in the "Sandbox" thread will not work on a 64-bit platform, for the same reason that FFI does not work on a 64-bit platform.
Sorry, i din't followed that issue (cant remember). The reason because of mixing 32/64bit libraries, or something else?
Something else. It's basic type declaration issues, storing 64-bit pointers in 32-bit ints etc.
http://bugs.squeak.org/view.php?id=7237
http://lists.squeakfoundation.org/pipermail/vm-dev/2008-May/001945.html
Dave
Hello
2011/5/26 Igor Stasenko siguctua@gmail.com
loadInternalModule: moduleName loadExternalModule: moduleName path: pathToModuleLibrary
and by analogy, for FFI:
loadExternalLibrary: pathToExternalLibrary
Can you explain what the difference between this kind of module loading. Why not enough just loadLibrary: pathToLibrary?
On 26.05.2011, at 20:41, Denis Kudriashov wrote:
Hello
2011/5/26 Igor Stasenko siguctua@gmail.com
loadInternalModule: moduleName loadExternalModule: moduleName path: pathToModuleLibrary
and by analogy, for FFI:
loadExternalLibrary: pathToExternalLibrary
Can you explain what the difference between this kind of module loading. Why not enough just loadLibrary: pathToLibrary?
VM modules are not just arbitrary libraries. They provide primitive Smalltalk methods. They have special entry points like setInterpreter to be able to interface with the VM directly. Their location is known to the VM. Internal modules are already loaded. But they are still only "activated" when one of their primitive is called. They do the name-to-function-address mapping on their own rather than using the OS's name resolution mechanism.
Libraries loaded using FFI are just C libraries, nothing special about them. The provide C functions. Name lookup is done using OS functions. The VM can only guess about their location. Igor was proposing to take the guessing out of the VM and put it into the image.
- Bert -
On 26 May 2011 23:35, Bert Freudenberg bert@freudenbergs.de wrote:
On 26.05.2011, at 20:41, Denis Kudriashov wrote:
Hello
2011/5/26 Igor Stasenko siguctua@gmail.com
loadInternalModule: moduleName loadExternalModule: moduleName path: pathToModuleLibrary
and by analogy, for FFI:
loadExternalLibrary: pathToExternalLibrary
Can you explain what the difference between this kind of module loading. Why not enough just loadLibrary: pathToLibrary?
VM modules are not just arbitrary libraries. They provide primitive Smalltalk methods. They have special entry points like setInterpreter to be able to interface with the VM directly. Their location is known to the VM. Internal modules are already loaded. But they are still only "activated" when one of their primitive is called. They do the name-to-function-address mapping on their own rather than using the OS's name resolution mechanism. Libraries loaded using FFI are just C libraries, nothing special about them. The provide C functions. Name lookup is done using OS functions. The VM can only guess about their location. Igor was proposing to take the guessing out of the VM and put it into the image.
What i finding fun that once you load B3D plugin, it already loads GL library, because it is linked with it. But it doesn't means that FFI can locate/use it :)
- Bert -
On 26.05.2011, at 23:45, Igor Stasenko wrote:
On 26 May 2011 23:35, Bert Freudenberg bert@freudenbergs.de wrote:
On 26.05.2011, at 20:41, Denis Kudriashov wrote:
Hello
2011/5/26 Igor Stasenko siguctua@gmail.com
loadInternalModule: moduleName loadExternalModule: moduleName path: pathToModuleLibrary
and by analogy, for FFI:
loadExternalLibrary: pathToExternalLibrary
Can you explain what the difference between this kind of module loading. Why not enough just loadLibrary: pathToLibrary?
VM modules are not just arbitrary libraries. They provide primitive Smalltalk methods. They have special entry points like setInterpreter to be able to interface with the VM directly. Their location is known to the VM. Internal modules are already loaded. But they are still only "activated" when one of their primitive is called. They do the name-to-function-address mapping on their own rather than using the OS's name resolution mechanism. Libraries loaded using FFI are just C libraries, nothing special about them. The provide C functions. Name lookup is done using OS functions. The VM can only guess about their location. Igor was proposing to take the guessing out of the VM and put it into the image.
What i finding fun that once you load B3D plugin, it already loads GL library, because it is linked with it. But it doesn't means that FFI can locate/use it :)
That is one advantage of using proper plugins, yes. Because linking the plugin uses the system linker. It will e.g. link it correctly to libGL.so.1 if that's what your system is using. Even though the name given in the Makefile is just "GL".
One problem people are experiencing with FFI under Linux in particular is that there is no "libGL.so" on most machines. That symlink only gets installed by the developer packages. "Regular" software links against specific library versions (e.g. libGL.so.1).
I wonder how other systems solve that problem. E.g. GNOME 3 introduces "GObject Introspection" which basically provides type annotations to generate FFI calls. But it still would need to figure out the proper library version. Maybe that's listed in the type library file? I don't know.
- Bert -
Thanks for answer.
2011/5/27 Bert Freudenberg bert@freudenbergs.de
On 26.05.2011, at 20:41, Denis Kudriashov wrote:
Hello
2011/5/26 Igor Stasenko siguctua@gmail.com
loadInternalModule: moduleName loadExternalModule: moduleName path: pathToModuleLibrary
and by analogy, for FFI:
loadExternalLibrary: pathToExternalLibrary
Can you explain what the difference between this kind of module loading. Why not enough just loadLibrary: pathToLibrary?
VM modules are not just arbitrary libraries. They provide primitive Smalltalk methods. They have special entry points like setInterpreter to be able to interface with the VM directly. Their location is known to the VM. Internal modules are already loaded. But they are still only "activated" when one of their primitive is called. They do the name-to-function-address mapping on their own rather than using the OS's name resolution mechanism.
Libraries loaded using FFI are just C libraries, nothing special about them. The provide C functions. Name lookup is done using OS functions. The VM can only guess about their location. Igor was proposing to take the guessing out of the VM and put it into the image.
- Bert -
On 26 May 2011 20:41, Denis Kudriashov dionisiydk@gmail.com wrote:
Hello
2011/5/26 Igor Stasenko siguctua@gmail.com
loadInternalModule: moduleName loadExternalModule: moduleName path: pathToModuleLibrary
and by analogy, for FFI:
loadExternalLibrary: pathToExternalLibrary
Can you explain what the difference between this kind of module loading. Why not enough just loadLibrary: pathToLibrary?
If you loading VM plugin, is expects that dynamic library has setInterpreter() function to initialize it. Otherwise it reports failure. Then VM calls this function and if function returns "ok" , then module is registered in module list and you can start using primitives provided by it. With FFI its different: an external library can hold (or not) anything. There is no any conventions.
On 26.05.2011, at 13:48, Igor Stasenko wrote:
Hello, i am fed up seeing the messages like "FFI can't find a library" or "VM can't find my plugin".
I agree for FFI libraries. I do not agree for VM modules.
IMHO it is a terrible idea to lump the two together. They happen to share an implementation, but that is rather incidental.
Your new primitives should be added to the FFI. Heck, since it's FFI you should be able to just call whatever OS function you want to find out the right path. For FFI this okay because the code in the image needs to deal with highly platform-dependent stuff anyway.
VM modules (a.k.a. plugins) are a totally different kind of beast. They are just an implementation detail of the VM. Nothing in the image should need to care about where those plugins are. There are virtually no problems with built-in modules. There are very few and minor problems with external modules. And they are almost exclusive to Linux, where they have gotten much easier to fix since Ian made "squeakvm" be a shell script that sets the right paths and can easily be edited. I see no good reason to abandon this infrastructure.
- Bert -
On 26 May 2011 20:12, Bert Freudenberg bert@freudenbergs.de wrote:
On 26.05.2011, at 13:48, Igor Stasenko wrote:
Hello, i am fed up seeing the messages like "FFI can't find a library" or "VM can't find my plugin".
I agree for FFI libraries. I do not agree for VM modules.
IMHO it is a terrible idea to lump the two together. They happen to share an implementation, but that is rather incidental.
Not at all. Both are dynamic libraries, and VM don't need to load same library twice. A way to do it is keep an eye of everything what loaded. But anyways, i prefer a user code to reason what to load, not VM.
Your new primitives should be added to the FFI. Heck, since it's FFI you should be able to just call whatever OS function you want to find out the right path. For FFI this okay because the code in the image needs to deal with highly platform-dependent stuff anyway.
VM modules (a.k.a. plugins) are a totally different kind of beast. They are just an implementation detail of the VM. Nothing in the image should need to care about where those plugins are. There are virtually no problems with built-in modules. There are very few and minor problems with external modules. And they are almost exclusive to Linux, where they have gotten much easier to fix since Ian made "squeakvm" be a shell script that sets the right paths and can easily be edited. I see no good reason to abandon this infrastructure.
Maybe. Depends, what you proposing to answer to future "i can't load plugin" and "i don't know why it not loading" problems.
- Bert -
On 5/26/2011 20:12, Bert Freudenberg wrote:
On 26.05.2011, at 13:48, Igor Stasenko wrote:
Hello, i am fed up seeing the messages like "FFI can't find a library" or "VM can't find my plugin".
I agree for FFI libraries. I do not agree for VM modules.
IMHO it is a terrible idea to lump the two together. They happen to share an implementation, but that is rather incidental.
+1. It's a mistake to lump those two together for sure. +1 also for the idea of having more (and better) failure information on why loading of any of them failed (this holds both for the FFI as well as the plugins because their load failure is often even more mysterious).
However, a big -1 on the idea of a "dumb" FFI loading function if by "dumb" we mean it to ignore the standard system paths on the platform (LD_LIBRARY_PATH and friends). The whole idea of these is to provide executable with the proper information to find those modules; why on earth would we ignore the information that is specifically provided for that purpose?
Cheers, - Andreas
On 27 May 2011 10:53, Andreas Raab andreas.raab@gmx.de wrote:
On 5/26/2011 20:12, Bert Freudenberg wrote:
On 26.05.2011, at 13:48, Igor Stasenko wrote:
Hello, i am fed up seeing the messages like "FFI can't find a library" or "VM can't find my plugin".
I agree for FFI libraries. I do not agree for VM modules.
IMHO it is a terrible idea to lump the two together. They happen to share an implementation, but that is rather incidental.
+1. It's a mistake to lump those two together for sure. +1 also for the idea of having more (and better) failure information on why loading of any of them failed (this holds both for the FFI as well as the plugins because their load failure is often even more mysterious).
However, a big -1 on the idea of a "dumb" FFI loading function if by "dumb" we mean it to ignore the standard system paths on the platform (LD_LIBRARY_PATH and friends). The whole idea of these is to provide executable with the proper information to find those modules; why on earth would we ignore the information that is specifically provided for that purpose?
who said that you will ignore it? I said that VM should expose all useful paths to language side and it should be up to language side, where to look for library. But not magically like it doing now.
Cheers, - Andreas
On 5/27/2011 12:40, Igor Stasenko wrote:
On 27 May 2011 10:53, Andreas Raabandreas.raab@gmx.de wrote:
However, a big -1 on the idea of a "dumb" FFI loading function if by "dumb" we mean it to ignore the standard system paths on the platform (LD_LIBRARY_PATH and friends). The whole idea of these is to provide executable with the proper information to find those modules; why on earth would we ignore the information that is specifically provided for that purpose?
who said that you will ignore it? I said that VM should expose all useful paths to language side and it should be up to language side, where to look for library. But not magically like it doing now.
Then perhaps you can clarify what you mean by "I think we should trash too clever intrinsic library lookup mechanism and provide a simple primitive, which attempts to load a library by given file name & path ". To me, this sounded as if you want to deliberately abandon implicit search paths on the platforms. If not, then I don't understand what is different compared with today (other than having better failure information with which I agree).
Cheers, - Andreas
On 27 May 2011 12:52, Andreas Raab andreas.raab@gmx.de wrote:
On 5/27/2011 12:40, Igor Stasenko wrote:
On 27 May 2011 10:53, Andreas Raabandreas.raab@gmx.de wrote:
However, a big -1 on the idea of a "dumb" FFI loading function if by "dumb" we mean it to ignore the standard system paths on the platform (LD_LIBRARY_PATH and friends). The whole idea of these is to provide executable with the proper information to find those modules; why on earth would we ignore the information that is specifically provided for that purpose?
who said that you will ignore it? I said that VM should expose all useful paths to language side and it should be up to language side, where to look for library. But not magically like it doing now.
Then perhaps you can clarify what you mean by "I think we should trash too clever intrinsic library lookup mechanism and provide a simple primitive, which attempts to load a library by given file name & path ". To me, this sounded as if you want to deliberately abandon implicit search paths on the platforms. If not, then I don't understand what is different compared with today (other than having better failure information with which I agree).
Clarification: if search is implicitly managed by platform, then there's no problem. (and actually you cannot trash this logic since it sits in platform functions, and VM have to use these functions anyway). But VM should not add its own implicit logic in addition to what provided by platform.
Cheers, - Andreas
On 5/27/2011 13:01, Igor Stasenko wrote:
On 27 May 2011 12:52, Andreas Raabandreas.raab@gmx.de wrote:
Then perhaps you can clarify what you mean by "I think we should trash too clever intrinsic library lookup mechanism and provide a simple primitive, which attempts to load a library by given file name& path ". To me, this sounded as if you want to deliberately abandon implicit search paths on the platforms. If not, then I don't understand what is different compared with today (other than having better failure information with which I agree).
Clarification: if search is implicitly managed by platform, then there's no problem. (and actually you cannot trash this logic since it sits in platform functions, and VM have to use these functions anyway). But VM should not add its own implicit logic in addition to what provided by platform.
It seems to me that a good way to address the implicit logic is simply by by exposing (a list of) search paths to the image. In which case you can set the search paths to be as wide, or as narrow, as you'd like and the VM can list what additional search paths it would use by default, thus making this both forwards and backwards compatible.
Cheers, - Andreas
On 27 May 2011 13:25, Andreas Raab andreas.raab@gmx.de wrote:
On 5/27/2011 13:01, Igor Stasenko wrote:
On 27 May 2011 12:52, Andreas Raabandreas.raab@gmx.de wrote:
Then perhaps you can clarify what you mean by "I think we should trash too clever intrinsic library lookup mechanism and provide a simple primitive, which attempts to load a library by given file name& path ". To me, this sounded as if you want to deliberately abandon implicit search paths on the platforms. If not, then I don't understand what is different compared with today (other than having better failure information with which I agree).
Clarification: if search is implicitly managed by platform, then there's no problem. (and actually you cannot trash this logic since it sits in platform functions, and VM have to use these functions anyway). But VM should not add its own implicit logic in addition to what provided by platform.
It seems to me that a good way to address the implicit logic is simply by by exposing (a list of) search paths to the image. In which case you can set the search paths to be as wide, or as narrow, as you'd like and the VM can list what additional search paths it would use by default, thus making this both forwards and backwards compatible.
Indeed. In this way it is language side, are the only who responsible for finding a library, while VM could provide a hints , like default search paths, which are most appropriate for current platform.
So, if we agreed on that, lets sketch the plan what we need to provide:
- extended error handling - explicit module loading mechanism - providing hints for library search by VM
On 5/27/2011 13:51, Igor Stasenko wrote:
On 27 May 2011 13:25, Andreas Raabandreas.raab@gmx.de wrote:
On 5/27/2011 13:01, Igor Stasenko wrote:
On 27 May 2011 12:52, Andreas Raabandreas.raab@gmx.de wrote:
Then perhaps you can clarify what you mean by "I think we should trash too clever intrinsic library lookup mechanism and provide a simple primitive, which attempts to load a library by given file name& path ". To me, this sounded as if you want to deliberately abandon implicit search paths on the platforms. If not, then I don't understand what is different compared with today (other than having better failure information with which I agree).
Clarification: if search is implicitly managed by platform, then there's no problem. (and actually you cannot trash this logic since it sits in platform functions, and VM have to use these functions anyway). But VM should not add its own implicit logic in addition to what provided by platform.
It seems to me that a good way to address the implicit logic is simply by by exposing (a list of) search paths to the image. In which case you can set the search paths to be as wide, or as narrow, as you'd like and the VM can list what additional search paths it would use by default, thus making this both forwards and backwards compatible.
Indeed. In this way it is language side, are the only who responsible for finding a library, while VM could provide a hints , like default search paths, which are most appropriate for current platform.
So, if we agreed on that, lets sketch the plan what we need to provide:
- extended error handling
- explicit module loading mechanism
- providing hints for library search by VM
Almost :-) I really can't see a reason for requiring explicit module loading. Amongst other things it bloats applications (which will attempt to preload all libraries they might *possibly* need instead of those they *actually* need) and slow down system startup (for the same reason) or otherwise require people to write lots of either repeated code (as an example, just look at how many senders of Socket initializeNetwork there are all of which should be removed) or equally pointless extra primitive/ffi failure code. I can really see no good reason for requiring explicit loading other than ensure some library is available and that can be done with better error reporting.
Cheers, - Andreas
On 27 May 2011 14:07, Andreas Raab andreas.raab@gmx.de wrote:
On 5/27/2011 13:51, Igor Stasenko wrote:
On 27 May 2011 13:25, Andreas Raabandreas.raab@gmx.de wrote:
On 5/27/2011 13:01, Igor Stasenko wrote:
On 27 May 2011 12:52, Andreas Raabandreas.raab@gmx.de wrote:
Then perhaps you can clarify what you mean by "I think we should trash too clever intrinsic library lookup mechanism and provide a simple primitive, which attempts to load a library by given file name& path ". To me, this sounded as if you want to deliberately abandon implicit search paths on the platforms. If not, then I don't understand what is different compared with today (other than having better failure information with which I agree).
Clarification: if search is implicitly managed by platform, then there's no problem. (and actually you cannot trash this logic since it sits in platform functions, and VM have to use these functions anyway). But VM should not add its own implicit logic in addition to what provided by platform.
It seems to me that a good way to address the implicit logic is simply by by exposing (a list of) search paths to the image. In which case you can set the search paths to be as wide, or as narrow, as you'd like and the VM can list what additional search paths it would use by default, thus making this both forwards and backwards compatible.
Indeed. In this way it is language side, are the only who responsible for finding a library, while VM could provide a hints , like default search paths, which are most appropriate for current platform.
So, if we agreed on that, lets sketch the plan what we need to provide:
- extended error handling
- explicit module loading mechanism
- providing hints for library search by VM
Almost :-) I really can't see a reason for requiring explicit module loading. Amongst other things it bloats applications (which will attempt to preload all libraries they might *possibly* need instead of those they *actually* need) and slow down system startup (for the same reason) or otherwise require people to write lots of either repeated code (as an example, just look at how many senders of Socket initializeNetwork there are all of which should be removed) or equally pointless extra primitive/ffi failure code. I can really see no good reason for requiring explicit loading other than ensure some library is available and that can be done with better error reporting.
No need for repetitious code :) It will require changes in #primitiveFailed, or #primitiveFailedFor: errorCode on image side. And of course it makes sense to introduce this only for VMs which can use primitive error codes.
A primitiveFailed method could be changed to see if primitive failed because module not yet loaded and then it will try loading it and retry the primitive. So, for most users of primitives it will behave as today: the failure will be handled implicitly, and seamlessly, except that implicit code are in image not in VM!
As for explicit module loading at image startup: it has own uses. If you like to run in sandboxed mode, you must ensure that no other modules could be loaded except those, which you know that they are secure and should be there. So, during startup you will load modules which you will use, and then disable loading module mechanism for the rest of the session. In that way a primitives which attempting to use non-registered modules will fail. And then a error handler in #primitiveFail, if it attempt to load such module, it will receive an error: new module can't be loaded because module loading mechanism is disabled.
And that's why i proposed to use explicit mechanism even for internal modules, because some internal modules could contain caveats which malicious code could exploit to destroy your sand castle. And designers of sandboxed images could choose to prevent loading such modules, because it could compromise the security.
So, it is also a future proof, because today we ship VMs with one set of default internal modules, but tomorrow we may include some more. But developers of sandbox mode images don't need to care of these new additions and can feel confident, that their stuff will continue working on new VM as before, without compromising their security model. (unless, of course there will be changes in existing plugins which may open new security holes ;)
Cheers, - Andreas
vm-dev@lists.squeakfoundation.org
-
Andreas Raab -
Bert Freudenberg -
David T. Lewis -
Denis Kudriashov -
Igor Stasenko -
Luc Fabresse