Hello all,
Is there a way to get the pre-compiled linux vm to describe what the full path is to a library it is trying to load? I can hack the source to do so, but then I start wondering what I might have disturbed in the process. Out of the box, all I get is an external library with a nil handle and no clue what did or did not happen.
At various points in my struggle today, I saw what looked like a tendency to load plugins from the vmpath//so.pluginName (two slashes rather than one). It raises the question of whether the code is adding an extra slash when it is not given an explicit plugins path?? The separate concern is that I need information about what is happening when one of my libraries fails to load.
One possibility is that a particular .so can't load. How would I find out whether that this is case? Is there a way to test load it or to get the vm to tell me the cause of a failure to load it?
Bill
It has been a long day, and it might not have needed to be so. Within 15 minutes of seeing "undefined symbol: libusb_close" from dlerror(), I had the problem not only diagnosed but fixed; seeing the error in something related to USB support was enough to suggest I needed to link that library into the .so I was trying to load. I was still guessing, but it was an educated guess based on a concrete lead.
Is there a way to get such diagnostic information without hacking the vm source? If not, please consider the following as a proposed patch. I have tried to strike a balance between seeing enough to make sense of errant plugin paths and giving detailed information about failures when given an absolute path. Behavior changes (I intend/think) only if something won't load (it says what it was trying to do) and if the library is specified by absolute path, in which case it gives diagnostic output on any failure to load.
Bill
( Squeak-...-src/unix/vm/sqUnixExternalPrims.c ) static void *tryLoadModule(char *in, char *name) { char path[PATH_MAX], *out= path; void *handle= 0; int c; while ((c= *in++) && ':' != c) { /* copy next plugin path to path[] */ switch (c) { case '%': if ('n' == *in || 'N' == *in) { /* replace %n with name of plugin */ ++in; strcpy(out, name); out += strlen(name); continue; } if ('%' == *in) { ++in; *out++= '%'; continue; } /* fall through... */ default: *out++= c; continue; } } sprintf(out, "/" MODULE_PREFIX "%s" MODULE_SUFFIX, name); handle= dlopen(path, RTLD_NOW | RTLD_GLOBAL); fdebugf((stderr, "tryLoading(%s) = %p\n", path, handle));
// 8-10 wks - no silent failures if( !handle ){ // 8-10 - if a load fails, tell the user what we tried to load: fprintf(stdout, "Load failed for: %s\n", path);
// 8-10 - is the given path absolute? If so, try again with it // as given and report diagnostic information if it fails. if( (int)(name[0])==47 ) { handle=dlopen(name, RTLD_NOW | RTLD_GLOBAL); fprintf(stdout, "Load-by-name(%s) = %p\n", name, handle); if(!handle){ fprintf(stdout,"Error loading library: %s\n",dlerror()); } } }
if (!handle) { struct stat buf; if ((0 == stat(path, &buf)) && ! S_ISDIR(buf.st_mode)) fprintf(stderr, "%s\n", dlerror()); }
return handle; }
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Schwab,Wilhelm K [bschwab@anest.ufl.edu] Sent: Saturday, August 21, 2010 8:19 PM To: Pharo-project@lists.gforge.inria.fr; vm-dev@lists.squeakfoundation.org Subject: [Pharo-project] Linux vm
Hello all,
Is there a way to get the pre-compiled linux vm to describe what the full path is to a library it is trying to load? I can hack the source to do so, but then I start wondering what I might have disturbed in the process. Out of the box, all I get is an external library with a nil handle and no clue what did or did not happen.
At various points in my struggle today, I saw what looked like a tendency to load plugins from the vmpath//so.pluginName (two slashes rather than one). It raises the question of whether the code is adding an extra slash when it is not given an explicit plugins path?? The separate concern is that I need information about what is happening when one of my libraries fails to load.
One possibility is that a particular .so can't load. How would I find out whether that this is case? Is there a way to test load it or to get the vm to tell me the cause of a failure to load it?
Bill
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Hi Bill,
On Sat, Aug 21, 2010 at 7:39 PM, Schwab,Wilhelm K bschwab@anest.ufl.eduwrote:
It has been a long day, and it might not have needed to be so. Within 15 minutes of seeing "undefined symbol: libusb_close" from dlerror(), I had the problem not only diagnosed but fixed; seeing the error in something related to USB support was enough to suggest I needed to link that library into the .so I was trying to load. I was still guessing, but it was an educated guess based on a concrete lead.
Is there a way to get such diagnostic information without hacking the vm source? If not, please consider the following as a proposed patch. I have tried to strike a balance between seeing enough to make sense of errant plugin paths and giving detailed information about failures when given an absolute path. Behavior changes (I intend/think) only if something won't load (it says what it was trying to do) and if the library is specified by absolute path, in which case it gives diagnostic output on any failure to load.
What I do is manually repeat the link command using options that will cause the linker to complain about unresolved symbols. You'll get some false positives (dynamically-linked runtime routines) but you'll also get any actually undefined symbols. Use -Wl,--warn-unresolved-symbols -Wl,--no-allow-shlib-undefined when manually repeating the link command.
So while this is after the fact you should try remaking your plugin with the libusb support missing, copy the link command and repeat it adding -Wl,--warn-unresolved-symbols -Wl,--no-allow-shlib-undefined and this tie you should see the link fail while listing libusb_close et al.
HTH Eliot
Bill
( Squeak-...-src/unix/vm/sqUnixExternalPrims.c ) static void *tryLoadModule(char *in, char *name) { char path[PATH_MAX], *out= path; void *handle= 0; int c; while ((c= *in++) && ':' != c) { /* copy next plugin path to path[] */ switch (c) { case '%': if ('n' == *in || 'N' == *in) { /* replace %n with name of plugin */ ++in; strcpy(out, name); out += strlen(name); continue; } if ('%' == *in) { ++in; *out++= '%'; continue; } /* fall through... */ default: *out++= c; continue; } } sprintf(out, "/" MODULE_PREFIX "%s" MODULE_SUFFIX, name); handle= dlopen(path, RTLD_NOW | RTLD_GLOBAL); fdebugf((stderr, "tryLoading(%s) = %p\n", path, handle));
// 8-10 wks - no silent failures if( !handle ){ // 8-10 - if a load fails, tell the user what we tried toload: fprintf(stdout, "Load failed for: %s\n", path);
// 8-10 - is the given path absolute? If so, try again withit // as given and report diagnostic information if it fails. if( (int)(name[0])==47 ) { handle=dlopen(name, RTLD_NOW | RTLD_GLOBAL); fprintf(stdout, "Load-by-name(%s) = %p\n", name, handle); if(!handle){ fprintf(stdout,"Error loading library: %s\n",dlerror()); } } }
if (!handle) { struct stat buf; if ((0 == stat(path, &buf)) && ! S_ISDIR(buf.st_mode)) fprintf(stderr, "%s\n", dlerror()); }
return handle; }
From: pharo-project-bounces@lists.gforge.inria.fr [ pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Schwab,Wilhelm K [bschwab@anest.ufl.edu] Sent: Saturday, August 21, 2010 8:19 PM To: Pharo-project@lists.gforge.inria.fr; vm-dev@lists.squeakfoundation.org Subject: [Pharo-project] Linux vm
Hello all,
Is there a way to get the pre-compiled linux vm to describe what the full path is to a library it is trying to load? I can hack the source to do so, but then I start wondering what I might have disturbed in the process. Out of the box, all I get is an external library with a nil handle and no clue what did or did not happen.
At various points in my struggle today, I saw what looked like a tendency to load plugins from the vmpath//so.pluginName (two slashes rather than one). It raises the question of whether the code is adding an extra slash when it is not given an explicit plugins path?? The separate concern is that I need information about what is happening when one of my libraries fails to load.
One possibility is that a particular .so can't load. How would I find out whether that this is case? Is there a way to test load it or to get the vm to tell me the cause of a failure to load it?
Bill
Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Eliot,
Thanks for you prompt and helpful reply. I will admit I have not tried this yet, but I see where you are going and think it is better to cross-post this into Pharo-dev so it will be searchable via the Gmane archive than to actually see the messages that result; I will sooner or later get to the latter.
There is still the possible problem of an errant / in the default plugins path, and that the vm does not provide feedback on failures, at least not that I can find. Problems like I had yesterday go from a frustrating mystery to readily solved with a tiny hint that (in this case) is available from dlerror(); we should pass it along to the programmer.
Bill
==================================== Eliot Miranda eliot.miranda at gmail.com Sun Aug 22 02:53:30 UTC 2010
What I do is manually repeat the link command using options that will cause the linker to complain about unresolved symbols. You'll get some false positives (dynamically-linked runtime routines) but you'll also get any actually undefined symbols. Use -Wl,--warn-unresolved-symbols -Wl,--no-allow-shlib-undefined when manually repeating the link command.
So while this is after the fact you should try remaking your plugin with the libusb support missing, copy the link command and repeat it adding -Wl,--warn-unresolved-symbols -Wl,--no-allow-shlib-undefined and this tie you should see the link fail while listing libusb_close et al.
HTH Eliot
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Schwab,Wilhelm K [bschwab@anest.ufl.edu] Sent: Saturday, August 21, 2010 10:39 PM To: Pharo-project@lists.gforge.inria.fr; vm-dev@lists.squeakfoundation.org Subject: Re: [Pharo-project] Linux vm
It has been a long day, and it might not have needed to be so. Within 15 minutes of seeing "undefined symbol: libusb_close" from dlerror(), I had the problem not only diagnosed but fixed; seeing the error in something related to USB support was enough to suggest I needed to link that library into the .so I was trying to load. I was still guessing, but it was an educated guess based on a concrete lead.
Is there a way to get such diagnostic information without hacking the vm source? If not, please consider the following as a proposed patch. I have tried to strike a balance between seeing enough to make sense of errant plugin paths and giving detailed information about failures when given an absolute path. Behavior changes (I intend/think) only if something won't load (it says what it was trying to do) and if the library is specified by absolute path, in which case it gives diagnostic output on any failure to load.
Bill
( Squeak-...-src/unix/vm/sqUnixExternalPrims.c ) static void *tryLoadModule(char *in, char *name) { char path[PATH_MAX], *out= path; void *handle= 0; int c; while ((c= *in++) && ':' != c) { /* copy next plugin path to path[] */ switch (c) { case '%': if ('n' == *in || 'N' == *in) { /* replace %n with name of plugin */ ++in; strcpy(out, name); out += strlen(name); continue; } if ('%' == *in) { ++in; *out++= '%'; continue; } /* fall through... */ default: *out++= c; continue; } } sprintf(out, "/" MODULE_PREFIX "%s" MODULE_SUFFIX, name); handle= dlopen(path, RTLD_NOW | RTLD_GLOBAL); fdebugf((stderr, "tryLoading(%s) = %p\n", path, handle));
// 8-10 wks - no silent failures if( !handle ){ // 8-10 - if a load fails, tell the user what we tried to load: fprintf(stdout, "Load failed for: %s\n", path);
// 8-10 - is the given path absolute? If so, try again with it // as given and report diagnostic information if it fails. if( (int)(name[0])==47 ) { handle=dlopen(name, RTLD_NOW | RTLD_GLOBAL); fprintf(stdout, "Load-by-name(%s) = %p\n", name, handle); if(!handle){ fprintf(stdout,"Error loading library: %s\n",dlerror()); } } }
if (!handle) { struct stat buf; if ((0 == stat(path, &buf)) && ! S_ISDIR(buf.st_mode)) fprintf(stderr, "%s\n", dlerror()); }
return handle; }
________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Schwab,Wilhelm K [bschwab@anest.ufl.edu] Sent: Saturday, August 21, 2010 8:19 PM To: Pharo-project@lists.gforge.inria.fr; vm-dev@lists.squeakfoundation.org Subject: [Pharo-project] Linux vm
Hello all,
Is there a way to get the pre-compiled linux vm to describe what the full path is to a library it is trying to load? I can hack the source to do so, but then I start wondering what I might have disturbed in the process. Out of the box, all I get is an external library with a nil handle and no clue what did or did not happen.
At various points in my struggle today, I saw what looked like a tendency to load plugins from the vmpath//so.pluginName (two slashes rather than one). It raises the question of whether the code is adding an extra slash when it is not given an explicit plugins path?? The separate concern is that I need information about what is happening when one of my libraries fails to load.
One possibility is that a particular .so can't load. How would I find out whether that this is case? Is there a way to test load it or to get the vm to tell me the cause of a failure to load it?
Bill
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On Sunday 22 Aug 2010 8:09:22 am Schwab,Wilhelm K wrote:
Is there a way to get such diagnostic information without hacking the vm source? If not, please consider the following as a proposed patch. I have tried to strike a balance between seeing enough to make sense of errant plugin paths and giving detailed information about failures when given an absolute path
In your place, I would have tried ldd(1) or strace(1) to detect path load errors.
In general, failing to load a plugin module is not a fatal error. Sometimes, a primtive can fallback to using internal routines if the plugin is not available.
Subbu
Subbu,
It can still be a serious problem as performance can suffer. In fact, in this case, I am talking about Shared Libraries, not plugins. There is no fallback code; one succeeds in loading the library or the game is over.
strace might do the job, but what is wrong with having the vm log what it is trying to do?
Bill
________________________________________ From: K. K. Subramaniam [kksubbu.ml@gmail.com] Sent: Monday, August 23, 2010 2:33 AM To: vm-dev@lists.squeakfoundation.org Cc: Schwab,Wilhelm K; Pharo-project@lists.gforge.inria.fr Subject: Re: [Vm-dev] RE: Linux vm
On Sunday 22 Aug 2010 8:09:22 am Schwab,Wilhelm K wrote:
Is there a way to get such diagnostic information without hacking the vm source? If not, please consider the following as a proposed patch. I have tried to strike a balance between seeing enough to make sense of errant plugin paths and giving detailed information about failures when given an absolute path
In your place, I would have tried ldd(1) or strace(1) to detect path load errors.
In general, failing to load a plugin module is not a fatal error. Sometimes, a primtive can fallback to using internal routines if the plugin is not available.
Subbu
On Monday 23 Aug 2010 6:49:43 pm Schwab,Wilhelm K wrote:
strace might do the job, but what is wrong with having the vm log what it is trying to do?
You could also use ltrace(1).
The vm already logs entries in debug mode. Non-debug run-times can be traced with strace/ltrace. Performance can be optimized by changing LD_LIBRARY_PATH and -plugins options in the launcher script.
In any case, you can send in your suggestions (complete file, not a diff) for Ian's consideration.
Subbu
Subbu,
What is the debug mode? Is it a build option, or something more readily available to an end user?
As for the code, I provided the entire function. Depending on what falls out of the question above, I might find that all is well as-is; otherwise I can send the entire file (where?) if needed.
Bill
________________________________________ From: K. K. Subramaniam [kksubbu.ml@gmail.com] Sent: Monday, August 23, 2010 11:06 AM To: Schwab,Wilhelm K Cc: vm-dev@lists.squeakfoundation.org Subject: Re: [Vm-dev] RE: Linux vm
On Monday 23 Aug 2010 6:49:43 pm Schwab,Wilhelm K wrote:
strace might do the job, but what is wrong with having the vm log what it is trying to do?
You could also use ltrace(1).
The vm already logs entries in debug mode. Non-debug run-times can be traced with strace/ltrace. Performance can be optimized by changing LD_LIBRARY_PATH and -plugins options in the launcher script.
In any case, you can send in your suggestions (complete file, not a diff) for Ian's consideration.
Subbu
On Monday 23 Aug 2010 8:47:31 pm Schwab,Wilhelm K wrote:
Subbu,
What is the debug mode? Is it a build option, or something more readily available to an end user?
It is build option (see debug.h) -DDEBUG=1. If turned on, you get detailed dump on stderr.
As for the code, I provided the entire function. Depending on what falls out of the question above, I might find that all is well as-is; otherwise I can send the entire file (where?) if needed.
See Ian's announcement in http://lists.squeakfoundation.org/pipermail/vm- dev/2009-September/003206.html
Subbu
vm-dev@lists.squeakfoundation.org
-
Eliot Miranda -
K. K. Subramaniam -
Schwab,Wilhelm K