From: Mats Nygren nygren@sics.se
I have tried to delete FFI-references fore awhile. Hiding files in other directories, removing code from files including from generated files, the mess is still growing. Surely this is not how it should work. Adding a particular module shouldn't be able to mess up the whole machine generation process.
Agreed. I mentioned some of these problems in the message announcing 2.8pre2. It would help enormously if:
- file names were consistent, i.e. FooPlugin contains the plugin, with all the EXPORT declarations and suchlike; then
- sqFooPrims contains any generated primitives on which FooPlugin depends;
- thinks like Squeak3D didn't have zillions of smaller b3dXYZ files (with no obvious relation to their "dependent") as dependencies.
This last one really bothers me. I can see no good reason why Squeak3D should be generated along with the interp.c and other support files, and then be included in the sqNamedPrims. It has worked perfectly as a module for ages, has never needed to be listed in the sqNamedPrims, and lived quite happily in its own dir where the dependencies were obvious and easy to manage.
Unless this situation is sorted out soon I'm going to abandon the part of configure that deduces dependencies automatically, and insist on a file that contains explicit dependency information for the plugins.
This will be a real pain, because it would have to be edited whenever a plugin were added/removed/renamed/whatever. But the alternative (a growing list of completely random rules for figuring out which files belong in which modules) is a worse pain, and utterly unmanageable.
For a unix machine I see two alternatives (there should be more):
- Control the build process from within Squeak and have a description
in Squeak-syntax of the VM. (What modules should be included etc) This would need OSProcess and probably be unix-specific, in the beginning at least.
Simply a file containing lines of the form
NameOfMainPlugin.c [AuxPrimsFile1.c [AuxPrimsFile2.c [...]]]
would be more than enough to solve the problem, and isn't specific to any OS. (W.r.t. OPP: there's absolutely no need to use an elephant gun just to squash a bug.)
- Have a simple way of describing parameters for the build, from
outside of Squeak. Again this would probably be unix-specific in the beginning.
Arranging for the generated files to take note of a -DPLUGIN symbol to compile themselves in "external plugin mode", for example, would be perfect. I could trivially rebuild the sqNamedPrims.h file in the process, to reflect which plugins were "externalised".
Manually changing generated files should never be a part of describing things the new VM.
Agreed!
FWIW, anyone who has regenerated the source files from inside the image will also notice a Unix-specific FFI prims file that comes out in the mix. If this files isn't deleted then lots of other things will blow up (and probably go "pop!" too).
I find this very irritating at the moment, having decided to work with the syntactic part of VM-generation
Yes!
I don't want learn m4, configure, etc at the moment. (In fact I would like to avoid those for ever if possible)
Exactly! That's part of *my* job, not yours. And for the time being the only realistic solutions are:
- reorganise the naming and mix of files that are generated automatically along with interp.c; or
- give up on trying to figure out what belongs where (which was working perfectly right up the final Mac 2.8 when suddenly Squeak3D, SoundCodec and the FFI stuff [named all wrong] popped out of the translation process) and use a hand-edited plugin description file instead.
Could somebody fix this please.
For building when libffi isn't there the fix is trivial. I'll do it today.
For the organisation, that has to be fixed by somebody who has the right to meddle with the files generated by/stored in the image. Which isn't me. Andreas and John Mc are probably the most likely candidates.
The particular problem with FFI, since according to Bert (below) this isn't necesserary what everybody wants, why not make configure arrange for it to be left out, if it isn't found on the machine?
This is the intention, and this is how I thought it worked.
But note again: I had to remove an unwanted file from the generation process, and add a #ifdef NO_FFI_SUPPORT (or something similar, I think I "autoconfified" the name to HAVE_LIBFFI -- but it was missing completely in the first place, so I don't feel guilty about that) which was simply missing in one of the generated files. If you've regenerated the support files from within the image then zillions of things will (and should) break -- since the generated files are wrong.
If this is difficult to arrange then some things is severely wrong with the current organization of the build process.
It's not difficult to arrange, it just requires a little time and motivation from someone in charge of the image side of things.
Thanks for the help so far. I do wish to be able to work with this without first becoming an expert of every detail of the build process.
I know how you feel. (But as a minor consolation: if you've learned something this week, you never know when you'll be really glad to have that knowledge.)
Since FFI is a major security risc I normally don't want it to be compiled into the VM. I also set SQUEAK_SECURE. Am I paranoid? ;-)
It's not really that much more of a risk than the current named prims, which give access to every single system call on the machine from within Squeak. (It just makes passing arguments to them a little less problematic.)
Ciao,
Ian
On Wed, 30 Aug 2000, Ian Piumarta wrote:
Since FFI is a major security risc I normally don't want it to be compiled into the VM. I also set SQUEAK_SECURE. Am I paranoid? ;-)
It's not really that much more of a risk than the current named prims, which give access to every single system call on the machine from within Squeak. (It just makes passing arguments to them a little less problematic.)
Really? I thought with the non-FFI prims you can only load Squeak modules, which are distinguished by defining setInterpreter. It should not be possible to load any other shared object. And for the VM itself, the only the tables are used, not dlsym.
-- Bert
In message Pine.LNX.4.21.0008301721210.8315-100000@balloon.cs.uni-magdeburg.de Bert Freudenberg bert@isg.cs.uni-magdeburg.de wrote:
On Wed, 30 Aug 2000, Ian Piumarta wrote:
Since FFI is a major security risc I normally don't want it to be compiled into the VM. I also set SQUEAK_SECURE. Am I paranoid? ;-)
It's not really that much more of a risk than the current named prims, which give access to every single system call on the machine from within Squeak. (It just makes passing arguments to them a little less problematic.)
Really? I thought with the non-FFI prims you can only load Squeak modules, which are distinguished by defining setInterpreter. It should not be possible to load any other shared object. And for the VM itself, the only the tables are used, not dlsym.
That is supposed to be the behaviour but this ld.so thingy is being too 'helpful'. On less 'helpful' platforms, a random OS or VM function cannot be called. Personally, I find that reassuring.
tim
Arranging for the generated files to take note of a -DPLUGIN symbol to compile themselves in "external plugin mode", for example, would be perfect. I could trivially rebuild the sqNamedPrims.h file in the process, to reflect which plugins were "externalised".
It isn't clear to me whether you'all in this thread have noted that the Slang translator generates different C code for modules (and the VM hooks I'd suppose) depending on whether they are to be compiled with the VM or as separate plugin files. In other words, plugins generated for inclusion with the vm should not be compiled as standalone, and vice versa. Check out the changes inthe InterpreterPlugin class>>translation category (doh! obviously). 'locally' there means 'baked into the vm'. There are also way simple hooks for eg. leaving FFI out of the vm.
Someone made this remark before (Tim I think it was) and just ignore me if you already know this. However, it seems that this might be the solution to your problems.
As for the inconsistent naming, these are probably remains from plugins older than the new pluginize paradigm. I think anyone should feel free to fix the naming.
Try this (replace 'System' with any plugin that you know exists and can be
found
by the VM):
'From Squeak2.8 of 13 June 2000 [latest update: #2359] on 30 August 2000 at
5:44
:26 pm'!
!SystemDictionary methodsFor: 'miscellaneous' stamp: 'ikp 8/30/2000 17:44'! byeBye "Smalltalk byeBye"
<primitive: 'exit' module: 'System'> self error: 'this should never be reached'! !
Ouch, this looks like a bug to me, but my Unix eyesight isn't 20/20. Tried it--does not work on the Mac. What about Win?
<Bert just removed all of Squeak from his hard drive :-) >
Henrik
Henrik Gedenryd Henrik.Gedenryd@lucs.lu.se is widely believed to have written:
Arranging for the generated files to take note of a -DPLUGIN symbol to compile themselves in "external plugin mode", for example, would be perfect. I could trivially rebuild the sqNamedPrims.h file in the process, to reflect which plugins were "externalised".
It isn't clear to me whether you'all in this thread have noted that the Slang translator generates different C code for modules (and the VM hooks I'd suppose) depending on whether they are to be compiled with the VM or as separate plugin files. In other words, plugins generated for inclusion with the vm should not be compiled as standalone, and vice versa. Check out the changes inthe InterpreterPlugin class>>translation category (doh! obviously). 'locally' there means 'baked into the vm'. There are also way simple hooks for eg. leaving FFI out of the vm.
Someone made this remark before (Tim I think it was) and just ignore me if you already know this. However, it seems that this might be the solution to your problems.
I would think that it would be. Some cleanup is probably needed somewhere, but basically modules can be generated as internal or external and the generated code should go to a suitable place automagically.
See InterpreterPlugin class> translate:doInline:locally: for details.
EXPORTed entries in modules generated for internal linking have entries in sqNamedPrimitives.h and the others don't. They are linked with the plugin stuff later.
As I've suggested several times before, a lot of this confusion could be removed if someone could knock up a little tool that lists all the translatable components and allows one to tick each one that is to be made internal to the vm.
Currently you have to go to the unnacceptably enormous effort of editing the Interpreter class>translate:doInlining:forBrowserPlugin: method to remove any plugins you don't want internal, and then you have actually evaluate some code (gasp!) to generate the other modules for external usage. something like:- {AsynchFilePlugin. FilePlugin. MIDIPlugin. SerialPlugin. SocketPlugin. SoundPlugin. JoystickTabletPlugin. LargeIntegersPlugin} do: [:pl| pl translate] ... for example. It's hard work I know, but think of the glory. This snippet would generate directories (OK, folders for you MacMinions) with the plugin code in them. If you move your platform specific files into the right places, everything else should work. Make can work out what plugins are internal/external trivially, sqNamedPrimiitves.h will be correct anyway and all is well with the world. Which is why we wrote the code to do it that way. You could of course read http://minnow.cc.gatech.edu/squeak/811 and http://minnow.cc.gatech.edu/squeak/1448 and find out what we documented several months ago.
As for the inconsistent naming, these are probably remains from plugins older than the new pluginize paradigm. I think anyone should feel free to fix the naming.
Exactly. Andreas could probably think of consistent names for the B3D in seconds and other authors likewise.
Try this (replace 'System' with any plugin that you know exists and can be
found
by the VM):
[snip]
Ouch, this looks like a bug to me, but my Unix eyesight isn't 20/20. Tried it--does not work on the Mac. What about Win?
IIRC, only unix is this 'helpful'
tim
Ian Piumarta wrote:
- Control the build process from within Squeak and have a description
in Squeak-syntax of the VM. (What modules should be included etc) This would need OSProcess and probably be unix-specific, in the beginning at least.
Simply a file containing lines of the form
NameOfMainPlugin.c [AuxPrimsFile1.c [AuxPrimsFile2.c [...]]]
would be more than enough to solve the problem, and isn't specific to any OS. (W.r.t. OPP: there's absolutely no need to use an elephant gun just to squash a bug.)
I was just playing around with the Regular Expression plugin. This uses sources from a third-party regex library, written without knowledge of Squeak. And it really doesn't warrant being put in yet another DLL.
The problem is here that the extra sources (not the main plugin file, but the regex engine) need to have a couple of compile flags provided (defines) to compile right.
Perhaps the solution described above would be able to take such compiler flags into account...
If you disconnect your machine from the internet or your intranet, then start Squeak with a 2.8 VM and attempt to connect to a remote host using the host IP address versus it's domain name, you will crash your powerbook.
However if you need to first resolve a domain name Squeak will complain and stop, but this isn't always the case with some of the Squeak internet code where a name might have been resolved to an IP address before you disconnected.
To fix the problem I had to make a change to the sqMacNetwork.c code. I will be issuing a new Macintosh VM (2.8.2) in a day or two once I've confirmed that any issues with sleeping a powerbook, and/or disconnecting from ethernet or a dialup PPP connection have been resolved. It also means a new NS Plugin, and a 2.9 VM is also forthcoming.
For the few that do their own compiles and want a fix right away:
The three uses of internalSocketCreate
become
error = internalSocketCreate(...); if (error != noErr) { interpreterProxy->success(false); return; }
and the following routine was rewritten:
static EPInfo* getOrMakeMeAnEP(UInt8 aSocketType,short counter) { EPInfo *epi; OTLink *link; SInt32 i;
Recycle(); //Ensure broken EP get fixed up
if (counter > 25) return nil; // End recursion John 2000/8/29
if (gIdleEPCounter[aSocketType] < 5) //Magic Number ensure we have at least 5 EP available. makeMeAnEP(aSocketType);
link = OTLIFODequeue(gIdleEPs[aSocketType]); if (link == NULL) { for(i=0;i<10;i++) {OTIdle();}; return getOrMakeMeAnEP(aSocketType,counter+1); //Watch for recursive failure }
OTAtomicAdd32(-1, &gIdleEPCounter[aSocketType]); epi = OTGetLinkObject(link, EPInfo, link);
if (OTAtomicTestBit(&epi->stateFlags3, kSleepKilledMe)) { // //A broken epi on the idle stack, now the only way we can //get here (I think) is to have gone to sleep which breaks //all the end points. To clean up we must now fix them //So make it idle, of course it's broken //Then recursive call to get another one //This continues until we get a good one // makeEPIdle(epi); return getOrMakeMeAnEP(aSocketType,counter); //Not a recursion issue. } return epi; }
-- =========================================================================== John M. McIntosh johnmci@smalltalkconsulting.com 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com =========================================================================== Custom Macintosh programming & various Smalltalk dialects PGP Key: DSS/Diff/46FC3BE6 Fingerprint=B22F 7D67 92B7 5D52 72D7 E94A EE69 2D21 46FC 3BE6 ===========================================================================
squeak-dev@lists.squeakfoundation.org