--- Roger Vossler rvossler@qwest.net wrote:
I read the material concerning Block Closures (BC) on the GaTech web site, but it's a bit beyond my meager understanding.
Sorry it is just technical, hopefully the answers below will help. Others may want to add to my answers.
I have a couple of questions:
- Why is BC so important?
BC speeds up the Interpreter and correctly interprets the scopes of blocks. To elaborate on the second point: Blocks holding their own temps (block closures) is more correct from a scoping point of view than method contexts holding the temps for its embedded blocks (current Squeak). For example: #(a b c) do: [:s | [Transcript show: s] fork] will print 'abc' in the BC version but 'ccc' in the current Squeak version. The BC version is correctly scoped because the block variable s has a different binding on each iteration. However, in the current Squeak version the block variable s has only one binding held in the home method context and just changes value on each iteration. By the time the forked blocks are executed s is holding the last value.
- What does one gain from this capability?
You gain correctly scoped blocks, which allows you to more easily create truely independent functions/blocks. Sometimes this is preferred over creating special small "function" objects or using additional methods just so you have another scope to bind temps to. For examples of sophisticated uses of blocks that would not work in the current Squeak implementation unless they were rewritten using more methods/classes, see the block closure test cases in the BC image or on the BC swiki page in the Convert section.
- Do most other Smalltalk systems have this capability?
Yes.
- Is this capability "nice to have" or is it "really essential"?
It is not essential since code like the example above can be rewritten to use an intermediate method to provide the missing scope. For example, the following will work correctly in the current Squeak implementation: UndefinedObject compile: 'forkWriting: s [Transcript show: s] fork'. #(a b c) do: [:s | self forkWriting: s] BC just makes it much easier to make these blocks independently scoped.
- If BC is "really essential", how has Squeak managed to get along
without it thus far?
Block closure is not essential as noted above. But it does force people to sometimes write code the long way like the example above.
I'm not looking for detailed technical information, but rather, an indication of why the Squeak community needs to make this change which apparently has big impacts on image formats, VMs, et al. As someone noted: we don't want to do this more than once.
Two main reasons why Squeak needs BC are: 1. Block closures conform to the scoping rules of [...] (BC is more correct). 2. This BC implementation is faster. Since the interpreter had to be changed anyway to accomodate block closures I took the opportunity to make further changes to speed up the whole interpreting process. Either of these may be important enough, but together, I think they (plus possibly other VI4 enhancements) makes it worth the pain to switch the community to a new image format.
Of course, we can make the switch as painless as possible by providing the image converter with the new VM and have the new VM trigger the converter when it tries to open an old image. The major problem is that image conversion takes a long time (about an hour on a 3.2 image on my 533MHz Intel). Also trying to open a new image with an old VM will just report a bad format message.
Cheers, Anthony
__________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com
As Anthony pointed out the key factor is that he has implemented proper Closure behaviour. In the process he sucked in my new CompiledMethod format changes, fixed a few method header infelicities and reimplemented the callstack mechanism. The latter is responsible for most of the performance improvement; for some explanation of what happens you might find it useful to read Eliot Miranda's OOPSLA 97 paper on BrouHaHa. Anthony implemented a variant of that stack mechanism but exposed it to the Smalltalk code rather than hiding it. I'm not entirely sure I like that part, but I haven't had time to examine it properly yet. The worst problem of which I'm aware is the lack of support for swapSender: and callCC - important for coroutining etc.
The VI4 project is trying to gather together all the plausible changes that peolpe want and which would require an image format change or would other wise break back compatibility. For example, renaming the plugins more sensibly ( change Klatt to KlattPlugin, etc) would save some confusion (ever got lost trying to work out where the code for ZipPlugin is? - in InflatePlugin & DeflatePlugin classes!) but means changing assorted image code.
We could even convert all bitmaps to little-endian, since only Mac uses big-endian and we know that nobody important uses Macs.
tim
Tim Rowledge tim@sumeru.stanford.edu said:
[...] Anthony implemented a variant of that stack mechanism but exposed it to the Smalltalk code rather than hiding it. I'm not entirely sure I like that part, but I haven't had time to examine it properly yet. The worst problem of which I'm aware is the lack of support for swapSender: and callCC - important for coroutining etc.
That would be Bad. It would break existing code (Seaside) and it would close the door to interesting experiments.
Is this exposure something intrinisic to the BC image or would it be possible to wrap a compatibility interface around it?
We could even convert all bitmaps to little-endian, since only Mac uses big-endian and we know that nobody important uses Macs.
Shouldn't we support both endians? If an image opens with big-endian bitmaps on a little endian machines, couldn't we automatically correct them on startup? Sure it will take time, but if startup time is important, just save your image and the bitmaps get saved in the native endianness. I suppose you might need a class for each endianness (LittleEndianBitmap, BigEndianBitmap).
- Stephen
Stephen,
We could even convert all bitmaps to little-endian, since only Mac uses big-endian and we know that nobody important uses Macs.
Shouldn't we support both endians?
We do. Or how else would the system work?! The issue is not about _bitmaps_ (those are fine whatever platform you're on) but about the interpretation of what _pixels_ are where in any given Form. So it's about forms not bitmaps.
If an image opens with big-endian bitmaps on a little endian machines, couldn't we automatically correct them on startup? Sure it will take time, but if startup time is important, just save your image and the bitmaps get saved in the native endianness. I suppose you might need a class for each endianness (LittleEndianBitmap, BigEndianBitmap).
Again, this issue is not about bitmaps. It's about forms which do already include information about their 'supposed' endianness. And yes you could do what you're describing (and even do so in a pretty fast way if you look at Form>>unhibernate). It just needs to be figured out if it's worth the effort or not.
Cheers, - Andreas
Stephen,
We could even convert all bitmaps to little-endian, since only Mac uses big-endian and we know that nobody important uses Macs.
Shouldn't we support both endians?
We do. Or how else would the system work?! The issue is not about _bitmaps_ (those are fine whatever platform you're on) but about the interpretation of what _pixels_ are where in any given Form. So it's about forms not bitmaps.
Yes of course...by bitmap, I think Tim meant "bitmaps that get displayed" (or Form).
If an image opens with big-endian bitmaps on a little endian machines, couldn't we automatically correct them on startup? Sure it will take time, but if startup time is important, just save your image and the bitmaps get saved in the native endianness. I suppose you might need a class for each endianness (LittleEndianBitmap, BigEndianBitmap).
Again, this issue is not about bitmaps. It's about forms which do already include information about their 'supposed' endianness. And yes you could do what you're describing (and even do so in a pretty fast way if you look at Form>>unhibernate). It just needs to be figured out if it's worth the effort or not.
It doesn't seem like it should be a huge effort (especially if little and big endianness are already accomodated by Form). And for those multi-media apps, every little bit of performance improvement helps.
- Stephen
Stephen Pair wrote:
Yes of course...by bitmap, I think Tim meant "bitmaps that get displayed" (or Form).
Actually, what Tim meant on this occasion was a little joke.
Supporting different pixel formats (it's not just little vs big endian, not by a long way) is a lot more than simply changing the bitmap layout; there's quite large changes to the inner functioning of bitblt etc. Fortunately, Andreas & I spent a couple of years arguing about this and eventually FxBlt and the SurfacePlugin resulted. IIRC most of (even all by now?) FxBlt has migrated into BitBlt over time and I really must get around to trying to use it someday. It's on my list behind minor things like trying to make enough money to keep eating, VI4 & VMbuild projects, repairing half a dozen crunched model planes, making a boatload of furniture etc etc.
tim
I would like to know from SqC guys and other what is missing in BC that would impede its integration into the main image (like a decompiler...)?
I would like to know if the new architecture is a problem for other processors.
I think that it would be good to have a list of things that should be developped. I'm really concerned by the fact BC is a big changes and that some people may be afraid to change. This would be a pity to have a big change discarded just because this is a big change.
Stef
On Tuesday, April 30, 2002, at 09:32 PM, Anthony Hannan wrote:
--- Roger Vossler rvossler@qwest.net wrote:
I read the material concerning Block Closures (BC) on the GaTech web site, but it's a bit beyond my meager understanding.
Sorry it is just technical, hopefully the answers below will help. Others may want to add to my answers.
I have a couple of questions:
- Why is BC so important?
BC speeds up the Interpreter and correctly interprets the scopes of blocks. To elaborate on the second point: Blocks holding their own temps (block closures) is more correct from a scoping point of view than method contexts holding the temps for its embedded blocks (current Squeak). For example: #(a b c) do: [:s | [Transcript show: s] fork] will print 'abc' in the BC version but 'ccc' in the current Squeak version. The BC version is correctly scoped because the block variable s has a different binding on each iteration. However, in the current Squeak version the block variable s has only one binding held in the home method context and just changes value on each iteration. By the time the forked blocks are executed s is holding the last value.
- What does one gain from this capability?
You gain correctly scoped blocks, which allows you to more easily create truely independent functions/blocks. Sometimes this is preferred over creating special small "function" objects or using additional methods just so you have another scope to bind temps to. For examples of sophisticated uses of blocks that would not work in the current Squeak implementation unless they were rewritten using more methods/classes, see the block closure test cases in the BC image or on the BC swiki page in the Convert section.
- Do most other Smalltalk systems have this capability?
Yes.
- Is this capability "nice to have" or is it "really essential"?
It is not essential since code like the example above can be rewritten to use an intermediate method to provide the missing scope. For example, the following will work correctly in the current Squeak implementation: UndefinedObject compile: 'forkWriting: s [Transcript show: s] fork'. #(a b c) do: [:s | self forkWriting: s] BC just makes it much easier to make these blocks independently scoped.
- If BC is "really essential", how has Squeak managed to get along
without it thus far?
Block closure is not essential as noted above. But it does force people to sometimes write code the long way like the example above.
I'm not looking for detailed technical information, but rather, an indication of why the Squeak community needs to make this change which apparently has big impacts on image formats, VMs, et al. As someone noted: we don't want to do this more than once.
Two main reasons why Squeak needs BC are:
- Block closures conform to the scoping rules of [...] (BC is more
correct). 2. This BC implementation is faster. Since the interpreter had to be changed anyway to accomodate block closures I took the opportunity to make further changes to speed up the whole interpreting process. Either of these may be important enough, but together, I think they (plus possibly other VI4 enhancements) makes it worth the pain to switch the community to a new image format.
Of course, we can make the switch as painless as possible by providing the image converter with the new VM and have the new VM trigger the converter when it tries to open an old image. The major problem is that image conversion takes a long time (about an hour on a 3.2 image on my 533MHz Intel). Also trying to open a new image with an old VM will just report a bad format message.
Cheers, Anthony
Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com
--- stephane ducasse ducasse@iam.unibe.ch wrote:
I would like to know from SqC guys and other what is missing in BC that would impede its integration into the main image (like a decompiler..)?
Of course, the decompiler, swapSender: and everything else on the BC To-Do list (http://minnow.cc.gatech.edu/squeak/BlockClosureVersion) will be implemented before being released into the main image.
I would like to know if the new architecture is a problem for other processors.
Since BC only makes changes to the Interpreter slang code, I think it will likely work on other platforms. The problem I had a couple of months ago on different platforms was rare and avoidable in the future. I assumed that the order of execution of expressions in a single C statement (and thus slang statement) was left-to-right, but actually C says expressions in a single statement can be executed in any order, presumably for flexibility of the optimizer. I got bitten by this, but found and fixed it, and will be aware of it in the future. Otherwise the new BC Interpreter slang code is similar to the previous code, nothing fancy.
I think that it would be good to have a list of things that should be developped.
See (and add to) the BC To-Do list mentioned above.
I'm really concerned by the fact BC is a big changes and that some people may be afraid to change. This would be a pity to have a big change discarded just because this is a big change.
I think people will be willing to make the change if its worth it, especially if its faster. I still have some things I want to put into the next release to make it even faster, including Scott Crosby's method cache enhancements.
Cheers, Anthony
__________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com
Anthony Hannan wrote:
Two main reasons why Squeak needs BC are:
- Block closures conform to the scoping rules of [...] (BC is more correct).
- This BC implementation is faster. Since the interpreter had to be changed
anyway to accomodate block closures I took the opportunity to make further changes to speed up the whole interpreting process. Either of these may be important enough, but together, I think they (plus possibly other VI4 enhancements) makes it worth the pain to switch the community to a new image format.
stephane ducasse (home) wrote:
I would like to know from SqC guys and other what is missing in BC that would impede its integration into the main image (like a decompiler...)?
I would like to know if the new architecture is a problem for other processors.
I think that it would be good to have a list of things that should be developped. I'm really concerned by the fact BC is a big changes and that some people may be afraid to change. This would be a pity to have a big change discarded just because this is a big change.
I think Stephane has an important point here. Especially Anthony should work closely with SqC, in particular Dan, to make sure that the BC code is ok and clear for integration into V4. Otherwise there is a risk that this could go to waste. Good work has been known to go to waste before, causing people to leave Squeak in anger.
One cannot take this integration for granted just because there are N people who exclaim "Cool!" on the list. There could be deeper technical reasons for why it is not feasible; very few Squeakers fully understand all the details of what it takes for something to be solid. I don't. But there is clearly still a list of issues to resolve, like decompiling and so on.
If there are any technical problems these need to be resolved. If there are political problems then you need to assemble your allies and prepare for battle :-)
But hey, it was possible to have the rounded corners turned off by default, based on a purely rational argument, so nothing is impossible!
Now if only the dozens of people who promised to help with the modules would actually make good on their promises, then things would look really good.
Henrik
On Wed, 1 May 2002, Henrik Gedenryd wrote: [snip]
I think Stephane has an important point here. Especially Anthony should work closely with SqC, in particular Dan, to make sure that the BC code is ok and clear for integration into V4.
Is there really any reason to think that this isn't happening? Tim seems to be on the ball with V4.
Changing the image format is a *huge* change, after all.
[snip]
Now if only the dozens of people who promised to help with the modules would actually make good on their promises, then things would look really good.
IIRC, I only promises to kvetch and complain no matter *what* design was put forth. I admit that I've failed on this front but I'm sure that's not a problem :)
SqueakEnd is nigh. Dan will be giving a talk about modules. There are some projects for modularizing chunks. Hacking will happen. A prep list of Tasks To Do would likely stimulate some work.
One question I have is about what happens if you do the little "replace the pool&category part of the class definition with module ref" on a change set...do the things filed in end up inside the mentioned modules?
Is it worth setting up a more sophsticated mapping from old categories to new modules so we can more painlessly file in old stuff? Any hints on how to do this? :)
Cheers, Bijan Parsia.
Hi
We started to work with henrik and this is first draft fo what we made.
http://scgwiki.iam.unibe.ch:8080/SmalltalkWiki/156
henrik will certainly publish something better in the near future.
Stef
SqueakEnd is nigh. Dan will be giving a talk about modules. There are some projects for modularizing chunks. Hacking will happen. A prep list of Tasks To Do would likely stimulate some work.
One question I have is about what happens if you do the little "replace the pool&category part of the class definition with module ref" on a change set...do the things filed in end up inside the mentioned modules?
Is it worth setting up a more sophsticated mapping from old categories to new modules so we can more painlessly file in old stuff? Any hints on how to do this? :)
Cheers, Bijan Parsia.
On Wed, 1 May 2002, stephane ducasse wrote:
Hi
We started to work with henrik and this is first draft fo what we made.
http://scgwiki.iam.unibe.ch:8080/SmalltalkWiki/156
henrik will certainly publish something better in the near future.
I put a link to this on the SqueakEnd/Conference Swiki:
http://coweb.cc.gatech.edu/mmWorkshop/28
Cheers, Bijan Parsia.
Bijan Parsia wrote:
I put a link to this on the SqueakEnd/Conference Swiki:
http://coweb.cc.gatech.edu/mmWorkshop/28
Cheers, Bijan Parsia.
We 've now moved these pages to Minnow, see
http://minnow.cc.gatech.edu/squeak/Modules
the refatoring guide page is still in progress.
This page has been significantly expanded (I counted to about 30-35 pages in all).
There is now a brief
http://minnow.cc.gatech.edu/squeak/ModulesFAQ
that you can add to, and a new "Modules: progress made so far"
http://minnow.cc.gatech.edu/squeak/ModulesProgressMade
with lots of information and demos of the current state of the project.
Alexandre wrote a quick intro to modules with storing and loading.
http://minnow.cc.gatech.edu/squeak/SimpleModuleExample
SqueakEnd is nigh. Dan will be giving a talk about modules. There are some projects for modularizing chunks. Hacking will happen. A prep list of Tasks To Do would likely stimulate some work.
Such as
http://minnow.cc.gatech.edu/squeak/ModulesToDo
or
http://minnow.cc.gatech.edu/squeak/ToDoModularizingImage
? Essentially, modularizing is the top pripority, we're writing how to:s and I have tried to set some priorities in the second of these pages.
One question I have is about what happens if you do the little "replace the pool&category part of the class definition with module ref" on a change set...do the things filed in end up inside the mentioned modules?
The original text of
http://minnow.cc.gatech.edu/squeak/PortToModules
was right, you only need to edit the category string, see the corrected page now.
Henrik
From: Bijan Parsia
One question I have is about what happens if you do the little "replace the pool&category part of the class definition with module ref" on a change set...do the things filed in end up inside the mentioned modules?
Unless I'm misunderstanding something this works now... (At least in 3.3alpha - #4843). At least that's what I did to Bruce O'Neal's OPML & LWxml (lightweight XML) parsers (found at http://homepage.iprolink.ch/~bioneel/beo/Squeak/) to get them to filein. And a few other things I was interested in looking at.
Speaking of such things...
Is there an explanation of the Temporary category and how some things ended up there (and whether it's safe to move them elsewhere)?
Somewhere ?
-Andy-
Andy Stoffel wrote:
Speaking of such things...
Is there an explanation of the Temporary category and how some things ended up there
Yup. You had added things in the image that weren't part of the base distribution, so the converter didn't know where to put them, so it moved them to Temporary.
(and whether it's safe to move them elsewhere)?
Somewhere ?
Yup. For example #(People <yourInitials> <yourThing>) or #(Project <theProjectName>).
This is now at
http://minnow.cc.gatech.edu/squeak/ModulesFAQ
Henrik
squeak-dev@lists.squeakfoundation.org
-
Andreas Raab -
Andy Stoffel -
Anthony Hannan -
Bijan Parsia -
cg＠cdegroot.com -
Henrik Gedenryd -
stephane ducasse -
Stephen Pair -
Tim Rowledge