While I find all this discussion fascinating, I would like to share a few thoughts on the idea of including Traits in mainstream Squeak.
My first concern is performance. Since Traits carry no state, they will need to use accessor methods and message sends are clearly much slower than direct instance variable access.
My second concern is regarding how Traits should/could/would be used. All of the examples that I have seen so far, including the refactoring of the Collection hierarchy, seem to me to be "toy" problems. I would be interested to read about non-trivial cases where Traits would solve a significant real world problem.
It would seem to me that the kind of reuse that Traits enables is the kind that isn't apparent at design time, since individual classes of object are usually designed separately and often by completely different people. The common behavior and opportunity for code reuse might only become apparent after the initial design is implemented. It has been my experience that once something is implemented and working, unless there is a compelling reason to justify it, the code is never revisited. Management is generally focused on allocating resources to things that will yield either an opportunity for increased revenues, or reduced costs, and code "cleaning" is rarely viewed as either. If it ever gets done (code cleaning), it is usually silently piggybacked on required bug fixes by individual developers who feel it is important. Am I in the minority here, or is this how commercial software development works in general?
I would even suggest that single inheritance doesn't get used that much in practice. I see an awful lot of C++ where classes are root classes and implement a singleton object. I see poly-morphism used a lot, but not much inheritance. I also see STL used a lot to get common syntax container objects, that you get with the standard Smalltalk Collection hierarchy anyway.
So I guess I don't see what Traits gets us that we can't already do in Squeak with some judicious use of proxies, DNU and separate "behavior only" and "state only" classes using accessors for all instance variable access. Is the real benefit in having tools that formalize this and make it easy?
Is Traits implemented by just pointing a class's method dictionary entries to the approprate seemingly class-less compiled method defined in a Trait?
I guess I don't fundamentally oppose including Traits in mainstream Squeak, as long as it's still possible to ignore it and it doesn't cause a decrease in performance for commonly used objects (i.e. the Collection hierarchy). From what I've read in the whitepapers, it seems it should be safe and easy to ignore, so the only question I guess would be performance.
And lastly I will just say that as others (Andreas) have pointed out, maximising code reuse is not always desirable. I can see the argument in favor of code re-use to propagate common behavior without having to find every duplicated variation, but there is also the other side of the argument where a single bug in a common, highly reused method can cause a lot of things to break that might otherwise work properly.
-Dean
Hi dean
do you ***really*** think that the Collection hierarchy is a toy example.... I'm not sure that we are talking about the same.
On 30 août 05, at 02:28, Dean_Swan@Mitel.COM wrote:
While I find all this discussion fascinating, I would like to share a few thoughts on the idea of including Traits in mainstream Squeak.
My first concern is performance. Since Traits carry no state, they will need to use accessor methods and message sends are clearly much slower than direct instance variable access.
My second concern is regarding how Traits should/could/would be used. All of the examples that I have seen so far, including the refactoring of the Collection hierarchy, seem to me to be "toy" problems. I would be interested to read about non-trivial cases where Traits would solve a significant real world problem.
It would seem to me that the kind of reuse that Traits enables is the kind that isn't apparent at design time, since individual classes of object are usually designed separately and often by completely different people. The common behavior and opportunity for code reuse might only become apparent after the initial design is implemented. It has been my experience that once something is implemented and working, unless there is a compelling reason to justify it, the code is never revisited. Management is generally focused on allocating resources to things that will yield either an opportunity for increased revenues, or reduced costs, and code "cleaning" is rarely viewed as either. If it ever gets done (code cleaning), it is usually silently piggybacked on required bug fixes by individual developers who feel it is important. Am I in the minority here, or is this how commercial software development works in general?
Cleaning your design does not solve the problem that traits solve. Read the papers. Sometimes we are forced to duplicate code or to write a lot of delegate methods just because we cannot simply reuse method dictionary. This is what traits are about.
I would even suggest that single inheritance doesn't get used that much in practice. I see an awful lot of C++ where classes are root classes and implement a singleton object. I see poly-morphism used a lot, but not much inheritance. I also see STL used a lot to get common syntax container objects, that you get with the standard Smalltalk Collection hierarchy anyway.
So I guess I don't see what Traits gets us that we can't already do in Squeak with some judicious use of proxies, DNU and separate "behavior only" and "state only" classes using accessors for all instance variable access. Is the real benefit in having tools that formalize this and make it easy?
Is Traits implemented by just pointing a class's method dictionary entries to the approprate seemingly class-less compiled method defined in a Trait?
I guess I don't fundamentally oppose including Traits in mainstream Squeak, as long as it's still possible to ignore it and it doesn't cause a decrease in performance for commonly used objects (i.e. the Collection hierarchy). From what I've read in the whitepapers, it seems it should be safe and easy to ignore, so the only question I guess would be performance.
And lastly I will just say that as others (Andreas) have pointed out, maximising code reuse is not always desirable. I can see the argument in favor of code re-use to propagate common behavior without having to find every duplicated variation, but there is also the other side of the argument where a single bug in a common, highly reused method can cause a lot of things to break that might otherwise work properly.
-Dean
stéphane ducasse wrote:
Hi dean
do you ***really*** think that the Collection hierarchy is a toy example.... I'm not sure that we are talking about the same.
and
Cleaning your design does not solve the problem that traits solve. Read the papers. Sometimes we are forced to duplicate code or to write a lot of delegate methods just because we cannot simply reuse method dictionary. This is what traits are about.
There are ***really*** people who think something different than you. What about answering posts from them, especially if they try to think constructively about something that essentially you brought up, not with your laconic and annoying "read the papers". Otherwise people may learn and might have a laconic answer like "declined" for you in the future.
Here is what I think about the use of Traits in the Collection hierarchy:
- the Collection hierarchy works and there is no practical problem with its daily use. No problem is meant compared to other problems of Squeak. And if a problem shows up, it is usually of a philosophical nature like "Should a Set behave this or that way" and like all real philosophical problems it usually ignites a big discussion without any result. The dominance of philosophical problems with regard to Collections shows IMO that there is indeed no real problem with them.
- there is no understandability problem with them either. Again compared to other regions of Squeakland like e.g. the UI-library, the VM, or the application and its object oriented design you want to realize in Squeak. Of course you would have to think a bit to subclass let's say Set and find out how inheritance has been (mis)used to make it work, but that is a seldom and nonetheless minor problem. Generally everything within the Collection hierarchy is _very_ easy to understand.
- and there is no problem with the implementation of it, in the practical sense that a rare occurring bug is - once found - easily resolved.
All that makes Collections a really unconvincing example of what to gain from Traits. So for the Collection hierarchy it seems to boil down to a deal between space vs speed. You save space, but you lose speed.
- Such a deal alone should not be the reason for a language extension or change. Especially not for a really good and proven language like Smalltalk.
- Furthermore this deal is in the wrong direction! We desperately need speed, not space.
And that makes Collections a really bad candidate for illustrating the merits of Traits. For me this looks clearly like a counter-example.
Regards, Martin
Martin Wirblat wrote:
Here is what I think about the use of Traits in the Collection hierarchy:
[SNIP]
And that makes Collections a really bad candidate for illustrating the merits of Traits. For me this looks clearly like a counter-example.
Hi folks, first post here in ages and ages.
This is an interesting aspect. I read the traits papers a while back (so I may be rusty), and was fascinated by the work on the collection hierarchy. Let me say that most of my use of squeak does not revolve around programming in the sense of producing a finished piece of software, but rather as a tool to play with and hopefully understand data that crops up in the course of my work. I usually end up with dozens of object explorer windows, several workspaces, chunks of code everywhere, a real mess on screen.
If I feel the need to start creating new classes to represent some of this data, I'm usually just interested in implementing the bare bones to get it functioning. Sometimes I then realize that I'd like my newly created objects to be collection-like or stream-like. Subclassing straight off one of the collection classes isn't the right thing, as it brings with it instance variables to hold the collection's state - I already have the state, what I want is to easily add collection-like behaviour. Richard O'Keefe's enumeration adapter change set from a few years back can be a big help, but it's only a partion solution. In some cases I just end up duplicating as much of the collection protocol as I need.
So I was thinking that traits might help me here to 'flesh out' the collection protocol on top of some bare bones of state. I'll admit that I haven't looked very deeply at this, but have I misunderstood what traits are about?
Cheers, Nick
P.S. I realize that what's under discussion right now is including just the kernel of traits, not the actual collection refactoring, and I guess that there would be still be lots of work to do to get the collection traits into squeak.
This sounds like a good example of Traits making reuse drastically easier. Basically, you would start by adding a couple of collection-methods to the interface, by delegation or by copy&paste. Then you you notice, heck, I want all of that interface. So you add a Trait to Collection, move lots of methods from Collection to the Trait, until you're satisfied.
The you add the Trait to your new class. Oops, the requires browser should the class suddenly turned abstract, because Trait, in order to provide an implementation of #collect:, requires an implementation of #do:. You implement that using delegation to the collection you're holding anyway, and voila, you're done.
Very little code is duplicated, the extra complexity from delegation is kept to a minimum, and all is well.
Of course the requires list might be longer, and the traits might already exist and not exactly match what you need, and so forth, but that's the general idea.
Someone made a valid point here, which is that Traits have only been shown to be useful for refactorings, rather than up front design. <complete speculation> I think that Traits are better for refactoring than inheritance, because all the inheritance related refactorings (push/pull method/instance variable and so forth) play within the basic order and structure that the original inheritance design determined out. It seems to me that decisions made using Traits are much more reversible. Therefore in the presence of Traits, and little inheritance, I speculate that designs (in the form of code) will converge to minimal duplication and minimal complexity faster than they do when based on inheritance. </complete speculation>
Daniel
Nick Brown wrote:
Sometimes I then realize that I'd like my newly created objects to be collection-like or stream-like. Subclassing straight off one of the collection classes isn't the right thing, as it brings with it instance variables to hold the collection's state - I already have the state, what I want is to easily add collection-like behaviour. Richard O'Keefe's enumeration adapter change set from a few years back can be a big help, but it's only a partion solution. In some cases I just end up duplicating as much of the collection protocol as I need.
So I was thinking that traits might help me here to 'flesh out' the collection protocol on top of some bare bones of state. I'll admit that I haven't looked very deeply at this, but have I misunderstood what traits are about?
Cheers, Nick
P.S. I realize that what's under discussion right now is including just the kernel of traits, not the actual collection refactoring, and I guess that there would be still be lots of work to do to get the collection traits into squeak.
Quoting Nick Brown maillist@bredon-gill.demon.co.uk:
So I was thinking that traits might help me here to 'flesh out' the collection protocol on top of some bare bones of state. I'll admit that I haven't looked very deeply at this, but have I misunderstood what traits are about?
Yes. This is one of my favourite possible uses for traits. This is basically what Ruby lets you do with its Enumerable module (though it does it with mixins): http://www.rubycentral.com/book/ref_m_enumerable.html
I think it's awesome... you just have to implement #do: (in Ruby's case it's actually #each), add the right trait, and suddenly you have #collect:, #select:, #findAllSuchThat:, #withIndexDo:, etc, etc.
I want traits just so that I can have an Enumerable trait. :) Ruby also has a similar mixin module called Comparison which provides #<, #>, etc. as long as you provide #<=>, which returns -1, 0, or 1 as a basic ordering method. It seems pretty common to have a whole bunch of methods that can be provided based on a common implementation of a few "core" methods (these tend to be the #subclassResponsibility methods in my abstract classes). The problem is, of course, if you have an object that wants to be Comparable and Enumarable and have a bunch of other shared behaviours as well.
Julian
On 31 août 05, at 06:03, Julian Fitzell wrote:
Quoting Nick Brown maillist@bredon-gill.demon.co.uk:
So I was thinking that traits might help me here to 'flesh out' the collection protocol on top of some bare bones of state. I'll admit that I haven't looked very deeply at this, but have I misunderstood what traits are about?
Yes. This is one of my favourite possible uses for traits. This is basically what Ruby lets you do with its Enumerable module (though it does it with mixins): http://www.rubycentral.com/book/ref_m_enumerable.html
I think it's awesome... you just have to implement #do: (in Ruby's case it's actually #each), add the right trait, and suddenly you have #collect:, #select:, #findAllSuchThat:, #withIndexDo:, etc, etc.
EXACT!
I want traits just so that I can have an Enumerable trait. :) Ruby also has a similar mixin module called Comparison which provides #<, #>, etc. as long as you provide #<=>, which returns -1, 0, or 1 as a basic ordering method. It seems pretty common to have a whole bunch of methods that can be provided based on a common implementation of a few "core" methods (these tend to be the #subclassResponsibility methods in my abstract classes). The problem is, of course, if you have an object that wants to be Comparable and Enumarable and have a bunch of other shared behaviours as well.
I hope that we will make it :) Because this is one of the goal. Getting a better way of structuring our classes
Stef
Julian
-- julian@beta4.com Beta4 Productions (http://www.beta4.com)
Nick Brown wrote:
If I feel the need to start creating new classes to represent some of this data, I'm usually just interested in implementing the bare bones to get it functioning. Sometimes I then realize that I'd like my newly created objects to be collection-like or stream-like. Subclassing straight off one of the collection classes isn't the right thing, as it brings with it instance variables to hold the collection's state - I already have the state, what I want is to easily add collection-like behaviour. Richard O'Keefe's enumeration adapter change set from a few years back can be a big help, but it's only a partion solution. In some cases I just end up duplicating as much of the collection protocol as I need.
Interesting example. Here are some thoughts:
- It is probably a rare case to have something collection-like but being not a Collection i.e. having state in a way that direct subclassing of a Collection can't be used.
- In order to use Traits to make such an object behave like a Collection I guess that either
1) the Collection hierarchy must be heavily traitified out of the box or 2) one would have to change it that way at a particular spot or 3) one would have to duplicate some Collection methods to traits ;)
The first version looks to me like being really slow. Wouldn't it mean that almost or absolutely _no_ access of state is done directly in the standard classes other than in the accessors? That would surely be unacceptable slow.
And the second may not be the right general style of working other than for temporary experiments. I mean changing the Collection hierarchy implies checking every new Squeak version if the "private" Collections do still the same as the official ones.
Overall I have to restate here, what I tried to express some times now. One method of business is the analysis of weak points. Often it is much more rewarding to find and eliminate the weak points than to concentrate on building out strengthes. Squeak has weak points and one of it is the suboptimal speed. Working with it is often unpleasant UI-wise and of course there are applications that suffer from low speed.
But have you all ever thought how many programmers did not switch to Squeak, because they conducted benchmarks in their early evaluation of it, causing them to immediately drop the mouse like a hot potato?
So my feeling is really that speed is much more a weak point than any shortcomings of Smalltalk's single inheritance.
Regards, Martin
All,
Thanks to everyone who has replied. When I heard the use of traits in collections described first as a toy, then as a bad example, I had started to think I might have got the wrong end of the stick.
Martin Wirblat wrote:
Interesting example. Here are some thoughts:
- It is probably a rare case to have something collection-like but
being not a Collection i.e. having state in a way that direct subclassing of a Collection can't be used.
Agreed, although I do find it interesting to wonder how much of that is because it's awkward to do at the moment.
- In order to use Traits to make such an object behave like a
Collection I guess that either
- the Collection hierarchy must be heavily traitified out of the box
or 2) one would have to change it that way at a particular spot or 3) one would have to duplicate some Collection methods to traits ;)
I was assuming (1), in the sense that I'd like to think we could ultimately achieve that.
The first version looks to me like being really slow. Wouldn't it mean that almost or absolutely _no_ access of state is done directly in the standard classes other than in the accessors? That would surely be unacceptable slow.
[SNIP]
But have you all ever thought how many programmers did not switch to Squeak, because they conducted benchmarks in their early evaluation of it, causing them to immediately drop the mouse like a hot potato?
So my feeling is really that speed is much more a weak point than any shortcomings of Smalltalk's single inheritance.
Performance in Squeak is important, but is there a general performance problem with Squeak, or do newcomers get turned off because one of the first things they get exposed to is Morphic, which can in many ways feel slow? If I make a list of areas of Squeak where I've found a performance problem (other than Morphic), off the top of my head, I get: hashing in large collections, CrLfFileStream; that's about it.
I have no idea how much of a general performance impact there could be by forcing so much state access through accessors, nor how much of it could be regained by various means (I seem to recall somebody suggests that the accessors could be inlined, but can't find the message now). My feeling is that I would not worry very much about a performance problem caused by traits until I can see it.
Regards, Nick
Am 31.08.2005 um 18:47 schrieb Nick Brown:
Performance in Squeak is important, but is there a general performance problem with Squeak, or do newcomers get turned off because one of the first things they get exposed to is Morphic, which can in many ways feel slow? If I make a list of areas of Squeak where I've found a performance problem (other than Morphic), off the top of my head, I get: hashing in large collections, CrLfFileStream; that's about it.
Morphic is improving... the "10-Browser" benchmark is now over 1.4 times faster then in 3.8.
I have no idea how much of a general performance impact there could be by forcing so much state access through accessors, nor how much of it could be regained by various means (I seem to recall somebody suggests that the accessors could be inlined, but can't find the message now). My feeling is that I would not worry very much about a performance problem caused by traits until I can see it.
accessors are already a bit optimized: For the quickreturn methods the vm does not need to actually run the method, so no context is created. Activation thus is cheap, of course the cost of method lookup still remains. But if the performance really turns out to be problem, I am sure that it won't be too hard to implement just that performance relevant method in the classes, not the Trait.
And something that should be said again: Adding the possibility of using traits to squeak will *not* slow down any existing code at all.
Marcus
And something that should be said again: Adding the possibility of using traits to squeak will *not* slow down any existing code at all.
Thanks marcus. I think that I lost a part of my faith and energy in the process of talking about traits...
Traits are the most backward compatible, non-intrusive extension of Smalltalk to provide reuse orthogonal to inheritance we could imagine. The model of traits is the simplest one we could think of and (not as researcher since nathanael worked on extended versions that we will never push in Squeak since there are less backward compatible and more conceptually aggressive :).
Stef
Martin Wirblat wrote:
Nick Brown wrote:
If I feel the need to start creating new classes to represent some of this data, I'm usually just interested in implementing the bare bones to get it functioning. Sometimes I then realize that I'd like my newly created objects to be collection-like or stream-like. Subclassing straight off one of the collection classes isn't the right thing, as it brings with it instance variables to hold the collection's state - I already have the state, what I want is to easily add collection-like behaviour. Richard O'Keefe's enumeration adapter change set from a few years back can be a big help, but it's only a partion solution. In some cases I just end up duplicating as much of the collection protocol as I need.
Interesting example. Here are some thoughts:
- It is probably a rare case to have something collection-like but
being not a Collection i.e. having state in a way that direct subclassing of a Collection can't be used.
- In order to use Traits to make such an object behave like a
Collection I guess that either
- the Collection hierarchy must be heavily traitified out of the box
or 2) one would have to change it that way at a particular spot or 3) one would have to duplicate some Collection methods to traits ;)
The first version looks to me like being really slow. Wouldn't it mean that almost or absolutely _no_ access of state is done directly in the standard classes other than in the accessors? That would surely be unacceptable slow.
Why do you say this? There are ways and means to deal with this - if it is actually a significant problem. Time saved in development/maintenance may mitigate this, also.
And the second may not be the right general style of working other than for temporary experiments. I mean changing the Collection hierarchy implies checking every new Squeak version if the "private" Collections do still the same as the official ones.
Overall I have to restate here, what I tried to express some times now. One method of business is the analysis of weak points. Often it is much more rewarding to find and eliminate the weak points than to concentrate on building out strengthes. Squeak has weak points and one of it is the suboptimal speed. Working with it is often unpleasant UI-wise and of course there are applications that suffer from low speed.
But have you all ever thought how many programmers did not switch to Squeak, because they conducted benchmarks in their early evaluation of it, causing them to immediately drop the mouse like a hot potato?
So my feeling is really that speed is much more a weak point than any shortcomings of Smalltalk's single inheritance.
Possibly -- but I do not think Squeak was really ever meant to be a production environment -- would be nice to be able to throw a (preferences) switch to 'production' :-)
David
Hello David,
DPH> Why do you say this? There are ways and means to deal with this - if it DPH> is actually a significant problem. Time saved in DPH> development/maintenance may mitigate this, also.
Software being said to be sluggish is in a big distvantage. In road design where I plan to use squeak commercially there is a speed difference between 3.6 and 3.7 and not using accessors and pushing methods down the hierarchy makes a difference.
And though I like LookEnhancements I always notice the speed penalty.
I have big hopes for Exupery and Morphic cleanup/splitters, which seem to address the main speed handicaps.
As for traits, there are times when I feel I have to overburden some classes due to single inheritance and there definitely is code which isn't speed sensitive. So I appreciate this too and as people point out that you can use it but needn't to I think it's an improvement.
As always judicious choice is the key and the more tools you have at hand the better you are off.
DPH> Possibly -- but I do not think Squeak was really ever meant to be a DPH> production environment -- would be nice to be able to throw a DPH> (preferences) switch to 'production' :-)
At least seaside seems to aim at production and there seems to be a direction making Squeak usable for production. In the long run I think these attempts will give Squeak a better future.
Cheers,
Herbert mailto:herbertkoenig@gmx.net
Am 31.08.2005 um 20:41 schrieb Herbert König:
Hello David,
DPH> Why do you say this? There are ways and means to deal with this - if it DPH> is actually a significant problem. Time saved in DPH> development/maintenance may mitigate this, also.
Software being said to be sluggish is in a big distvantage. In road design where I plan to use squeak commercially there is a speed difference between 3.6 and 3.7 and not using accessors and pushing methods down the hierarchy makes a difference.
I don't think that the speed difference 3.6 vs. 3.7 due to using accessors.
I am sure that the morphic cleanup changes (isEmpty vs. direct testing and so on) had some influence, but the majority of the speed-loss is due tue the different window look, esp. the buttons with gradients, using true-type for window titles and stuff like that.
In 3.9, performance is now better than 3.7, but not yet as fast as 3.6.
Overall, I have the feeling that real improvments in performance will not come from micro-optimizing, but from using better algorithms and better concepts. And the first step of this is cleaning up.
And though I like LookEnhancements I always notice the speed penalty.
On my system, 3.9 with LookEnhancements is *faster* than the 3.9 without (3.9a6686 vs. 3.9a6684). Can you make a test? Would be interesting to see.
Marcus
Herbert
I can tell you that we are ****really**** concerned by speed and by making sure that people can commercially use Squeak! Really!!!!! This is why all the speed up changes in Morphic for 3.9 done by eddie and others (if they do not break anything) will be included once tested (by others :)).
So we do not want to trade beauty for speed. But we need to clean Squeak and make sure that changes can happen.
Stef
On 31 août 05, at 20:41, Herbert König wrote:
Hello David,
DPH> Why do you say this? There are ways and means to deal with this - if it DPH> is actually a significant problem. Time saved in DPH> development/maintenance may mitigate this, also.
Software being said to be sluggish is in a big distvantage. In road design where I plan to use squeak commercially there is a speed difference between 3.6 and 3.7 and not using accessors and pushing methods down the hierarchy makes a difference.
And though I like LookEnhancements I always notice the speed penalty.
I have big hopes for Exupery and Morphic cleanup/splitters, which seem to address the main speed handicaps.
As for traits, there are times when I feel I have to overburden some classes due to single inheritance and there definitely is code which isn't speed sensitive. So I appreciate this too and as people point out that you can use it but needn't to I think it's an improvement.
As always judicious choice is the key and the more tools you have at hand the better you are off.
DPH> Possibly -- but I do not think Squeak was really ever meant to be a DPH> production environment -- would be nice to be able to throw a DPH> (preferences) switch to 'production' :-)
At least seaside seems to aim at production and there seems to be a direction making Squeak usable for production. In the long run I think these attempts will give Squeak a better future.
Cheers,
Herbert mailto:herbertkoenig@gmx.net
O
Hi folks, first post here in ages and ages.
This is an interesting aspect. I read the traits papers a while back (so I may be rusty), and was fascinated by the work on the collection hierarchy. Let me say that most of my use of squeak does not revolve around programming in the sense of producing a finished piece of software, but rather as a tool to play with and hopefully understand data that crops up in the course of my work. I usually end up with dozens of object explorer windows, several workspaces, chunks of code everywhere, a real mess on screen.
If I feel the need to start creating new classes to represent some of this data, I'm usually just interested in implementing the bare bones to get it functioning. Sometimes I then realize that I'd like my newly created objects to be collection-like or stream-like. Subclassing straight off one of the collection classes isn't the right thing, as it brings with it instance variables to hold the collection's state - I already have the state, what I want is to easily add collection-like behaviour. Richard O'Keefe's enumeration adapter change set from a few years back can be a big help, but it's only a partion solution. In some cases I just end up duplicating as much of the collection protocol as I need.
So I was thinking that traits might help me here to 'flesh out' the collection protocol on top of some bare bones of state. I'll admit that I haven't looked very deeply at this, but have I misunderstood what traits are about?
It is. You could reuse a Trait-Enumerator traits all over the place without duplicating it.
Stef
Cheers, Nick
P.S. I realize that what's under discussion right now is including just the kernel of traits, not the actual collection refactoring, and I guess that there would be still be lots of work to do to get the collection traits into squeak.
On 8/30/05, Martin Wirblat sql.mawi@t-link.de wrote:
- the Collection hierarchy works and there is no practical problem with
its daily use.
Generally, this is true. But this hierarchy has some flaws which traits can improve. For example, by reducing duplicated code, maintenance is greatly simplified.
The dominance of philosophical problems with regard to Collections shows IMO that there is indeed no real problem with them.
The dominance of philosophical problems I find with regard to your reasoning shows that there is no real problem with your reasoning. :-)
Generally everything within the Collection hierarchy is _very_ easy to understand.
Yes; I believe that this is one key reason why it was chosen for the research paper. Because Collection relies upon very straightforward analogies to real-world objects, the code is quite simple, for the most part. One would not expect that such code would benefit much from traitwise refactoring, so the work done in the paper is all the more impressive.
- and there is no problem with the implementation of it, in the
practical sense that a rare occurring bug is - once found - easily resolved.
Yes and no. When code is duplicated, who ensures that a bug found in one place is corrected in every place?
All that makes Collections a really unconvincing example of what to gain from Traits. So for the Collection hierarchy it seems to boil down to a deal between space vs speed. You save space, but you lose speed.
What loss of speed are you talking about? Is it the overhead of using access methods rather than directly using instance variables, or is there something else?
It seems that the developers' hope here is that whatever speed is lost due to using traits will be offset by cleaner code available sooner, in every part of Squeak.
- Such a deal alone should not be the reason for a language extension or
change. Especially not for a really good and proven language like Smalltalk.
Ah, here I think we have the most important objection. It's not Smalltalk if it's got traits. You're right, there. This isn't a small change like adding color to BitBlt, or letting curly braces make an array. It's making the structure of the language different in a fundamental way.
Smalltalk is proven by years of use. Let us not take that fact to mean that it cannot be improved upon. Let us also not make changes without due care.
I am skeptical. I'm willing to be shown that traits are a good idea, and I'm willing to be shown that they aren't. I'm willing to take the chance because it's nothing we can't pull out of Squeak if we don't like it. (Or redo if we want it done differently, for that matter. I'm looking forward to the day when somebody makes traits that can directly access instance variables.)
At the moment, the main efficency that traits seem to provide is _human_ efficiency, in that a human being has to spend less time (we hope) reading, writing, debugging, and maintaining code. This is a Good Thing.
In the end, we may not want a heavily-used class hierarchy such as Collection to be implemented with traits. But we won't know for sure what traits are good for, until we've given traits a try.
--Tom Phoenix
I'm actually quite surprised by some of the discussion here. I'm not surprised people are skeptical; this is a good thing. But I'm a little surprised by the defense of inheritance and by things "being good enough because they work now". I mean, to me, Smalltalk has always been about ideals. Sure, things that work "well enough" should be at the bottom of everyone's list of things to break, but the flaws in the collection hierarchy are obvious--even if we've all managed to work around them (or not conern ourselves with them).
As for defending inheritance, I learned about OO programming back in 1990. A johnny-come-lately, I guess, since I'd been programming for ten years and (in lieu of any formal training or knowledge of internet) struggling to come up with systems of expressions in traditional languages that would give me just a fraction of what OOP did. And yet, within weeks of learning OO, the limitations of inheritance were apparent even to me. Yeah, you can almost always insert a feature at the highest point in the hierarchy you need it, but then you end up with big, fat objects stuffed with lots of unused code. It's inelegant, it's wasteful and--in the case of collections--it leads to code duplication. It also strikes me as a security hazard.
Is any of this seriously debated? I mean, traits notwithstanding, has anyone who has ever built a large hierarchy of objects NOT run into the above problems?
===Blake===
On 31 août 05, at 00:48, Blake wrote:
I'm actually quite surprised by some of the discussion here. I'm not surprised people are skeptical; this is a good thing. But I'm a little surprised by the defense of inheritance and by things "being good enough because they work now". I mean, to me, Smalltalk has always been about ideals. Sure, things that work "well enough" should be at the bottom of everyone's list of things to break, but the flaws in the collection hierarchy are obvious--even if we've all managed to work around them (or not conern ourselves with them).
thanks this is nice to see that people think that way.
As for defending inheritance, I learned about OO programming back in 1990. A johnny-come-lately, I guess, since I'd been programming for ten years and (in lieu of any formal training or knowledge of internet) struggling to come up with systems of expressions in traditional languages that would give me just a fraction of what OOP did. And yet, within weeks of learning OO, the limitations of inheritance were apparent even to me. Yeah, you can almost always insert a feature at the highest point in the hierarchy you need it, but then you end up with big, fat objects stuffed with lots of unused code. It's inelegant, it's wasteful and--in the case of collections--it leads to code duplication. It also strikes me as a security hazard.
Is any of this seriously debated? I mean, traits notwithstanding, has anyone who has ever built a large hierarchy of objects NOT run into the above problems?
===Blake===
If we can not try new ideas with Squeak, if we can not make the "revolution" happen with Squeak, where can we do it? I think traits is a great idea, and I believe Squeak is the best "environment" to try, not because of Squeak but because of the community involved in it. I applaud this decision and I'm anxious to see the results, no matter if they are good or bad, because we will learn something new.
Let's the revolution begin!!
Hernan.
stéphane ducasse wrote:
On 31 août 05, at 00:48, Blake wrote:
I'm actually quite surprised by some of the discussion here. I'm not surprised people are skeptical; this is a good thing. But I'm a little surprised by the defense of inheritance and by things "being good enough because they work now". I mean, to me, Smalltalk has always been about ideals. Sure, things that work "well enough" should be at the bottom of everyone's list of things to break, but the flaws in the collection hierarchy are obvious--even if we've all managed to work around them (or not conern ourselves with them).
thanks this is nice to see that people think that way.
As for defending inheritance, I learned about OO programming back in 1990. A johnny-come-lately, I guess, since I'd been programming for ten years and (in lieu of any formal training or knowledge of internet) struggling to come up with systems of expressions in traditional languages that would give me just a fraction of what OOP did. And yet, within weeks of learning OO, the limitations of inheritance were apparent even to me. Yeah, you can almost always insert a feature at the highest point in the hierarchy you need it, but then you end up with big, fat objects stuffed with lots of unused code. It's inelegant, it's wasteful and--in the case of collections--it leads to code duplication. It also strikes me as a security hazard.
Is any of this seriously debated? I mean, traits notwithstanding, has anyone who has ever built a large hierarchy of objects NOT run into the above problems?
===Blake===
Dean_Swan@Mitel.COM wrote:
My first concern is performance. Since Traits carry no state, they will need to use accessor methods and message sends are clearly much slower than direct instance variable access.
This is currently true. I wouldn't worry about this too much for the following reasons: 1. We have Bryce on our side! the overhead of self sent accessors shall soon be reduced to a painful, but fading memory (half kidding, don't think he ever promised that, but the essence is true) 2. 95% of code is not performance critical. Of the remaining 5%, I'd bet only a part spends significant time accessing variables. 3. We're not converting everything to use accessors now. 4. If we discover that creating a Trait *has* caused a performance regression due to this effect, we can override the method in the class, trading off code duplication for speed. However at that point, we're trading off a tiny bit of duplication for some real speed, as opposed to the current situation, where we can't use Traits anywhere, so in lots of places we have code duplication that's gaining us nothing 5. If we discover this happens often, there is nothing preventing us from optimizing Traits away automatically - always duplicate the method into the class, and inline accessors. Make the tools keep this CompiledMethod in sync with the original, and prevent us from editing it by mistake. At that point this performance concern should be quite dead.
My second concern is regarding how Traits should/could/would be used.
Nobody claims to really know this. We'll find out soon enough.
All of the examples that I have seen so far, including the refactoring of the Collection hierarchy, seem to me to be "toy" problems. I would be interested to read about non-trivial cases where Traits would solve a significant real world problem.
Try to add 10000 Behaviors (or some other object that does not override Object>>hash) into a Set. Try to do the same to an IdentitySet. This bug is know from 2005. One reason it is not solved, is because of the separate implementations. The code duplication in the collections hierarchy affects more than you know, every day. Yes, by hurting their performance, among things.
It has been my experience that once something is implemented and working, unless there is a compelling reason to justify it, the code is never revisited.
Mine is very different. Andrew and I have spent a few days work recently removing a class that was not necessary from Traits, just to simplify the implementation.
Management is generally focused on allocating resources to things that will yield either an opportunity for increased revenues, or reduced costs, and code "cleaning" is rarely viewed as either.
Then management is not up to speed on their software management techniques. Conceptual coherency, as a tool for controlling software development costs, is known at least for 25 years (see Mythical Man Month). I've read that introducing them to the concept of "Design debt" helps them understand the situation in their own terms.
If it ever gets done (code cleaning), it is usually silently piggybacked on required bug fixes by individual developers who feel it is important. Am I in the minority here, or is this how commercial software development works in general?
I think your case is quite common, but certainly not the only case. But even in this case, reducing the cost of refactoring allows the Code Cleaning Special Forces to be more effective when they do on a mission.
So I guess I don't see what Traits gets us that we can't already do in Squeak with some judicious use of proxies, DNU and separate "behavior only" and "state only" classes using accessors for all instance variable access. Is the real benefit in having tools that formalize this and make it easy?
Getting the same benefits with less run time and design complexity sounds good to me... having the tools support the language features is a very nice bonus.
Daniel
Daniel Vainsencher writes:
Dean_Swan@Mitel.COM wrote:
My first concern is performance. Since Traits carry no state, they will need to use accessor methods and message sends are clearly much slower than direct instance variable access.
This is currently true. I wouldn't worry about this too much for the following reasons:
- We have Bryce on our side! the overhead of self sent accessors shall
soon be reduced to a painful, but fading memory (half kidding, don't think he ever promised that, but the essence is true)
I'm planning on leaving self send specific optimisations for a while. It seems sensible to put send improvement efforts into places that will speed up all sends not just self sends. Besides, if I optimised self sends then I'd need to find another send benchmark that measured non-self sends. ;-)
Exupery is currently only twice as fast as the interpreter for sends. There is plenty of room for improvement both by improving the general case code and by adding inlining to really optimise frequent sends.
For compiled code, there's probably a space savings by inlining self sends. Many methods are very small and the compiled send code is large. For normal sends Exupery always needs to generate a full send sequence just in case the object isn't of the expected type.
If people feel like putting energy into making Squeak a lot faster then I've just started stress testing Exupery. Once the stress test stops crashing, some beta testers would help. The main problem is figuring out what the minimum amount needed to make Exupery practical is. After getting to that minimum there's plenty of exciting things still to do. The stress test involves compiling methods from every test class in the system. So writing tests of any part of the system would help too, especially parts that are slow and should be faster.
Bryce
Bryce Kampjes wrote:
If people feel like putting energy into making Squeak a lot faster then I've just started stress testing Exupery. Once the stress test stops crashing, some beta testers would help.
I'm extremely keen to help out on this front. Could you perhaps write a paragraph or two on the process of running the stress tests? I'd like to see it crashing, and to start to get the experience needed to fix it when it does.
Tony
Tony Garnock-Jones writes:
I'm extremely keen to help out on this front. Could you perhaps write a paragraph or two on the process of running the stress tests? I'd like to see it crashing, and to start to get the experience needed to fix it when it does.
First, Exupery only runs on Linux/x86 at the moment. If you're running that then follow the instructions on either building or installing Exupery from here.
http://minnow.cc.gatech.edu/squeak/3842
If you're planning on debugging the compiler then you'll want to build your own kernel. Debugging normally starts by analysing generated machine code in gdb. I'm not sure how well this would work without a local build.
Once you've got a local working version of Exupery then:
ExuperyProfiler profileAndCompile: [5 timesRepeat: [ExuperyBenchmarks new compilerBenchmark]]
Will profile the expression given and compile the 10 most used methods. This method doesn't do primitive inlining yet so don't expect your bytecode performance to beat VW. I'll add that as soon as the stress test passes.
ExuperyProfiler stressTest
Will run the stress test. In somewhere between 1 and 10 minutes it will crash. Some of the tests freeze, I think this is due to the progress dialog morph but haven't investigated. Hitting Alt-. then proceeding will get past this. The progress dialog morph lock-up is easy to detect, Squeak drops to 0% CPU without crashing.
Once it's crashed look in the Exupery.log file.
If you're lucky the log shows that 10 methods have been compiled since the code cache was initialised. Try compiling these methods manually and figure out which one's causing the crash. Then I normally write a test that reliably creates the crash.
If you're unlucky then the crash was caused by memory corruption. A stack backtrace will probably show that the GC was executing and there may be no compiled methods. Memory corruption crashes can be a big pain because the corruption may have happened a long time ago.
After getting a test that reproduces the bug then have a look at the crash. It's worthwhile to look at both the current active context (foo->activeContext on Linux) and the C stack trace.
A context looks like:
(gdb) x/20x foo->activeContext 0x434d5ccc: 0x1736e35f 0x434d5c70 0x000000b1 0x0000000b 0x434d5cdc: 0x40da1f8c 0x808b415b 0x434dcc50 0x00000015 0x434d5cec: 0x434dcee0 0x4035b004 0x4035b004 0x434dcc50 0x434d5cfc: 0x434dced8 0x00000003 0x000001ff 0x000001ff 0x434d5d0c: 0x0000010d 0x00002b3f 0x434a1ab4 0x434a1ab4
0x000000b1 is the byte code program counter 0x0000000b is the stack pointer
0x808b415b is Exupery's return address.
From this it's possible to figure out that it crashed while executing
a compiled context and trace back to where in the method and which basic block it last entered. Exupery only updates the context when it leaves a method so the stack pointer and Exupery basic block pointer will point to where it re-entered the method.
From here, options involve exploring with gdb break-points, adding
printf statements (self cCode: 'printf("Entering method\n")'. will add a printf in Slang), or adding calls to validation code (Interpreter methods in the "debug support" catagory).
Staring at the generated methods sometimes helps. But less so when debugging random bugs. Place a halt at then end of Exupery>>run then try opening inspectors on the instance variables holding compiled methods (most of them). I save all the stages results in instance variables to make it easier to debug. The inspector will open up a graphical view of the method, it's an animated springs and repulsion Connectors graph. The explorers are normal.
After figuring out why the machine code is crashing then we need to know where Exupery went wrong. This is a game of chasing the bug back through the different versions of the method until we find the first version with a fault.
Bryce
P.S. I'm currently working at London Bridge, that's reasonably close to you isn't it? If you're serious about debugging then some pairing may help.
Hi bryce
How would you see the integration of your work with the rest? Especially the VM maker guys.
Stef
My first concern is performance. Since Traits carry no state, they will need to use accessor methods and message sends are clearly much slower than direct instance variable access.
This is currently true. I wouldn't worry about this too much for the following reasons:
- We have Bryce on our side! the overhead of self sent accessors
shall soon be reduced to a painful, but fading memory (half kidding, don't think he ever promised that, but the essence is true)
I'm planning on leaving self send specific optimisations for a while. It seems sensible to put send improvement efforts into places that will speed up all sends not just self sends. Besides, if I optimised self sends then I'd need to find another send benchmark that measured non-self sends. ;-)
Exupery is currently only twice as fast as the interpreter for sends. There is plenty of room for improvement both by improving the general case code and by adding inlining to really optimise frequent sends.
For compiled code, there's probably a space savings by inlining self sends. Many methods are very small and the compiled send code is large. For normal sends Exupery always needs to generate a full send sequence just in case the object isn't of the expected type.
If people feel like putting energy into making Squeak a lot faster then I've just started stress testing Exupery. Once the stress test stops crashing, some beta testers would help. The main problem is figuring out what the minimum amount needed to make Exupery practical is. After getting to that minimum there's plenty of exciting things still to do. The stress test involves compiling methods from every test class in the system. So writing tests of any part of the system would help too, especially parts that are slow and should be faster.
Bryce
stéphane ducasse writes:
Hi bryce
How would you see the integration of your work with the rest? Especially the VM maker guys.
The first step will be making Exupery useful for a small group of people. Until then, the risk of needing more VM changes is too great. The standard VM's are not the place to experiment in. Once we're confident that the VM isn't going to need further changes to make Exupery useful then merging will make sense. I'm not there yet.
I've got a VMMaker package in SqueakSource, those changes will need to be merged in. Also I'll need to figure out a better division between the plugin and the VM modifications. But merging into the VM should really wait until Exupery is a little more stable. Exupery itself is just a single Monticello package, that could stay on SqueakMap.
Bryce
Hi bryce
can you explain to me what is the squeak part of Exupery? You have your own compiler? bytecode emitter? Did you base it on fullClosure/ RB AST?
Stef
On 2 sept. 05, at 21:56, Bryce Kampjes wrote:
stéphane ducasse writes:
Hi bryce
How would you see the integration of your work with the rest? Especially the VM maker guys.
The first step will be making Exupery useful for a small group of people. Until then, the risk of needing more VM changes is too great. The standard VM's are not the place to experiment in. Once we're confident that the VM isn't going to need further changes to make Exupery useful then merging will make sense. I'm not there yet.
I've got a VMMaker package in SqueakSource, those changes will need to be merged in. Also I'll need to figure out a better division between the plugin and the VM modifications. But merging into the VM should really wait until Exupery is a little more stable. Exupery itself is just a single Monticello package, that could stay on SqueakMap.
Bryce
stéphane ducasse writes:
Hi bryce
can you explain to me what is the squeak part of Exupery? You have your own compiler? bytecode emitter? Did you base it on fullClosure/ RB AST?
Stef
Hi Stef,
Exupery is a bytecode to native machine code compiler. The compiler itself is written entirely in Squeak. The compiler written in Squeak produces a ByteArray containing generated machine code which is passed to a primitive to load into a machine code cache. There are a few modifications to the VM so it will run native compiled methods and a plug-in which provides the machine code cache.
The goal is to make Squeak much faster, Exupery is only designed to compile hotspot methods, all other methods will be interpreted. It's designed to produce very high quality code at the cost of slow compile times. It's meant to compile in a background Squeak thread so there shouldn't be visible compilation pauses.
The Squeak part is everything except some run-time support code. The Squeak code reads a CompiledMethod then translates that into intermediate code which is optimised, converted into instructions, register allocated, and assembled producing the the ByteArray containing machine code.
So, so long as traits or the fullClosure compiler hasn't changed the semantics of any bytecodes they should still be compatable with Exupery. From memory the fullClosure compiler adds two bytecodes, Exupery will need to support them to be able to optimise full closures. But currently Exupery can not compile methods that create blocks this isn't yet an issue.
Bryce
On 3 sept. 05, at 18:08, Bryce Kampjes wrote:
stéphane ducasse writes:
Hi bryce
can you explain to me what is the squeak part of Exupery? You have your own compiler? bytecode emitter? Did you base it on fullClosure/ RB AST?
Stef
Hi Stef,
Exupery is a bytecode to native machine code compiler.
I knew that but I thought that the bytecode to native code was done in C.
The compiler itself is written entirely in Squeak. The compiler written in Squeak produces a ByteArray containing generated machine code which is passed to a primitive to load into a machine code cache. There are a few modifications to the VM so it will run native compiled methods and a plug-in which provides the machine code cache.
so this means that you launch the bytetonative for some methods in the cache at run-time.
The goal is to make Squeak much faster, Exupery is only designed to compile hotspot methods, all other methods will be interpreted. It's designed to produce very high quality code at the cost of slow compile times. It's meant to compile in a background Squeak thread so there shouldn't be visible compilation pauses.
Ok
The Squeak part is everything except some run-time support code. The Squeak code reads a CompiledMethod then translates that into intermediate code which is optimised, converted into instructions, register allocated, and assembled producing the the ByteArray containing machine code.
So, so long as traits or the fullClosure compiler hasn't changed the semantics of any bytecodes they should still be compatable with Exupery.
Normally no bytecode was touches :)
From memory the fullClosure compiler adds two bytecodes, Exupery will need to support them to be able to optimise full closures. But currently Exupery can not compile methods that create blocks this isn't yet an issue.
Ok I understand better. Do you use the IR representation of the FullClosure compiler?
Stef
Bryce
stéphane ducasse writes:
On 3 sept. 05, at 18:08, Bryce Kampjes wrote:
stéphane ducasse writes:
Hi bryce
can you explain to me what is the squeak part of Exupery? You have your own compiler? bytecode emitter? Did you base it on fullClosure/ RB AST?
Stef
Hi Stef,
Exupery is a bytecode to native machine code compiler.
I knew that but I thought that the bytecode to native code was done in C.
Smalltalk is much nicer to write compilers in. ;-)
The compiler itself is written entirely in Squeak. The compiler written in Squeak produces a ByteArray containing generated machine code which is passed to a primitive to load into a machine code cache. There are a few modifications to the VM so it will run native compiled methods and a plug-in which provides the machine code cache.
so this means that you launch the bytetonative for some methods in the cache at run-time.
Bytecode to native compilation is done by Smalltalk. Smalltalk decides when and what needs to be compiled. Once it's compiled the method is loaded into the code cache and registered. I've got a profiler that will compile the most frequently called methods, the short term plan is to run this in a background thread.
Exupery initialiseExupery. Exupery compileMethod: #add: class: OrderedCollection.
First initialises the code cache then compiles OrderedCollection>>add: and add's it to the native code cache. From then on all calls to OrderedCollection>>add: will use the native code method.
The goal is to make Squeak much faster, Exupery is only designed to compile hotspot methods, all other methods will be interpreted. It's designed to produce very high quality code at the cost of slow compile times. It's meant to compile in a background Squeak thread so there shouldn't be visible compilation pauses.
Ok
The Squeak part is everything except some run-time support code. The Squeak code reads a CompiledMethod then translates that into intermediate code which is optimised, converted into instructions, register allocated, and assembled producing the the ByteArray containing machine code.
So, so long as traits or the fullClosure compiler hasn't changed the semantics of any bytecodes they should still be compatable with Exupery.
Normally no bytecode was touches :)
From memory the fullClosure compiler adds two bytecodes, Exupery will need to support them to be able to optimise full closures. But currently Exupery can not compile methods that create blocks this isn't yet an issue.
Ok I understand better. Do you use the IR representation of the FullClosure compiler?
I use several different IRs. They are all created especially for Exupery. The first includes operations such as boxing and deboxing booleans. This is so I can remove unnessisary boolean conversion code. Then I use an IR that is a little lower-level than machine code. From this I form instructions by using simple pattern matching code.
Bryce
Bryce Kampjes wrote:
stéphane ducasse writes:
On 3 sept. 05, at 18:08, Bryce Kampjes wrote:
stéphane ducasse writes:
Hi bryce
can you explain to me what is the squeak part of Exupery? You have your own compiler? bytecode emitter? Did you base it on fullClosure/ RB AST?
Stef
Hi Stef,
Exupery is a bytecode to native machine code compiler.
I knew that but I thought that the bytecode to native code was done in C.
Smalltalk is much nicer to write compilers in. ;-)
The compiler itself is written entirely in Squeak. The compiler written in Squeak produces a ByteArray containing generated machine code which is passed to a primitive to load into a machine code cache. There are a few modifications to the VM so it will run native compiled methods and a plug-in which provides the machine code cache.
so this means that you launch the bytetonative for some methods in the cache at run-time.
Bytecode to native compilation is done by Smalltalk. Smalltalk decides when and what needs to be compiled. Once it's compiled the method is loaded into the code cache and registered. I've got a profiler that will compile the most frequently called methods, the short term plan is to run this in a background thread.
Exupery initialiseExupery. ExuperycompileMethod: #add: class: OrderedCollection.
First initialises the code cache then compiles OrderedCollection>>add: and add's it to the native code cache. From then on all calls to OrderedCollection>>add: will use the native code method.
The goal is to make Squeak much faster, Exupery is only designed to compile hotspot methods, all other methods will be interpreted. It's designed to produce very high quality code at the cost of slow compile times. It's meant to compile in a background Squeak thread so there shouldn't be visible compilation pauses.
Ok
The Squeak part is everything except some run-time support code. The Squeak code reads a CompiledMethod then translates that into intermediate code which is optimised, converted into instructions, register allocated, and assembled producing the the ByteArray containing machine code.
So, so long as traits or the fullClosure compiler hasn't changed the semantics of any bytecodes they should still be compatable with Exupery.
Normally no bytecode was touches :)
From memory the fullClosure compiler adds two bytecodes, Exupery will need to support them to be able to optimise full closures. But currently Exupery can not compile methods that create blocks this isn't yet an issue.
Ok I understand better. Do you use the IR representation of the FullClosure compiler?
I use several different IRs. They are all created especially for Exupery. The first includes operations such as boxing and deboxing booleans. This is so I can remove unnessisary boolean conversion code. Then I use an IR that is a little lower-level than machine code. From this I form instructions by using simple pattern matching code.
Bryce
Are the IRs described anywhere (url) other than in the code?
Also, would other ports just use a different pattern-matcher in that last step? (i.e. is the low-level IR general enough to allow other architectures of machines?)
David
David P Harris writes:
Are the IRs described anywhere (url) other than in the code?
Also, would other ports just use a different pattern-matcher in that last step? (i.e. is the low-level IR general enough to allow other architectures of machines?)
The IR's are only described in code. They're still very simple tree based structures. There's more to a port than just writing a new instruction scheduler (the matcher), there would also need to be a new assembler. But the IR should be general enough to support any modern architecture, there's probably a few tweaks needed though.
Bryce
I discussed with marcus at his birthday party about exupery and this is really cool that this is nearly all in Smalltalk. Keep doing this work.
Stef
On 3 sept. 05, at 20:17, Bryce Kampjes wrote:
stéphane ducasse writes:
On 3 sept. 05, at 18:08, Bryce Kampjes wrote:
stéphane ducasse writes:
Hi bryce
can you explain to me what is the squeak part of Exupery? You have your own compiler? bytecode emitter? Did you base it on fullClosure/ RB AST?
Stef
Hi Stef,
Exupery is a bytecode to native machine code compiler.
I knew that but I thought that the bytecode to native code was done in C.
Smalltalk is much nicer to write compilers in. ;-)
The compiler itself is written entirely in Squeak. The compiler written in Squeak produces a ByteArray containing generated machine code which is passed to a primitive to load into a machine code cache. There are a few modifications to the VM so it will run native compiled methods and a plug-in which provides the machine code cache.
so this means that you launch the bytetonative for some methods in the cache at run-time.
Bytecode to native compilation is done by Smalltalk. Smalltalk decides when and what needs to be compiled. Once it's compiled the method is loaded into the code cache and registered. I've got a profiler that will compile the most frequently called methods, the short term plan is to run this in a background thread.
Exupery initialiseExupery. Exupery compileMethod: #add: class: OrderedCollection.First initialises the code cache then compiles OrderedCollection>>add: and add's it to the native code cache. From then on all calls to OrderedCollection>>add: will use the native code method.
The goal is to make Squeak much faster, Exupery is only designed to compile hotspot methods, all other methods will be interpreted. It's designed to produce very high quality code at the cost of slow compile times. It's meant to compile in a background Squeak thread so there shouldn't be visible compilation pauses.
Ok
The Squeak part is everything except some run-time support code. The Squeak code reads a CompiledMethod then translates that into intermediate code which is optimised, converted into instructions, register allocated, and assembled producing the the ByteArray containing machine code.
So, so long as traits or the fullClosure compiler hasn't changed the semantics of any bytecodes they should still be compatable with Exupery.
Normally no bytecode was touches :)
From memory the fullClosure compiler adds two bytecodes, Exupery will need to support them to be able to optimise full closures. But currently Exupery can not compile methods that create blocks this isn't yet an issue.
Ok I understand better. Do you use the IR representation of the FullClosure compiler?
I use several different IRs. They are all created especially for Exupery. The first includes operations such as boxing and deboxing booleans. This is so I can remove unnessisary boolean conversion code. Then I use an IR that is a little lower-level than machine code. From this I form instructions by using simple pattern matching code.
Bryce
stéphane ducasse writes:
I discussed with marcus at his birthday party about exupery and this is really cool that this is nearly all in Smalltalk. Keep doing this work.
Thanks,
Happy birthday Marcus.
Having proper closures will be really cool too.
Bryce
On 3-Sep-05, at 9:08 AM, Bryce Kampjes wrote:
stéphane ducasse writes:
Hi bryce
can you explain to me what is the squeak part of Exupery? You have your own compiler? bytecode emitter? Did you base it on fullClosure/ RB AST?
Stef
Hi Stef,
Exupery is a bytecode to native machine code compiler. The compiler itself is written entirely in Squeak. The compiler written in Squeak produces a ByteArray containing generated machine code which is passed to a primitive to load into a machine code cache. There are a few modifications to the VM so it will run native compiled methods and a plug-in which provides the machine code cache.
Where would I be able to find the most up-to-date list of Squeak bytecodes?
Jason
On 9/3/05, j thej@shaw.ca wrote:
Where would I be able to find the most up-to-date list of Squeak bytecodes?
I don't know whether it's "the most up-to-date", but one list is found in the source for Interpreter class >> initializeBytecodeTable. Does that give you what you need? Hope this helps!
--Tom Phoenix
On 3-Sep-05, at 2:41 PM, Tom Phoenix wrote:
On 9/3/05, j thej@shaw.ca wrote:
Where would I be able to find the most up-to-date list of Squeak bytecodes?
I don't know whether it's "the most up-to-date", but one list is found in the source for Interpreter class >> initializeBytecodeTable. Does that give you what you need? Hope this helps!
--Tom Phoenix
I searched for the class "Interpreter" but only found results in teh "Multilingual-TextConversion" category.
Where am I suppose to be looking?
jason
j writes:
I searched for the class "Interpreter" but only found results in teh "Multilingual-TextConversion" category.
Where am I suppose to be looking?
Install VMMaker from SqueakMap.
The Interpreter source is managed as a separate project outside the base image.
Bryce
I was thinking about the sokoban game in squeak. While it's adictive as hell, the version on Squeakmap is slow in a few places and has a few bugs. I had fixed these problems but later lost the image...
Squeakmap is generally wonderful, however, if a maintainer goes awol, the bits rot.
Then I considered the Wiki. -- someone publishes something and then other people edit it.
Wouldn't it be great if there were a changeset wiki where people can fork projects, synchronize forks, and comment on the varrious versions.
The upside is that for something like the VMMaker, someone like me can publish a private version of it. I could start working with an older version then as the trunk stabalizes, I can use the software to synchronize my fork with whatever noteworthy changes were made to the trunk...
I'll look into this as time permits.
Alan Grimes wrote:
The upside is that for something like the VMMaker, someone like me can publish a private version of it. I could start working with an older version then as the trunk stabalizes, I can use the software to synchronize my fork with whatever noteworthy changes were made to the trunk...
You can do this today. It's called Monticello and the repository is called Squeaksource[.com]. At Impara, for example (which is running a Squeaksource server), we have private versions for many of the main Squeak packages that allow us to update them as we see fit.
Cheers, - Andreas
Have a look at the description of Monticello and SqueakSource. We used that since at least one year and half.
http://www.iam.unibe.ch/~scg/Teaching/Smalltalk/Exercises- Smalltalk0405.pdf
Stef
On 4 sept. 05, at 03:52, Alan Grimes wrote:
I was thinking about the sokoban game in squeak. While it's adictive as hell, the version on Squeakmap is slow in a few places and has a few bugs. I had fixed these problems but later lost the image...
Squeakmap is generally wonderful, however, if a maintainer goes awol, the bits rot.
Then I considered the Wiki. -- someone publishes something and then other people edit it.
Wouldn't it be great if there were a changeset wiki where people can fork projects, synchronize forks, and comment on the varrious versions.
The upside is that for something like the VMMaker, someone like me can publish a private version of it. I could start working with an older version then as the trunk stabalizes, I can use the software to synchronize my fork with whatever noteworthy changes were made to the trunk...
I'll look into this as time permits.
-- Friends don't let friends use GCC 3.4.4 GCC 3.3.6 produces code that's twice as fast on x86!
squeak-dev@lists.squeakfoundation.org
-
Alan Grimes -
Andreas Raab -
Blake -
Bryce Kampjes -
Daniel Vainsencher -
David P Harris -
Dean_Swan＠Mitel.COM -
Herbert König -
Hernan Wilkinson -
j -
Julian Fitzell -
Marcus Denker -
Martin Wirblat -
Nick Brown -
stéphane ducasse -
Tom Phoenix -
Tony Garnock-Jones