I'm looking for ways to speed up my application, which has gotten intolerably slow. It spends a lot of its time doing operations on DateAndTime objects, which are basically Brent Pinkney's implementation of the ANSI standard for squeak. However, I see he has made some changes since the version I got. See http://minnow.cc.gatech.edu/squeak/1871
I notice that this class doesn't seem to have made it into the mainline image (3.6), though there is a TimeStamp class there. Aside from the fact that it has a different vocabulary, it doesn't have any associated concept of durations (e.g., a week), which I also need. There is at least one obvious functional difference: DateAndTime knows its offset from UTC, while TimeStamp doesn't. Not a must have feature for me.
(However, I see some Mar 2004 messages with fixes, that sort of give the impression it is in the main update stream...?)
My first question is if anyone could fill me in on the history of Date/Times and Durations in the image. I recall a couple of years ago there was some discussion of this, but I don't know how it turned out. In particular, there are issues of functionality, ANSI compatibility, and, as I've just discovered, speed. (DateAndTime generally converts stuff to a large integer and then operates; TimeStamp does not, at least in the < operator, which is eating a lot of my time).
I also recall there was another change set being bandied about that introduced ANSI compatibility into the image. That had a different Date/Time implementation. Details at http://minnow.cc.gatech.edu/squeak/2384
Second, I see some discussion of LargeInteger plugins. That might make my operations go much faster. Are those builtin by default? I don't see anything in /usr/lib/squeak, even for the 3.6 Debianized vm that I got, I think, from Ned Konz. Would such a plugin help? How do I get or build it? How do I know if I'm already using it?
To give you a sense of why I care, here are some timings in the profiler as I add a single task to my tasklist: The first column is time in seconds (not ms) 43 LargeNegativeInteger>>quo: 21 SmallInteger>>* 18 LargePositiveInteger>>asFloat 14 LargePositiveInteger>>+ That's a total of 68% of total execution time right there.
Overall, the operations spends 64% of its time in the main morphic loop, within which 19% operations on DateAndTime or Durations 36% evaluating a sort block, within which 35% (i.e., almost all) operations on DateAndTime Obviously, waiting over a minute to add a task is far from user friendly! (Athlon 800Mhz).
There is also the possibility of making it do fewer of these operations to begin with, but if there's a quick way to get a speed up that would be great.
On Monday 28 June 2004 6:10 pm, Ross Boylan wrote:
Second, I see some discussion of LargeInteger plugins. That might make my operations go much faster. Are those builtin by default? I don't see anything in /usr/lib/squeak, even for the 3.6 Debianized vm that I got, I think, from Ned Konz. Would such a plugin help? How do I get or build it? How do I know if I'm already using it?
do print-it's on:
Smalltalk listBuiltinModules #() Smalltalk listLoadedModules #('SoundPlugin 26 March 2004 (e)' 'ZipPlugin 26 March 2004 (e)' 'SocketPlugin 26 March 2004 (e)' 'LargeIntegers v1.2 26 March 2004 (e)' 'B2DPlugin 26 March 2004 (e)' 'FT2Plugin 26 March 2004 (e)' 'BitBltPlugin 26 March 2004 (e)' 'SecurityPlugin 26 March 2004 (e)' 'FilePlugin 26 March 2004 (e)' 'MiscPrimitivePlugin 26 March 2004 (e)')
On Mon, Jun 28, 2004 at 07:00:20PM -0700, Ned Konz wrote:
On Monday 28 June 2004 6:10 pm, Ross Boylan wrote:
Second, I see some discussion of LargeInteger plugins. That might make my operations go much faster. Are those builtin by default? I don't see anything in /usr/lib/squeak, even for the 3.6 Debianized vm that I got, I think, from Ned Konz. Would such a plugin help? How do I get or build it? How do I know if I'm already using it?
do print-it's on:
Smalltalk listBuiltinModules #() Smalltalk listLoadedModules #('SoundPlugin 26 March 2004 (e)' 'ZipPlugin 26 March 2004 (e)' 'SocketPlugin 26 March 2004 (e)' 'LargeIntegers v1.2 26 March 2004 (e)' 'B2DPlugin 26 March 2004 (e)' 'FT2Plugin 26 March 2004 (e)' 'BitBltPlugin 26 March 2004 (e)' 'SecurityPlugin 26 March 2004 (e)' 'FilePlugin 26 March 2004 (e)' 'MiscPrimitivePlugin 26 March 2004 (e)')
LargeIntegers is listed as a built in module, but not a loaded module. From the comments on the methods, this appears to mean the plugin is not active.
Unfortunately, after hunting around the image and the swiki, I can not find how to activate the plugin/module in squeak. There's a lot of info on how to make them, but I don't see much on how to use them! I'd appreciate any assistance.
For reference:
Smalltalk listBuiltinModules #('ADPCMCodecPlugin 26 January 2004 (i)' 'AsynchFilePlugin 26 January 2004 (i)' 'BMPReadWriterPlugin 26 January 2004 (i)' 'B2DPlugin 26 January 2004 (i)' 'BitBltPlugin 26 January 2004 (i)' 'DSAPrims 26 January 2004 (i)' 'ZipPlugin 26 January 2004 (i)' 'DropPlugin 26 January 2004 (i)' 'SqueakFFIPrims 26 January 2004 (i)' 'FFTPlugin 26 January 2004 (i)' 'FileCopyPlugin 26 January 2004 (i)' 'FilePlugin 26 January 2004 (i)' 'FloatArrayPlugin 26 January 2004 (i)' 'GeniePlugin v2.0 26 January 2004 (i)' 'JPEGReadWriter2Plugin 26 January 2004 (i)' 'JPEGReaderPlugin 26 January 2004 (i)' 'JoystickTabletPlugin 26 January 2004 (i)' 'Klatt 26 January 2004 (i)' 'LargeIntegers v1.2 26 January 2004 (i)' 'MIDIPlugin 26 January 2004 (i)' 'Matrix2x3Plugin 26 January 2004 (i)' 'MiscPrimitivePlugin 26 January 2004 (i)' 'Mpeg3Plugin 26 January 2004 (i)' 'RePlugin 26 January 2004 (i)' 'SerialPlugin 26 January 2004 (i)' 'SocketPlugin 26 January 2004 (i)' 'SoundCodecPrims 26 January 2004 (i)' 'SoundGenerationPlugin 26 January 2004 (i)' 'SoundPlugin 26 January 2004 (i)' 'StarSqueakPlugin 26 January 2004 (i)' 'SurfacePlugin Feb 10 2004 (i)')
Smalltalk listLoadedModules #('FloatArrayPlugin 26 January 2004 (i)' 'B2DPlugin 26 January 2004 (i)' 'LargeIntegers v1.2 26 January 2004 (i)' 'BitBltPlugin 26 January 2004 (i)' 'FilePlugin 26 January 2004 (i)' 'MiscPrimitivePlugin 26 January 2004 (i)')
On Monday 28 June 2004 9:51 pm, Ross Boylan wrote:
LargeIntegers is listed as a built in module, but not a loaded module. From the comments on the methods, this appears to mean the plugin is not active.
Looks like it's there to me:
Smalltalk listLoadedModules #( 'FloatArrayPlugin 26 January 2004 (i)' 'B2DPlugin 26 January 2004 (i)' 'LargeIntegers v1.2 26 January 2004 (i)' 'BitBltPlugin 26 January 2004 (i)' 'FilePlugin 26 January 2004 (i)' 'MiscPrimitivePlugin 26 January 2004 (i)')
On Mon, Jun 28, 2004 at 10:17:20PM -0700, Ned Konz wrote:
On Monday 28 June 2004 9:51 pm, Ross Boylan wrote:
LargeIntegers is listed as a built in module, but not a loaded module. From the comments on the methods, this appears to mean the plugin is not active.
Looks like it's there to me:
You're right. For my next trick, I'll learn to read.
So does this mean that the crummy performance I've been getting is with the assistance of the plugin?
Obviously, I need to look elsewhere for redemption in that case.
Out of curiosity, since the list of BuiltinModules was much larger than the list of LoadedModules, how do I activate the other modules/plugins?
For that matter, if I unload the LargeInteger module (I was able to find that command), how can I reload it? Other than restarting the vm, that is... I suppose I also don't know if Linux is one of the systems that supports unloading modules. A final issue is whether one can unload a builtin module at all.
On Monday 28 June 2004 10:32 pm, Ross Boylan wrote:
On Mon, Jun 28, 2004 at 10:17:20PM -0700, Ned Konz wrote:
On Monday 28 June 2004 9:51 pm, Ross Boylan wrote:
LargeIntegers is listed as a built in module, but not a loaded module. From the comments on the methods, this appears to mean the plugin is not active.
Looks like it's there to me:
You're right. For my next trick, I'll learn to read.
So does this mean that the crummy performance I've been getting is with the assistance of the plugin?
I don't know. Maybe.
If you can send a profile log to the list, maybe someone could suggest something.
Look at suspect LargeInteger methods and see if they call the primitives.
You can comment out the primitive calls in specific methods if you want to see how much slower it is *without* the plugin.
But usually it's not the fault of the plugin in cases like this. It sounds to me like maybe there's too much garbage collection going on, or too many unnecessary conversions, or something.
For instance, if you're using SortedCollection for big collections of DateTime instances that are being changed a lot, you should probably be using some other kind of collection that's more efficient.
Obviously, I need to look elsewhere for redemption in that case.
Out of curiosity, since the list of BuiltinModules was much larger than the list of LoadedModules, how do I activate the other modules/plugins?
You use them. When a method in one of them is needed, it's loaded.
For that matter, if I unload the LargeInteger module (I was able to find that command), how can I reload it? Other than restarting the vm, that is... I suppose I also don't know if Linux is one of the systems that supports unloading modules. A final issue is whether one can unload a builtin module at all.
I don't know how you would.
On Mon, Jun 28, 2004 at 11:40:35PM -0700, Ned Konz wrote:
On Monday 28 June 2004 10:32 pm, Ross Boylan wrote:
So does this mean that the crummy performance I've been getting is with the assistance of the plugin?
I don't know. Maybe.
If you can send a profile log to the list, maybe someone could suggest something.
The message tallies I have don't seem to have good options for export. Is there a good way to do that?
For reasons given below, the absence of the profile will probably not be a great loss.
Look at suspect LargeInteger methods and see if they call the primitives.
Good point. On inspection, none of them do. For example, 30% if the total time is in LargeNegativeInteger(Integer)>>quo:. The (Integer) there is, of course, another clue that the LargeInteger plugin isn't involved. :)
Somewhat surprisingly, SmallInteger(Integer)>>+ is also using a lot of time.
You can comment out the primitive calls in specific methods if you want to see how much slower it is *without* the plugin.
But usually it's not the fault of the plugin in cases like this. It sounds to me like maybe there's too much garbage collection going on, or too many unnecessary conversions, or something.
That raise an interesting issue: how is garbage collection reflected in the profiles?
For instance, if you're using SortedCollection for big collections of DateTime instances that are being changed a lot, you should probably be using some other kind of collection that's more efficient.
Probably the right thing for me to do in this case is look higher up the chain. Specifically, the offending operation is inserting something into an already well-populated GUI. For simplicity, I just blew everything away and recreated it. It's time to look into something a bit more refined.
I do think it would also be good to think about working on the performance of DateAndTime, as Avi suggested--especially if it's being used by ordinary Date's or Time's. For example, a lot of my time is going into simple comparison operations on DateAndTimes. It seems to me that's an operation that should be pretty fast. But to do it, asUTC gets called (necessitated in part by the fact that DateAndTime's represent times in a specific timezone), and that's an expensive operation.
On Jun 28, 2004, at 10:32 PM, Ross Boylan wrote:
So does this mean that the crummy performance I've been getting is with the assistance of the plugin?
I would think so. The date/time classes took a real hit in both stability and performance in 3.7. I think the major bugs have been fixed, but I'm still seeing anywhere from a 4x to 8x slowdown for common operations. It's not just the new DateAndTime class that's affected, but the existing classes like Date as well, because they have all been rewritten to use DateAndTime. Compare the results of this line in 3.6 and 3.7:
MessageTally spyOn: [10000 timesRepeat: [Date today < Date today]]
At this point in the release cycle, I pretty much have to grit my teeth and bear it, but I think we should reevaluate the design of this code for 3.8. The Duration classes are very nice, but the choice to use "nanosecond precision", and thus be constantly dealing with LargeIntegers, was IMO a poor one.
Avi
Ross Boylan wrote:
...
For that matter, if I unload the LargeInteger module (I was able to find that command), how can I reload it? Other than restarting the vm, that is...
I suppose I also don't know if Linux is one of the systems that supports unloading modules.
Linux supports this. Note: Smalltalk unloadModule: 'LargeIntegers' would be the correct call (just the string visible in the prim calls is the arg) in this case, but...(see below).
A final issue is whether one can unload a builtin module at all.
For me this works. But this doesn't help, since it is quickly loaded again, if somewhere in the system LargeIntegers arithmetics will be used, and there are many places (e.g. opening a browser in Morphic is sufficient for me to load it again)...
For a better - and platform independent - control of plugin calls (switching them on/off) look for package PrimCallController at SqueakMap. I have made it just for testing plugins (and especially the LargeIntegers plugin).
BTW: I don't think that the LargeIntegers plugin is the bottleneck in your case.
Greetings Stephan
I suppose I also don't know if Linux is one of the systems that supports unloading modules.
Linux supports this. Note: Smalltalk unloadModule: 'LargeIntegers'
This is expected to work on all currently supported platforms. I've not yet come across any that cannot handle it.
A final issue is whether one can unload a builtin module at all.
Same thing; except that it merely 'unlinks' the plugin from the internal list of active plugins. As soon as you invoke a method that refers to it, the plugin will be reloaded. NOTE however that (unless anyone has removed this) the plugin lookup starts by looking for an EXTERNAL plugin and thus any found external plugin will override the internal one. This for example means that it is trivial to override the facilities of the SecurityPlugin merely by implementing one that does the appropriate null actions.
tim
On Monday, June 28, 2004, at 09:10 PM, Ross Boylan wrote:
I'm looking for ways to speed up my application, which has gotten intolerably slow. It spends a lot of its time doing operations on DateAndTime objects, which are basically Brent Pinkney's implementation of the ANSI standard for squeak. However, I see he has made some changes since the version I got. See http://minnow.cc.gatech.edu/squeak/1871
I notice that this class doesn't seem to have made it into the mainline image (3.6), though there is a TimeStamp class there. Aside from the fact that it has a different vocabulary, it doesn't have any associated concept of durations (e.g., a week), which I also need. There is at least one obvious functional difference: DateAndTime knows its offset from UTC, while TimeStamp doesn't. Not a must have feature for me.
(However, I see some Mar 2004 messages with fixes, that sort of give the impression it is in the main update stream...?)
Yes, the ANSI DateAndTime (Chronology) changes are in the current update stream (3.7beta). The current stable release (3.6) on squeak.org was released last fall, so it's not going to contain anything from this year. 3.7 final should be coming out within the next month or so.
- Doug
My first question is if anyone could fill me in on the history of Date/Times and Durations in the image. I recall a couple of years ago there was some discussion of this, but I don't know how it turned out. In particular, there are issues of functionality, ANSI compatibility, and, as I've just discovered, speed. (DateAndTime generally converts stuff to a large integer and then operates; TimeStamp does not, at least in the < operator, which is eating a lot of my time).
I also recall there was another change set being bandied about that introduced ANSI compatibility into the image. That had a different Date/Time implementation. Details at http://minnow.cc.gatech.edu/squeak/2384
Second, I see some discussion of LargeInteger plugins. That might make my operations go much faster. Are those builtin by default? I don't see anything in /usr/lib/squeak, even for the 3.6 Debianized vm that I got, I think, from Ned Konz. Would such a plugin help? How do I get or build it? How do I know if I'm already using it?
To give you a sense of why I care, here are some timings in the profiler as I add a single task to my tasklist: The first column is time in seconds (not ms) 43 LargeNegativeInteger>>quo: 21 SmallInteger>>* 18 LargePositiveInteger>>asFloat 14 LargePositiveInteger>>+ That's a total of 68% of total execution time right there.
Overall, the operations spends 64% of its time in the main morphic loop, within which 19% operations on DateAndTime or Durations 36% evaluating a sort block, within which 35% (i.e., almost all) operations on DateAndTime Obviously, waiting over a minute to add a task is far from user friendly! (Athlon 800Mhz).
There is also the possibility of making it do fewer of these operations to begin with, but if there's a quick way to get a speed up that would be great.
On Mon, Jun 28, 2004 at 11:11:08PM -0400, Doug Way wrote:
On Monday, June 28, 2004, at 09:10 PM, Ross Boylan wrote:
I'm looking for ways to speed up my application, which has gotten intolerably slow. It spends a lot of its time doing operations on DateAndTime objects, which are basically Brent Pinkney's implementation of the ANSI standard for squeak. However, I see he has made some changes since the version I got. See http://minnow.cc.gatech.edu/squeak/1871
I notice that this class doesn't seem to have made it into the mainline image (3.6), though there is a TimeStamp class there. Aside from the fact that it has a different vocabulary, it doesn't have any associated concept of durations (e.g., a week), which I also need. There is at least one obvious functional difference: DateAndTime knows its offset from UTC, while TimeStamp doesn't. Not a must have feature for me.
(However, I see some Mar 2004 messages with fixes, that sort of give the impression it is in the main update stream...?)
Yes, the ANSI DateAndTime (Chronology) changes are in the current update stream (3.7beta). The current stable release (3.6) on squeak.org was released last fall, so it's not going to contain anything from this year. 3.7 final should be coming out within the next month or so.
- Doug
Good. Are those derived from Brent Pinkney's work, the general ANSIfication work, or something else?
squeak-dev@lists.squeakfoundation.org
-
Avi Bryant -
Doug Way -
Ned Konz -
Ross Boylan -
Stephan Rudlof -
tim Rowledge