Tom tmb@lumo.com wrote:
I was curious where the various implementations of a few dynamic languages stand now and I implemented some simple benchmarks. You can find them at http://www.lumo.com/squeak-bench/ In general, Squeak does pretty well compared to Python.
But why does Squeak do poorly compared to Java (even without JIT)? Is it mainly because Java has static typing and no escaping method contexts (blocks)?
On Fri, 30 Mar 2001 ajh18@cornell.edu wrote:
Tom tmb@lumo.com wrote:
I was curious where the various implementations of a few dynamic languages stand now and I implemented some simple benchmarks. You can find them at http://www.lumo.com/squeak-bench/ In general, Squeak does pretty well compared to Python.
But why does Squeak do poorly compared to Java (even without JIT)? Is it mainly because Java has static typing and no escaping method contexts (blocks)?
I don't think it does *poorly* by and large. For the first three tests, it's within a factor of three. Now considering that Squeak's bytecode interpreter *isn't* optimized for speed, but for being reasonably speedy while remaining comprehensible to mere mortals (i.e., *not* doing tricky stuff), I think it does quite well. (Note that this is a design goal of the Python VM too.)
Now, speaking totally randomly, I'll point out that whenever you use primative types in Java, especially numerics, you'd *better* see a big boost in speed, since you won't have to unboxed them. In inner loops this can make a big difference (which I hypothozie is why the mandelbrot tests show a bigger difference).
Note that, in squeak, you can achieve some of this gain by using IntegerArrays and FloatArrays.
It'd be interesting to see these tests ported to Dolphin Smalltalk (which has a highly tuned intepreter) and VisualWorks (which has a big ole beefy JIT).
Cheers, Bijan Parsia.
On Fri, Mar 30, 2001 at 10:48:17AM -0500, Bijan Parsia wrote:
I don't think it does *poorly* by and large. For the first three tests, it's within a factor of three.
I agree--I thought Squeak did quite well. To me, Squeak is closest in flavor to Python (even though the languages are quite different under the hood), and I wanted to see how they compared.
Also, take the benchmarks with a grain of salt--they haven't been reviewed by anybody yet, so there may be bugs/problems with them.
Now, speaking totally randomly, I'll point out that whenever you use primative types in Java, especially numerics, you'd *better* see a big boost in speed, since you won't have to unboxed them. In inner loops this can make a big difference (which I hypothozie is why the mandelbrot tests show a bigger difference).
Well, for interpreted code, I think boxing is mostly an issue related to the garbage collector.
Note that cmandelgrid uses user-defined boxed complex numbers in Java; in some sense, it's the most interesting test. I think it does so well still because the JDK has an excellent garbage collector. What I found particularly interesting is that g++ didn't manage to make cmandelgrid equivalent to mandelgrid (far from it, actually) even though in C++, there is no storage allocation involved.
I may try CMU CL at some point; to me, it represents what a good open source project can achieve with the compilation of a dynamic language.
In any case, at some point, I hope I'll be able to give the squeak jitter a try.
Cheers, Thomas.
On Fri, 30 Mar 2001, Tom wrote:
On Fri, Mar 30, 2001 at 10:48:17AM -0500, Bijan Parsia wrote:
I don't think it does *poorly* by and large. For the first three tests, it's within a factor of three.
I agree--I thought Squeak did quite well. To me, Squeak is closest in flavor to Python (even though the languages are quite different under the hood), and I wanted to see how they compared.
You might consider Ruby, which is way more Smalltalky.
Also, take the benchmarks with a grain of salt
Oh we do.
--they haven't been reviewed by anybody yet, so there may be bugs/problems with them.
Now, speaking totally randomly, I'll point out that whenever you use primative types in Java, especially numerics, you'd *better* see a big boost in speed, since you won't have to unboxed them. In inner loops this can make a big difference (which I hypothozie is why the mandelbrot tests show a bigger difference).
Well, for interpreted code, I think boxing is mostly an issue related to the garbage collector.
I don't think so. How do you mean?
Take a loop:
for i = 1 to 100 3.0*3.0
If the inner floats are boxed, you'll have to unboxed them each time through the loop. Plus do method lookup on the #*.
A method cache can minimize the latter (which is one reason, I suspect, that Squeak beats Python). But the former provides a constant overhead for each time through the loop.
Which adds up.
(of course the above code is a *terrible* example, since it's eliminable by simple optimizations; through in the requiste variables ;))
In Common Lisp you can add declarations and compile with speed up and safety down and get very fast code.
In Self, the dynamic profiler and optimizer can do some tricks.
In Squeak, you could use an FloatArray or such.
Note that cmandelgrid uses user-defined boxed complex numbers in Java; in some sense, it's the most interesting test.
I haven't yet inspected the code, but *hwere* the boxing and unboxing occurs makes a difference.
I think it does so well still because the JDK has an excellent garbage collector.
I have no idea why you think that matters. I aslo don't know why you think the JDK has a good, much less excellent, GC. Everything I've heard spits in its general direction.
Is any gcing even occuring in your tests? [snip]
Cheers, Bijan Parsia.
squeak-dev@lists.squeakfoundation.org
-
ajh18＠cornell.edu -
Bijan Parsia -
Tom