Hi folks!
Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
Cog! ==== Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download): Pystone(1.1) time for 50000 passes = 0.06 This machine benchmarks at 833333.3 pystones/second
NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
Regular Squeak ============== Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src): Pystone(1.1) time for 50000 passes = 0.503 This machine benchmarks at 99403.6 pystones/second
Regular CPython =============== CPython 3.1.2 (newest in Ubuntu Meerkat, minimal): gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py Pystone(1.1) time for 50000 passes = 0.57 This machine benchmarks at 87719.3 pystones/second
NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
Pypy 1.4 ======== wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2 gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py Pystone(1.1) time for 50000 passes = 0.15 This machine benchmarks at 333333 pystones/second gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000 Pystone(1.1) time for 5000000 passes = 4.8 This machine benchmarks at 1.04167e+06 pystones/second
NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
Shedskin 0.7 ============ sudo apt-get install g++ libpcre3-dev libgc-dev python-dev sudo dpkg -i shedskin_0.7_all.deb wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz shedskin pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ make g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone* -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 2500000.000000 pystones/second
NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
Summary =======
- The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
- Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.
- Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
- Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
regards, Göran
Hi Göran,
is it possible to repeat the runs with a much higher iteration count so that we get less temporal quantization? The numbers below are extremely round simply because the number of iterations are so low (on modern hardware). Also, is the source available? URL? Ta!
thanks; this has brightened my day!!
best Eliot
2011/2/7 Göran Krampe goran@krampe.se
Hi folks!
Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
Cog!
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download): Pystone(1.1) time for 50000 passes = 0.06 This machine benchmarks at 833333.3 pystones/second
NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
Regular Squeak
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src): Pystone(1.1) time for 50000 passes = 0.503 This machine benchmarks at 99403.6 pystones/second
Regular CPython
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal): gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py Pystone(1.1) time for 50000 passes = 0.57 This machine benchmarks at 87719.3 pystones/second
NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
Pypy 1.4
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2 gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py Pystone(1.1) time for 50000 passes = 0.15 This machine benchmarks at 333333 pystones/second gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000 Pystone(1.1) time for 5000000 passes = 4.8 This machine benchmarks at 1.04167e+06 pystones/second
NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
Shedskin 0.7
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev sudo dpkg -i shedskin_0.7_all.deb wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz shedskin pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ make g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone* -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 2500000.000000 pystones/second
NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
Summary
- The regular Squeak VM has not been overrun by CPython in these 6 years
time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
- Cog is brutally fast on this one. Compared to CPython and regular Squeak
almost 10x faster.
- Pypy is about 20% faster than Cog if given enough time to actually start
jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
- Shedskin is the "state of the art" of statically compiling Python via C++
using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
regards, Göran
On Mon, 7 Feb 2011, Eliot Miranda wrote:
Hi Göran,
is it possible to repeat the runs with a much higher iteration count so that we get less temporal quantization? The numbers below are extremely round simply because the number of iterations are so low (on modern hardware). Also, is the source available? URL? Ta!
The code is available on SqueakMap: http://map.squeak.org/package/81a1617a-228e-4fcf-80fe-fada0ccde940/autoversi...
Levente
thanks; this has brightened my day!!
best Eliot
2011/2/7 Göran Krampe goran@krampe.se
Hi folks!
Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
Cog!
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download): Pystone(1.1) time for 50000 passes = 0.06 This machine benchmarks at 833333.3 pystones/second
NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
Regular Squeak
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src): Pystone(1.1) time for 50000 passes = 0.503 This machine benchmarks at 99403.6 pystones/second
Regular CPython
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal): gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py Pystone(1.1) time for 50000 passes = 0.57 This machine benchmarks at 87719.3 pystones/second
NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
Pypy 1.4
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2 gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py Pystone(1.1) time for 50000 passes = 0.15 This machine benchmarks at 333333 pystones/second gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000 Pystone(1.1) time for 5000000 passes = 4.8 This machine benchmarks at 1.04167e+06 pystones/second
NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
Shedskin 0.7
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev sudo dpkg -i shedskin_0.7_all.deb wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz shedskin pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ make g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone* -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 2500000.000000 pystones/second
NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
Summary
- The regular Squeak VM has not been overrun by CPython in these 6 years
time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
- Cog is brutally fast on this one. Compared to CPython and regular Squeak
almost 10x faster.
- Pypy is about 20% faster than Cog if given enough time to actually start
jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
- Shedskin is the "state of the art" of statically compiling Python via C++
using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
regards, Göran
One small correction:
Rerunning Cog and CPython too with 5000000 loops didn't change the numbers much, but for Shedskin it turns out more proper:
gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 3546099.290780 pystones/second
50000 loops was just too small. So in fact it is about 4.4x faster than Cog. And yes, the ported pystone for shedskin had the loops hard coded. :)
regards, Göran
This is really interesting to see that type inferencing can really help. This is definitively a topic I would like to explore in the coming years.
Stef
Hi folks!
Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
Cog!
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download): Pystone(1.1) time for 50000 passes = 0.06 This machine benchmarks at 833333.3 pystones/second
NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
Regular Squeak
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src): Pystone(1.1) time for 50000 passes = 0.503 This machine benchmarks at 99403.6 pystones/second
Regular CPython
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal): gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py Pystone(1.1) time for 50000 passes = 0.57 This machine benchmarks at 87719.3 pystones/second
NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
Pypy 1.4
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2 gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py Pystone(1.1) time for 50000 passes = 0.15 This machine benchmarks at 333333 pystones/second gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000 Pystone(1.1) time for 5000000 passes = 4.8 This machine benchmarks at 1.04167e+06 pystones/second
NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
Shedskin 0.7
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev sudo dpkg -i shedskin_0.7_all.deb wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz shedskin pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ make g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone* -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 2500000.000000 pystones/second
NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
Summary
The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.
Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
regards, Göran
Just a detail, Python range(a,b) are semi open [a,b) think like C for(i=0;i<n; i++)
In the Smalltalk proc8:withwith:with:with:: the range was interpreted as [a,b]. This Smalltalk (and Cog) unfairly perform too much work.
Nicolas
2011/2/8 stephane ducasse stephane.ducasse@gmail.com:
This is really interesting to see that type inferencing can really help. This is definitively a topic I would like to explore in the coming years.
Stef
Hi folks!
Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
Cog!
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download): Pystone(1.1) time for 50000 passes = 0.06 This machine benchmarks at 833333.3 pystones/second
NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
Regular Squeak
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src): Pystone(1.1) time for 50000 passes = 0.503 This machine benchmarks at 99403.6 pystones/second
Regular CPython
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal): gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py Pystone(1.1) time for 50000 passes = 0.57 This machine benchmarks at 87719.3 pystones/second
NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
Pypy 1.4
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2 gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py Pystone(1.1) time for 50000 passes = 0.15 This machine benchmarks at 333333 pystones/second gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000 Pystone(1.1) time for 5000000 passes = 4.8 This machine benchmarks at 1.04167e+06 pystones/second
NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
Shedskin 0.7
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev sudo dpkg -i shedskin_0.7_all.deb wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz shedskin pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ make g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone* -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 2500000.000000 pystones/second
NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
Summary
The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.
Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
regards, Göran
On Tue, 8 Feb 2011, Nicolas Cellier wrote:
Just a detail, Python range(a,b) are semi open [a,b) think like C for(i=0;i<n; i++)
In the Smalltalk proc8:withwith:with:with:: the range was interpreted as [a,b]. This Smalltalk (and Cog) unfairly perform too much work.
And there's another one. In #proc0:block: the loops are (1 to: loops) do: instead of 1 to: loops do:.
Levente
Nicolas
2011/2/8 stephane ducasse stephane.ducasse@gmail.com:
This is really interesting to see that type inferencing can really help. This is definitively a topic I would like to explore in the coming years.
Stef
Hi folks!
Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
Cog!
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download): Pystone(1.1) time for 50000 passes = 0.06 This machine benchmarks at 833333.3 pystones/second
NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
Regular Squeak
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src): Pystone(1.1) time for 50000 passes = 0.503 This machine benchmarks at 99403.6 pystones/second
Regular CPython
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal): gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py Pystone(1.1) time for 50000 passes = 0.57 This machine benchmarks at 87719.3 pystones/second
NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
Pypy 1.4
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2 gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py Pystone(1.1) time for 50000 passes = 0.15 This machine benchmarks at 333333 pystones/second gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000 Pystone(1.1) time for 5000000 passes = 4.8 This machine benchmarks at 1.04167e+06 pystones/second
NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
Shedskin 0.7
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev sudo dpkg -i shedskin_0.7_all.deb wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz shedskin pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ make g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone* -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 2500000.000000 pystones/second
NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
Summary
The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.
Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
regards, Göran
2011/2/8 Levente Uzonyi leves@elte.hu:
On Tue, 8 Feb 2011, Nicolas Cellier wrote:
Just a detail, Python range(a,b) are semi open [a,b) think like C for(i=0;i<n; i++)
In the Smalltalk proc8:withwith:with:with:: the range was interpreted as [a,b]. This Smalltalk (and Cog) unfairly perform too much work.
And there's another one. In #proc0:block: the loops are (1 to: loops) do: instead of 1 to: loops do:.
Levente
Yes I noticed, but empty loop time is subtracted. I wonder if it makes a difference.
Nicolas
Nicolas
2011/2/8 stephane ducasse stephane.ducasse@gmail.com:
This is really interesting to see that type inferencing can really help. This is definitively a topic I would like to explore in the coming years.
Stef
Hi folks!
Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
Cog!
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download): Pystone(1.1) time for 50000 passes = 0.06 This machine benchmarks at 833333.3 pystones/second
NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
Regular Squeak
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src): Pystone(1.1) time for 50000 passes = 0.503 This machine benchmarks at 99403.6 pystones/second
Regular CPython
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal): gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py Pystone(1.1) time for 50000 passes = 0.57 This machine benchmarks at 87719.3 pystones/second
NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
Pypy 1.4
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2 gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py Pystone(1.1) time for 50000 passes = 0.15 This machine benchmarks at 333333 pystones/second gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000 Pystone(1.1) time for 5000000 passes = 4.8 This machine benchmarks at 1.04167e+06 pystones/second
NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
Shedskin 0.7
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev sudo dpkg -i shedskin_0.7_all.deb wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz shedskin pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ make g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone* -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 2500000.000000 pystones/second
NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
Summary
The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.
Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
regards, Göran
On Tue, 8 Feb 2011, Nicolas Cellier wrote:
2011/2/8 Levente Uzonyi leves@elte.hu:
On Tue, 8 Feb 2011, Nicolas Cellier wrote:
Just a detail, Python range(a,b) are semi open [a,b) think like C for(i=0;i<n; i++)
In the Smalltalk proc8:withwith:with:with:: the range was interpreted as [a,b]. This Smalltalk (and Cog) unfairly perform too much work.
And there's another one. In #proc0:block: the loops are (1 to: loops) do: instead of 1 to: loops do:.
Levente
Yes I noticed, but empty loop time is subtracted. I wonder if it makes a difference.
~10% on my pc using Cog r2361. Probably the blocks make the difference.
Levente
Nicolas
Nicolas
2011/2/8 stephane ducasse stephane.ducasse@gmail.com:
This is really interesting to see that type inferencing can really help. This is definitively a topic I would like to explore in the coming years.
Stef
Hi folks!
Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
Cog!
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download): Pystone(1.1) time for 50000 passes = 0.06 This machine benchmarks at 833333.3 pystones/second
NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
Regular Squeak
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src): Pystone(1.1) time for 50000 passes = 0.503 This machine benchmarks at 99403.6 pystones/second
Regular CPython
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal): gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py Pystone(1.1) time for 50000 passes = 0.57 This machine benchmarks at 87719.3 pystones/second
NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
Pypy 1.4
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2 gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py Pystone(1.1) time for 50000 passes = 0.15 This machine benchmarks at 333333 pystones/second gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000 Pystone(1.1) time for 5000000 passes = 4.8 This machine benchmarks at 1.04167e+06 pystones/second
NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
Shedskin 0.7
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev sudo dpkg -i shedskin_0.7_all.deb wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz shedskin pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ make g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone* -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 2500000.000000 pystones/second
NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
Summary
The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.
Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
regards, Göran
On Tue, Feb 8, 2011 at 5:01 AM, Nicolas Cellier < nicolas.cellier.aka.nice@gmail.com> wrote:
2011/2/8 Levente Uzonyi leves@elte.hu:
On Tue, 8 Feb 2011, Nicolas Cellier wrote:
Just a detail, Python range(a,b) are semi open [a,b) think like C for(i=0;i<n; i++)
In the Smalltalk proc8:withwith:with:with:: the range was interpreted as
[a,b].
This Smalltalk (and Cog) unfairly perform too much work.
And there's another one. In #proc0:block: the loops are (1 to: loops) do:
instead of 1 to: loops do:.
Levente
Yes I noticed, but empty loop time is subtracted. I wonder if it makes a difference.
There is a huge difference between 1 to: loops do: [:i|] and (1 to: loops) do: [:i|]. IIRC the empty block is written as 1 to: loops do: [:i|].
SimpleStackBasedCogit: Time millisecondsToRun: [| loops | loops := 100000000. 1 to: loops do: [:i|]] 757
Time millisecondsToRun: [| loops | loops := 100000000. (1 to: loops) do: [:i|]] 16953
StackToRegisterMappingCogit Time millisecondsToRun: [| loops | loops := 100000000. 1 to: loops do: [:i|]] 244
Time millisecondsToRun: [| loops | loops := 100000000. (1 to: loops) do: [:i|]] 17149
Nicolas
Nicolas
2011/2/8 stephane ducasse stephane.ducasse@gmail.com:
This is really interesting to see that type inferencing can really
help.
This is definitively a topic I would like to explore in the coming
years.
Stef
Hi folks!
Since we are on the verge of 4.2, and we have a brand new Cog VM to
play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
At that time Squeak was around 5 times faster than CPython. How do we
stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit
CPU.
Cog!
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary
download):
Pystone(1.1) time for 50000 passes = 0.06 This machine benchmarks at 833333.3 pystones/second
NOTE: AFAICT running with more passes does not improve it. Also, not
sure if I could get more out of this if I built from source on my box.
Regular Squeak
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357
(built from src):
Pystone(1.1) time for 50000 passes = 0.503 This machine benchmarks at 99403.6 pystones/second
Regular CPython
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal): gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py Pystone(1.1) time for 50000 passes = 0.57 This machine benchmarks at 87719.3 pystones/second
NOTE: 3.2 is reportedly a teeny bit faster. Also not built from
source.
Pypy 1.4
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2 gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
./lib-python/2.5.2/test/pystone.py
Pystone(1.1) time for 50000 passes = 0.15 This machine benchmarks at 333333 pystones/second gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
./lib-python/2.5.2/test/pystone.py 5000000
Pystone(1.1) time for 5000000 passes = 4.8 This machine benchmarks at 1.04167e+06 pystones/second
NOTE: Also not built from source. Here we run pystone a second time
with 100x more loops and get a substantially better number.
Shedskin 0.7
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev sudo dpkg -i shedskin_0.7_all.deb wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz shedskin pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ make g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I.
-I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone
gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls
-la pystone*
-rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 2500000.000000 pystones/second
NOTE: I am wondering a bit about this. It tells the same whatever
loops I give it... But ok, perhaps it is all fine.
Summary
- The regular Squeak VM has not been overrun by CPython in these 6
years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
- Cog is brutally fast on this one. Compared to CPython and regular
Squeak almost 10x faster.
- Pypy is about 20% faster than Cog if given enough time to actually
start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
- Shedskin is the "state of the art" of statically compiling Python
via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
regards, Göran
2011/2/8 Eliot Miranda eliot.miranda@gmail.com:
On Tue, Feb 8, 2011 at 5:01 AM, Nicolas Cellier nicolas.cellier.aka.nice@gmail.com wrote:
2011/2/8 Levente Uzonyi leves@elte.hu:
On Tue, 8 Feb 2011, Nicolas Cellier wrote:
Just a detail, Python range(a,b) are semi open [a,b) think like C for(i=0;i<n; i++)
In the Smalltalk proc8:withwith:with:with:: the range was interpreted as [a,b]. This Smalltalk (and Cog) unfairly perform too much work.
And there's another one. In #proc0:block: the loops are (1 to: loops) do: instead of 1 to: loops do:.
Levente
Yes I noticed, but empty loop time is subtracted. I wonder if it makes a difference.
There is a huge difference between 1 to: loops do: [:i|] and (1 to: loops) do: [:i|]. IIRC the empty block is written as 1 to: loops do: [:i|]. SimpleStackBasedCogit: Time millisecondsToRun: [| loops | loops := 100000000. 1 to: loops do: [:i|]] 757 Time millisecondsToRun: [| loops | loops := 100000000. (1 to: loops) do: [:i|]] 16953 StackToRegisterMappingCogit Time millisecondsToRun: [| loops | loops := 100000000. 1 to: loops do: [:i|]] 244 Time millisecondsToRun: [| loops | loops := 100000000. (1 to: loops) do: [:i|]] 17149
I was more after the difference with empty loops, like:
[| loops x | loops := 10000000. x := 1. (1 to: loops) do: [:i| x := x + 1. x squared]] timeToRun - [| loops | loops := 10000000. (1 to: loops) do: [:i| ]] timeToRun 5405 5397
[| loops x | loops := 10000000. x := 1. 1 to: loops do: [:i| x := x + 1. x squared]] timeToRun - [| loops | loops := 10000000. 1 to: loops do: [:i| ]] timeToRun 5272 5205
Nicolas
On 02/08/2011 05:26 PM, Eliot Miranda wrote:
On Tue, Feb 8, 2011 at 5:01 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com mailto:nicolas.cellier.aka.nice@gmail.com> wrote:
2011/2/8 Levente Uzonyi <leves@elte.hu <mailto:leves@elte.hu>>: > > On Tue, 8 Feb 2011, Nicolas Cellier wrote: > >> >> Just a detail, Python range(a,b) are semi open [a,b) >> think like C for(i=0;i<n; i++) >> >> In the Smalltalk proc8:withwith:with:with:: the range was interpreted as [a,b]. >> This Smalltalk (and Cog) unfairly perform too much work. > > And there's another one. In #proc0:block: the loops are (1 to: loops) do: instead of 1 to: loops do:. > > > Levente > Yes I noticed, but empty loop time is subtracted. I wonder if it makes a difference.There is a huge difference between 1 to: loops do: [:i|] and (1 to: loops) do: [:i|]. IIRC the empty block is written as 1 to: loops do: [:i|].
When I did the port I did not try to rewrite the code to "proper Smalltalk style" since it would then typically not compare the VMs. At least that was my thought (it was a while since I did this).
So I wrote it with *explicit* creation of Intervals since I presumed Python created ranges - but perhaps Python... never mind, google tells us:
http://wiki.python.org/moin/PythonSpeed/PerformanceTips#Usexrangeinsteadofra...
Aha! So yes, it was a fair comparison before, but now in Python3 they don't instantiate the range but create an iterator instead. Then we should change it too I guess :)
I will make a new version of it.
regards, Göran
Hi!
So a few small notes were made by a few people on how Sqystone is written, so I tweaked it a bit and these are the more correct numbers, all run with 5000000 loops.
On 02/08/2011 12:01 AM, Göran Krampe wrote:
Cog!
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download): Pystone(1.1) time for 50000 passes = 0.06 This machine benchmarks at 833333.3 pystones/second
Pystone(1.1) time for 5000000 passes = 5.448 This machine benchmarks at 917768.0 pystones/second
Regular Squeak
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src): Pystone(1.1) time for 50000 passes = 0.503 This machine benchmarks at 99403.6 pystones/second
Pystone(1.1) time for 5000000 passes = 46.448 This machine benchmarks at 107647.3 pystones/second
Regular CPython
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal): gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py Pystone(1.1) time for 50000 passes = 0.57 This machine benchmarks at 87719.3 pystones/second
NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py 5000000 Pystone(1.1) time for 5000000 passes = 58.6 This machine benchmarks at 85324.2 pystones/second
Pypy 1.4
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2 gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py Pystone(1.1) time for 50000 passes = 0.15 This machine benchmarks at 333333 pystones/second gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000 Pystone(1.1) time for 5000000 passes = 4.8 This machine benchmarks at 1.04167e+06 pystones/second
NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
Shedskin 0.7
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev sudo dpkg -i shedskin_0.7_all.deb wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz shedskin pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ make g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone* -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 2500000.000000 pystones/second
5000000 loops gave a more precise number:
gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 3378378.378378 pystones/second
Summary
- The regular Squeak VM has not been overrun by CPython in these 6 years
time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
Actually a bit more (not latest 3.2 Python used though):
Squeak = 1.28 x CPython
- Cog is brutally fast on this one. Compared to CPython and regular
Squeak almost 10x faster.
Cog = 8.4 x Squeak Cog = 10.75 x CPython
- Pypy is about 20% faster than Cog if given enough time to actually
start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
Cog = 0.88 x Pypy
- Shedskin is the "state of the art" of statically compiling Python via
C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
Shedskin = 3.68 x Cog
regards, Göran
On 09.02.2011, at 01:50, Göran Krampe goran@krampe.se wrote:
Shedskin can only run a subset of
Python.
Shedskin = 3.68 x Cog
A Slang version would be the equivalent of this, right?
- Bert -
On 02/09/2011 09:01 AM, Bert Freudenberg wrote:
On 09.02.2011, at 01:50, Göran Krampegoran@krampe.se wrote:
Shedskin can only run a subset of
Python.
Shedskin = 3.68 x Cog
A Slang version would be the equivalent of this, right?
- Bert -
Mmmm, well, Shedskin takes Python source as input, does type inference and produces C++ source, then compiled as normal. But yes, it is a restricted Python, but AFAICT not at all as restricted as Slang.
So I wouldn't say Slang is really fair - being C in disguise :)
BUT... Unladen Swallow and IronPython might be interesting to test too. Unladen has a goal of being at least 5x faster than CPython, which in the light of Cog seems like a rather modest goal. AFAIK it is not fast at all right now though.
Still, as we all know - Python *is* and probably always will be *CPython* - all other implementations will be "imitations" with limited auidences. Now I note there is a py3k-jit branch, but someone mentioned that it hasn't moved in 7 months.
Pypy is probably the only project that possibly could step in and "become" a next generation platform for Python - or what do you guys that have more insight into the Python community than I do?
regards, Göran
On Feb 9, 2011, at 10:06 AM, Göran Krampe wrote:
On 02/09/2011 09:01 AM, Bert Freudenberg wrote:
On 09.02.2011, at 01:50, Göran Krampegoran@krampe.se wrote:
Shedskin can only run a subset of
Python.
Shedskin = 3.68 x Cog
A Slang version would be the equivalent of this, right?
- Bert -
Mmmm, well, Shedskin takes Python source as input, does type inference and produces C++ source, then compiled as normal. But yes, it is a restricted Python, but AFAICT not at all as restricted as Slang.
restricted python is really a subset of python so that some type analysis can be performed. So this is clearly not like Slang.
So I wouldn't say Slang is really fair - being C in disguise :)
BUT... Unladen Swallow and IronPython might be interesting to test too. Unladen has a goal of being at least 5x faster than CPython, which in the light of Cog seems like a rather modest goal. AFAIK it is not fast at all right now though.
Still, as we all know - Python *is* and probably always will be *CPython* - all other implementations will be "imitations" with limited auidences. Now I note there is a py3k-jit branch, but someone mentioned that it hasn't moved in 7 months.
Pypy is probably the only project that possibly could step in and "become" a next generation platform for Python - or what do you guys that have more insight into the Python community than I do?
at least we should have a build system that monitor the performance of the vm as pypy are doing.
regards, Göran
Hi Göran:
Those numbers are certainly fun, but wouldn't it be much more interesting to get the SqueakVM and Cog back into the Language Benchmark Game?
http://shootout.alioth.debian.org/
The Dhrystone benchmarks also do not really have the reputation of being, well, interesting, i.e., kind of resemble standard workloads.
Bringing Cog to the Benchmark game would also allow comparison with for instance LuaJIT2.
Best regards Stefan
On 09 Feb 2011, at 01:50, Göran Krampe wrote:
Hi!
So a few small notes were made by a few people on how Sqystone is written, so I tweaked it a bit and these are the more correct numbers, all run with 5000000 loops.
On 02/08/2011 12:01 AM, Göran Krampe wrote:
Cog!
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download): Pystone(1.1) time for 50000 passes = 0.06 This machine benchmarks at 833333.3 pystones/second
Pystone(1.1) time for 5000000 passes = 5.448 This machine benchmarks at 917768.0 pystones/second
Regular Squeak
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src): Pystone(1.1) time for 50000 passes = 0.503 This machine benchmarks at 99403.6 pystones/second
Pystone(1.1) time for 5000000 passes = 46.448 This machine benchmarks at 107647.3 pystones/second
Regular CPython
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal): gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py Pystone(1.1) time for 50000 passes = 0.57 This machine benchmarks at 87719.3 pystones/second
NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py 5000000 Pystone(1.1) time for 5000000 passes = 58.6 This machine benchmarks at 85324.2 pystones/second
Pypy 1.4
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2 gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py Pystone(1.1) time for 50000 passes = 0.15 This machine benchmarks at 333333 pystones/second gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000 Pystone(1.1) time for 5000000 passes = 4.8 This machine benchmarks at 1.04167e+06 pystones/second
NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
Shedskin 0.7
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev sudo dpkg -i shedskin_0.7_all.deb wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz shedskin pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ make g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone* -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 2500000.000000 pystones/second
5000000 loops gave a more precise number:
gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone This machine benchmarks at 3378378.378378 pystones/second
Summary
- The regular Squeak VM has not been overrun by CPython in these 6 years
time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
Actually a bit more (not latest 3.2 Python used though):
Squeak = 1.28 x CPython
- Cog is brutally fast on this one. Compared to CPython and regular
Squeak almost 10x faster.
Cog = 8.4 x Squeak Cog = 10.75 x CPython
- Pypy is about 20% faster than Cog if given enough time to actually
start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
Cog = 0.88 x Pypy
- Shedskin is the "state of the art" of statically compiling Python via
C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
Shedskin = 3.68 x Cog
regards, Göran
vm-dev@lists.squeakfoundation.org
-
Bert Freudenberg -
Eliot Miranda -
Göran Krampe -
Levente Uzonyi -
Nicolas Cellier -
Stefan Marr -
stephane ducasse