A rant, maybe a bad rant due to babbling about philosophical reasons, made me wonder how the programming languages stack up against this stuff: recursion. Or should I say the runtimes, since the programming language itself is nothing but a bunch of text. I know that the algorithm itself is bad, but that’s the whole point. I know that fibonacci(0) yields a wrong result, but for the sake of lazyness, I kept the original algorithm.
The source code of all the tests is available here in order to make the tests to be reproducible. There wasn’t a high number of runs, particularly for the rutimes that take more than the next ice age. But the results are pretty consistent for specific runtimes. The relevant systems specs are: Ubuntu 10.04 amd64 (up to date), Q9400 CPU.
Now, less talk, more results.
JavaScript (node.js/V8)
node -v: v0.4.12
time node fib.js
node fib.js 6.40s user 0.02s system 99% cpu 6.423 total
node fib.js 6.39s user 0.02s system 99% cpu 6.410 total
It may seem slow, but for a language with dynamic typing, it puts the rest from the same category to shame. Or most of them. Bear with me.
PHP
php -v: PHP 5.3.8 (cli)
time php fib.php
php fib.php 77.57s user 0.06s system 99% cpu 1:17.66 total
php fib.php 78.05s user 0.07s system 99% cpu 1:18.18 total
Compared to the V8 runtime, PHP seems to take an eternity. It happens that PHP isn’t bad at recursion because it uses the stack, but because the lack of speed of the runtime. But we’re not even halfway there. Stay tuned. PHP isn’t the only thing that sucks at recursion.
C
gcc -v: gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
make fib
time ./fib
./fib 2.84s user 0.00s system 100% cpu 2.840 total
./fib 2.84s user 0.00s system 100% cpu 2.835 total
It wasn’t a surprise that C came up to this result. Which makes the V8 result even more interesting.
Edit:
Forgot about the compiler optimization. Caught by Jabbles on HN.
gcc -O4 fib.c -o fib
time ./fib
./fib 0.65s user 0.01s system 100% cpu 0.657 total
gcc -O3 fib.c -o fib
time ./fib
./fib 0.66s user 0.00s system 100% cpu 0.657 total
gcc -O2 fib.c -o fib
time ./fib
./fib 1.54s user 0.00s system 100% cpu 1.535 total
gcc -O1 fib.c -o fib
time ./fib
./fib 0.00s user 0.00s system 0% cpu 0.001 total
time ./fib
165580141
./fib 2.06s user 0.00s system 99% cpu 2.060 total
For some reason, the O1 flag hates this code. Printing fibonacci(40) yields a result closer to the result without any O flag. This brings it past the Java result, but only for O3+.
/End Edit.
Lua
lua -v: Lua 5.1.4
time lua fib.lua
lua fib.lua 28.02s user 0.03s system 99% cpu 28.081 total
lua fib.lua 28.86s user 0.02s system 99% cpu 28.883 total./luajit -v: LuaJIT 2.0.0-beta8
time ./luajit fib.lua
./luajit fib.lua 10.59s user 0.00s system 99% cpu 10.591 total
./luajit fib.lua 10.58s user 0.01s system 99% cpu 10.610 total
Tested both of the implementations that I know of. I guess this article isn’t that funny for the generations of Lua coders that laugh about V8 in somebody’s face. Don’t get me wrong, I like Lua due to its simplicity, but in the speed realm, I still need to do some tests to verify some of those claims that sometimes appear to be overly inflated.
Edit:
With the following Lua script:
local function fibonacci(n) if n < 2 then return 1 else return fibonacci(n - 2) + fibonacci(n - 1) end end fibonacci(40) |
the results are getting better:
time lua fib.lua
lua fib.lua 24.17s user 0.08s system 99% cpu 24.281 total
lua fib.lua 24.24s user 0.01s system 99% cpu 24.307 total[with LuaJIT v2.0.0-beta8 GIT HEAD]
time ./luajit fib.lua
./luajit fib.lua 2.02s user 0.00s system 99% cpu 2.026 total
./luajit fib.lua 2.02s user 0.00s system 99% cpu 2.023 total
Now, some of the Lua chops can have a lulz about V8. This project is getting more interesting, especially for pairing LuaJIT with luafcgid. I forgot about the local keyword since my Lua experience is limited to basic testing. Nice comeback!
/End Edit.
Python
python -V: Python 2.6.5
time python fib.py
python fib.py 59.42s user 0.02s system 99% cpu 59.494 total
python fib.py 59.27s user 0.05s system 99% cpu 59.375 total./configure
make -j 4
./python -V: Python 2.7.2
time ./python fib.py
./python fib.py 61.29s user 0.03s system 99% cpu 1:01.35 total
./python fib.py 61.38s user 0.06s system 99% cpu 1:01.48 total./configure
make -j 4
./python -V: Python 3.2.2
./python fib.py 71.23s user 0.08s system 99% cpu 1:11.33 total
./python fib.py 70.31s user 0.06s system 99% cpu 1:10.39 total./pypy -V
Python 2.7.1 (d8ac7d23d3ec, Aug 17 2011, 11:51:19)
[PyPy 1.6.0 with GCC 4.4.3]
./pypy fib.py 4.61s user 0.07s system 99% cpu 4.708 total
./pypy fib.py 4.81s user 0.01s system 99% cpu 4.853 total
Happens to have 2.6.5 around because Ubuntu says so. But in order to make the potential trolls to STFU about not using the latest versions, I made some fresh builds of 2.7.2 and 3.2.2. It gets even suckier with recent versions. In fact, the CPython runtime is struggling to catch up the PHP runtime on the slowness realm. The only Python runtime that is actually very impressive about the recursion stuff is PyPy. Which brings me to the first statement: the language is just a bunch of text. The runtime is the piece that sucks or does not suck. PyPy proves that with talented people shepherding the project, the language of the runtime implementation is quite irrelevant. This is the first implementation of a JIT that passes V8 as well.
Ruby
ruby -v: ruby 1.8.7 (2010-01-10 patchlevel 249) [x86_64-linux]
time ruby fib.rb
ruby fib.rb 233.40s user 66.94s system 99% cpu 5:00.55 total
ruby fib.rb 231.75s user 68.08s system 99% cpu 4:59.99 total./configure
make -j 4
./miniruby -v: ruby 1.9.3dev (2011-09-23 revision 33323) [x86_64-linux]
time ./miniruby fib.rb
./miniruby fib.rb 36.05s user 0.01s system 99% cpu 36.073 total
./miniruby fib.rb 35.92s user 0.04s system 99% cpu 35.978 total
Everytime a Ruby fan says that “thou shalt not care about the runtime speed” makes me laugh so hard up to the point of bursting into tears. Seriously, I couldn’t imagine that MRI sucks that hard at recursion. I barely had the patience to even run this code. KRI washes part of the shame though while it scores closely to the Lua implementation. If you’re asking why I used the miniruby binary, the reason is that the ruby binary complained about not having rubygems.rb. I am bad at figuring out what’s missing from a Ruby stack. But it made the fib.rb work.
Perl
perl -v: This is perl, v5.10.1 (*) built for x86_64-linux-gnu-thread-multi
time perl fib.pl
perl fib.pl 125.60s user 0.17s system 99% cpu 2:05.92 total
perl fib.pl 124.06s user 0.10s system 99% cpu 2:04.20 total./Configure [accepted all defaults, specifically built without threading]
make
./perl -v: This is perl 5, version 14, subversion 2 (v5.14.2) built for x86_64-linux
./perl fib.pl 100.12s user 0.09s system 99% cpu 1:40.29 total
./perl fib.pl 100.38s user 0.05s system 99% cpu 1:40.63 total
At first I didn’t want to bother with Perl, but then I remembered the legions of Perl fans ranting about the PHP recursion. I know that this is an inflammatory statement, but next time, people, please keep up with the facts. I guess you aren’t that smug now.
D
gdc -v: gcc version 4.3.4 (Ubuntu 1:1.046-4.3.4-3ubuntu1)
gdc -o fib fib.c (same source as the C binary)
time ./fib
./fib 2.82s user 0.00s system 100% cpu 2.817 total
./fib 2.82s user 0.00s system 100% cpu 2.814 total
Predictable results from a language from the same family as C/C++. Slightly faster binary that the C version (although the same source code), but I guess most people won’t notice.
Java
javac -version: gcj-4.4 (Ubuntu 4.4.3-1ubuntu4.1) 4.4.3
java -version
java version “1.6.0_20”
OpenJDK Runtime Environment (IcedTea6 1.9.9) (6b20-1.9.9-0ubuntu1~10.04.2)
OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
javac fib.java
time java fib
java fib 0.86s user 0.02s system 99% cpu 0.882 total
java fib 0.86s user 0.02s system 100% cpu 0.872 total
I admit that sometimes I use to tell this joke: knock! knock!; who’s there?; [very long pause]; Java. I guess that now is a good time to swallow my own words. Not only that Java puts the other JIT implementations to shame, the rest of the VMs to shame, it also obliterates the statically compiled C and D binaries at their own favorite game aka the runtime speed. My first reaction was: WTF, there’s got to be a mistake! Printing some junk to STDIO confirmed the same results between C and Java. Newbie warning: this is my first Java application. No, really! Don’t bash me for the lack of understanding of the usage of the static keyword. I don’t understand if it actually helps the runtime. I managed to put together the code by reading how to write a simple HelloWorldApp. Experienced Java chops may explain it though.
You used an outdated version of Ruby and a developement version of Ruby. Here are the results from the latest PRODUCTION version of Ruby (1.9.2), compared to Java.
asiekierka@debian-main:~/fib/fib$ time ruby fib.rb
real 0m25.585s
user 0m25.582s
sys 0m0.016s
asiekierka@debian-main:~/fib/fib$ time ruby fib.rb
real 0m27.165s
user 0m27.162s
sys 0m0.008s
asiekierka@debian-main:~/fib/fib$ javac fib.java
asiekierka@debian-main:~/fib/fib$ time java fib
real 0m0.834s
user 0m0.812s
sys 0m0.020s
asiekierka@debian-main:~/fib/fib$ time java fib
real 0m0.828s
user 0m0.820s
sys 0m0.012s
asiekierka@debian-main:~/fib/fib$ ruby -v
ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-linux]
asiekierka@debian-main:~/fib/fib$ javac -version
javac 1.6.0_23
asiekierka@debian-main:~/fib/fib$ java -version
java version “1.6.0_23”
OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre7-1)
OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
I am aware which versions I used. 1.8.7 happens to ship with Ubuntu, while 1.9.3-dev didn’t seem inappropriate. I doubt there are large improvements, but I may give it a run for its money and update the article with more info.
Also, different Ruby VMs. Like you compared Lua and LuaJIT, here’s JRuby 1.6.4:
asiekierka@debian-main:~/Pobrane$ time java -jar jruby-*.jar ../fib/fib/fib.rb
real 0m10.564s
user 0m10.521s
sys 0m0.092s
asiekierka@debian-main:~/Pobrane$ time java -jar jruby-*.jar ../fib/fib/fib.rb
real 0m9.829s
user 0m10.265s
sys 0m0.076s
As you can see, it’s faster.
Have you not forgot the -O flag for gdc?
@asiekierka: unfortunately I don’t have other JVM languages installed. Presumably, Jython is also faster than plain CPython.
@Andreas: I forgot, but the -O flags passed to the C version made me remember about this. I’ll make an update.
Typo alert: in “bare with me”, “bare” should be “bear”. (Unless you meant to imply that readers should remove their clothing in solidarity.)
No lisp variants?
A three-line racket implementation:
(define (fib n)
(if (< n 2) 1 (+ (fib (- n 2)) (fib (- n 1)))))
(print (fib 40))
Running
time racket fib.rkt
gives me:
real 0m3.187s
user 0m3.180s
sys 0m0.000s
Not bad for an untyped scripting language.
To make this experiment a little more interesting you should make use of the produced result in some way (maybe print the result). Otherwise, a “smart” compiler removes the function call because the result is never used.
@Sam LG: thanks! Fixed.
@MDawg: unfortunately my Lisp-foo is NULL.
@Mattias: yes, it may happen with explicit compiler optimization (such as -O1 for C). I guess that writing to STDIO shouldn’t impact the end results that much. I was aware of this after the feedback started to come in.
Hi. Very interesting problem. Just thought it would be interesting to add benchmarkings for some more languages, in particular Fortran 95 and perhaps lisp? I made a simple fortran program. Here are my timings, including the timing for C:
$ gcc -v
gcc version 4.6.1 (Debian 4.6.1-4)
$ gcc -O4 fib.c -o cfib
$ time ./cfib
./cfib 0.46s user 0.00s system 99% cpu 0.468 total
$ gfortran -O3 fib.f90 -o fibf90
$ time ./fibf90
./fibf90 0.39s user 0.00s system 99% cpu 0.394 total
@Karl: You could try a service like pastebin or Gist. I’ll remove the code.
@SaltwaterC: Thanks, good idea. Here is the pastebin url:
http://pastebin.com/ddSPUb8E
The following Lua program runs much faster with LuaJIT (and git HEAD is another 30% better than beta8):
local function fib(n)
if n < 2 then return 1 end
return fib(n-1) + fib(n-2)
end
fib(40)
Funny that you have complained about Java without trying it yourself 🙂 My guess is that you have had some bad experiences with GUI programs. It’s easy to make a sluggish GUI if you don’t use Threads properly.
@The LuaJIT guy: thanks! I updated the page with that info. Missed the local keyword.
@;-): I also had bad experiences with Java on the server, some because poor choices into the application core, some because of poor choices from the guys that take care of Tomcat. In the end, throwing away the legacy app and switching to a new stack has its gains. However, it managed to stay in production for a respectable amount of time.
I get about 1.26 or 1.27 seconds with Go on a 2.66 GHz i7 Macbook Pro running OS X Lion.
https://gist.github.com/1421189
Closer to Java than C or D but I haven’t run your code on my system for a true comparison.