For one of my small private projects, I wanted to know if I could implement the program in Java instead of C++. The only critical feature for this program was speed, so I needed to know how fast Java is today using the latest VM, compared to C++ using a modern optimizing compiler.
There are lots of tests and benchmarks available on the web, but usually I don't trust any statistics I did not fake myself. So I did my own benchmark with 3 simple real-world test cases and one additional academic test. The tests I found on the web have been mostly usesless, especially if you are programming games. These are the ones I used:
- Sorting a huge array (5 mio doubles) using Heapsort, testing usual, random memory access quite well.
- Calling multiple nested loops (4), each one 280 times, computing a number in the inner loop based on the loop counters.
- Copying an array (8 MB of doubles) into another one, multiple times (100 times).
- Calculating 44 fibonacci numbers, testing recursive calls. Totally useless test for real world programs, but because every benchmark seems to use it, it was interesting to include it as well.
I tried to be very fair because I really needed realistic results to base my decision on. You can get the source code of my tests here to verify or review my tests here:
benchmark.zip. If you like, extend it or use it for whatever you like.
The results
To make it short: As I expected C++ outperforms Java in every single test. But interestingly Java is getting closer now, here are my average results:
| Java | C++ |
Fibonacci | 10767 ms | 7468 ms |
Heapsort | 5454 ms | 4344 ms |
NestedLoops | 8022 ms | 3453 ms |
CopyData | 1204 ms | 1125 ms |
Details and Findings
Knowing a bit about the C++ and Java optimizers, I was also able to revert the results completely. For example although using equivalent Java and C++ code, I was able to make the C++ version take 16 seconds for the nested loops-test while Java only needed 1 second by adding some virtual method calls which I knew Java could optimize. I also was able to make the C++ version run in only 78 ms while the equivalent Java version needed even 10 seconds by only using static loops - the C++ optimizer then would not even loop anything and just add the values together. This is partially what some of the wrong benchmarks out there are doing - even if not intentional. If you take a look at the sourcecode of my benchmark, you'll also see that I only measured the actual calculations, I was not interested in the start up times at all. This would have been very unfair because java needs some more time to start up, and it doesn't matter at all if an end user program (especially game) needs some extra seconds in the beginning. That's also the reason I used the client and not the server java VM: No end user is using the server VM at all. Another detail was interesting: Java was running out of heap memory very fast. I even had to reduce the sizes of the arrays in the tests, Java wasn't even able to allocate two arrays of several million doubles. Of course this wasn't a problem on the C++ side at all.
My conclusion: Java is not as fast as C++ although asserted by several people. Simply as that. But with today's incredible hardware, nothing will prevent you from writing simple games with it, unless you are not trying to create a fast Software Rasterizer or similar.