Prev/Next Blog
by date

Torque: C# and Java on Framework.NET, Mono, JSEE

Name:	Alexander "taualex" Gaevoy
Date Posted:	Apr 14, 2006
Rating:	Not Rated
Public:	YES
Comments:	YES
RSS Feed:	or Subscribe with .
Profile Page:	View profile page for Alexander "taualex" Gaevoy

Blog post

Howdy

After recent discussions about using C# or Java with Torque with Jason Swearingen and Josh Ritter, I just decided to do some research and benchmark all those platforms.
The test code was small, so memory and the swapping were avoided; the C# code was created and compiled in VS2003 and VS2005, then run on FW.net v1.1 and v2.0, then the same code was ported to Java and compiled with Sun JDK 1.4. I also compiled the class file with ikvmc.mono into native executable, and it runs(!).
All the tests were executed on AMD 3700+/1Gb/SATAII(3Gb/s) under Win XP Pro (I didnt try it yet on Linux or Mac yet); Mono v1.1.14, JDK 1.4, FW v1.1, FW v2.0.
Java code compiled with JDK 1.5 doesnt run on Mono at'all.

Here're the execution time (average time after 3 runs):

-= C# =-
Executable compiled under FW.net v1.1:
.NET v1.1  - 12.484 ms (used heap - 4180 Kb)
Mono       - 35.358 ms (used heap - 6044 Kb)

Executable compiled under FW.net v2.0:
.NET v2.0  - 20.984 ms (used heap - 4144 Kb)
Mono       - 34.8 ms (used heap - 6628 Kb)

-= Java =-
JAR compiled under JDK 1.4:
java (1.4)  - 15.610 ms (used heap - 6196 Kb) 
mono (ikvm) - 50.782 ms (used heap - 27024 Kb)

EXE compiled with ikvmc from JAR:
native      - 15.890 ms (used heap - 11276 Kb)

Well...
Interesting facts:
* Mono still lacks the perfomance of FW.net under Windows (worse time: ~50% at min, ~110% at max).
* .NET v1.1 is faster then v2.0 (but may change after some updates from M$) on simple computation tasks (bloated vtables?).
* MONO executes .NET v2.0 code faster then v1.1 (was targeted, I guess)
* IKVM can compile java code into native and it runs as fast as JDK compiled JAR on WIndows, but uses more heap.
* FW.net uses less heap then Mono, but Mono startup time is less.
* Java is faster then FW.net v2.0, but slower FW.net v1.1 on Windows.

I dont have the test sources with me, but they are really simple:

start_time = current_time();
for (i = 0; i < int.MaxValue(); i++)
{
   r *= i + 1;
}
total_time = current_time() - start_time;

int is equal on all the tested platforms, 32 bits.
Any thoughts?

Recent Blog Posts

List:	05/26/07 - Game Devs, show off your game! 12/12/06 - Why I'm not moving to TGE 1.5 10/19/06 - Battlefield 2142 In-game Ads Use Spyware Tactics 04/14/06 - Torque: C# and Java on Framework.NET, Mono, JSEE 03/09/06 - DirectX - is there any light in the tunnel?.. 03/05/06 - Frustrations... 12/19/04 - Plan for Alexander "taualex" Gaevoy

Submit your own resources!

Phil Carlisle (Apr 14, 2006 at 19:02 GMT)

I dont think your test regime is really possible to draw many conclusions from. You'll have to make a much broader codebase to really do this kind of comparison, basing the results off your current test would be on very shaky ground IMHO because they might have optimised for that particular case.

you need more coverage.

Joe Rossi (Apr 14, 2006 at 19:43 GMT)

You can try some tests from here, and look at what the other languages are benching :
http://shootout.alioth.debian.org/

Alexander "taualex" Gaevoy (Apr 15, 2006 at 22:30 GMT)

@Phil: you have a good point, on the other hand, such test shows how fast VM on a simple task (just crunching numbers, no function calls, no disk I/O); I proved (atleast for myself) that the current version of mono is slower then FW.net, and the difference between fw.net and java is not that big. I will definetly continue the research to cover more cases, it's just the first stage ;).

@Joe: thank for the link! though, I've seen it before, I wanted to make test myself, and, in average, it follows the results from that site.

You must be a member and be logged in to either append comments or rate this resource.