How much does Java impact performance?

Last modified

There has been much debate about the performance tradeoffs of Java as opposed to C or C++. My company, Mathsoft (http://www.mathsoft.com), asked me to research how much of a performance penalty we pay by using Java as opposed to C. So I found a program that was representative of the kind of work we do (scientific data processing) that was simple enough to translate by hand from C to Java and wrote C and Java wrappers around it. I also built a wrapper in Java that called the C subroutines using JNI. Finally, I ran this on a Sun Ultra 60 using JDK 1.1.7 and JDK 1.2 and collected the data, which I publish here.

The algorithm

The algorithm I chose is a simple radix-2 Fast Fourier Transform (FFT), which is a fairly common scientific subroutine. The subroutine makes no system calls: this test is solely a measure of the compilers' ability to generate efficient byte codes and the ability of the JVM to execute byte codes. However, different JVMs have different levels of performance.

Implementations

The algorithm was implemented in 3 similar ways: the original C program was slightly modified to develop a measure of how long it takes to make a subroutine call ("pure_c"); the program was translated into Java trying to retain as much of the flavor of the original as possible ("pure_java"); and a Java program which calls a "glue" subroutine which is written in C and which calls the FFT subroutines ("mixed_java").

Optimization

The FFT works with complex numbers. There are many ways to implement complex numbers, and how they are implemented affects software performance. Interestingly enough, what is more optimal in C is sometimes less optimal than Java, so an important general lesson is you have to measure before you can optimize.
The C program as written passes a vector of structs to the subroutine. The Java translation passes a vector of class Complex to the subroutine. The JNI version also passes a vector of class Complex to the subroutine. However, JNI has a primitive which will convert an entire array of primitive data types; and it has a primitive to convert an object into a struct; but there is no primitive to convert an array of objects into an array of structs. So, I rewrote the programs and the subroutines to work with 2 arrays of doubles instead of an array of structs or an array of objects.

JVMs

There has been much chatter on the usenet about whether the JDK 1.2 JVMs are faster or slower than the JDK 1.1 JVMs. The FFT is a terrific test of the combination of the compiler and the JVM. In retrospect, I should have tested the compiler and the JVM separately. A 1.2 JVM will run a 1.1.x class file with narry a peep; however, a 1.1.x JVM will run a 1.2 class file only if the programmer is very careful.
For this study, I tested with the Sun JDKs. Given more time, I propose to test with additional JDKs from other vendors.

The Results

There are 2 JDKs; optimized or unoptimized; 3 variations, and 12 sizes or 144 cases in all (although some cases didn't run).

The first question I wanted to answer is what is the effect of selection of JDK on performance? Does it make sense to put up with the trials of running JDK 1.2?

Bits Mixed 1.1.7B Pure Java 1.1.7B Mixed 1.2 Pure Java 1.2 C

In retrospect, the Sun is too fast

SunOS orange 5.6 Generic_105181-09 sun4u sparc SUNW,Ultra-60...Wed Aug 18 14:27:57 PDT 1999
user.language = en
java.home = /sw/java/jdk1.1.7B/bin/..
java.vendor.url.bug = http://java.sun.com/cgi-bin/bugreport.cgi
file.encoding.pkg = sun.io
java.version = 1.1.7B
file.separator = /
line.separator =

file.encoding = 8859_1
java.vendor = Sun Microsystems Inc.
user.timezone = PST
user.name = jeffs
os.arch = sparc
os.name = Solaris
java.vendor.url = http://www.sun.com/
user.dir = /a/homer/users/jeffs/jniTest/trix/simple-fft-1.5
java.class.path =
.:/sw/java/jdk1.1.7B/bin/../classes:/sw/java/jdk1.1.7B/bin/../lib/classes.jar:/sw/java/jdk1.1.7B/bin/../lib/rt.jar:/sw/java/jdk1.1.7B/bin/../lib/i18n.jar:/sw/java/jdk1.1.7B/bin/../lib/classes.zip
java.class.version = 45.3
os.version = 2.x
path.separator = :
user.home = /homes/jeffs
Unoptimized Pure_C . 4 bits 0 milliseconds
Unoptimized Pure_C . 5 bits 0 milliseconds
Unoptimized Pure_C . 6 bits 0 milliseconds
Unoptimized Pure_C . 8 bits 0 milliseconds
Unoptimized Pure_C . 10 bits 0 milliseconds
Unoptimized Pure_C . 12 bits 4 milliseconds
Unoptimized Pure_C . 13 bits 9 milliseconds
Unoptimized Pure_C . 14 bits 22 milliseconds
Unoptimized Pure_C . 15 bits 46 milliseconds
Unoptimized Pure_C . 16 bits 107 milliseconds
Unoptimized Pure_C . 17 bits 267 milliseconds
Unoptimized Pure_C . 18 bits 595 milliseconds
Optimized Pure_C . 4 bits 0 milliseconds
Optimized Pure_C . 5 bits 0 milliseconds
Optimized Pure_C . 6 bits 0 milliseconds
Optimized Pure_C . 8 bits 0 milliseconds
Optimized Pure_C . 10 bits 1 milliseconds
Optimized Pure_C . 12 bits 5 milliseconds
Optimized Pure_C . 13 bits 11 milliseconds
Optimized Pure_C . 14 bits 25 milliseconds
Optimized Pure_C . 15 bits 58 milliseconds
Optimized Pure_C . 16 bits 134 milliseconds
Optimized Pure_C . 17 bits 314 milliseconds
Optimized Pure_C . 18 bits 712 milliseconds
Unoptimized Mixed_Java 1.1.7B. 4 bits 0 milliseconds
Unoptimized Mixed_Java 1.1.7B. 5 bits 0 milliseconds
Unoptimized Mixed_Java 1.1.7B. 6 bits 1 milliseconds
Unoptimized Mixed_Java 1.1.7B. 8 bits 3 milliseconds
Unoptimized Mixed_Java 1.1.7B. 10 bits 27 milliseconds
Unoptimized Mixed_Java 1.1.7B. 12 bits 546 milliseconds
Unoptimized Mixed_Java 1.1.7B. 13 bits 2281 milliseconds
Unoptimized Mixed_Java 1.1.7B. 14 bits 9428 milliseconds
Unoptimized Mixed_Java 1.1.7B. 15 bits 47414 milliseconds
Unoptimized Mixed_Java 1.1.7B. 16 bits 186131 milliseconds
Optimized Mixed_Java 1.1.7B. 4 bits 0 milliseconds
Optimized Mixed_Java 1.1.7B. 5 bits 0 milliseconds
Optimized Mixed_Java 1.1.7B. 6 bits 0 milliseconds
Optimized Mixed_Java 1.1.7B. 8 bits 0 milliseconds
Optimized Mixed_Java 1.1.7B. 10 bits 1 milliseconds
Optimized Mixed_Java 1.1.7B. 12 bits 5 milliseconds
Optimized Mixed_Java 1.1.7B. 13 bits 12 milliseconds
Optimized Mixed_Java 1.1.7B. 14 bits 24 milliseconds
Optimized Mixed_Java 1.1.7B. 15 bits 52 milliseconds
Optimized Mixed_Java 1.1.7B. 16 bits 115 milliseconds
Unoptimized Pure_Java 1.1.7B. 4 bits 0 milliseconds
Unoptimized Pure_Java 1.1.7B. 5 bits 1 milliseconds
Unoptimized Pure_Java 1.1.7B. 6 bits 2 milliseconds
Unoptimized Pure_Java 1.1.7B. 8 bits 9 milliseconds
Unoptimized Pure_Java 1.1.7B. 10 bits 42 milliseconds
Unoptimized Pure_Java 1.1.7B. 12 bits 199 milliseconds
Unoptimized Pure_Java 1.1.7B. 13 bits 432 milliseconds
Unoptimized Pure_Java 1.1.7B. 14 bits 936 milliseconds
Unoptimized Pure_Java 1.1.7B. 15 bits 2010 milliseconds
Unoptimized Pure_Java 1.1.7B. 16 bits 4317 milliseconds
Unoptimized Pure_Java 1.1.7B. 17 bits 9166 milliseconds
Optimized Pure Java 1.1.7B 4 bits 0 milliseconds
Optimized Pure Java 1.1.7B 5 bits 1 milliseconds
Optimized Pure Java 1.1.7B 6 bits 1 milliseconds
Optimized Pure Java 1.1.7B 8 bits 5 milliseconds
Optimized Pure Java 1.1.7B 10 bits 23 milliseconds
Optimized Pure Java 1.1.7B 12 bits 110 milliseconds
Optimized Pure Java 1.1.7B 13 bits 233 milliseconds
Optimized Pure Java 1.1.7B 14 bits 507 milliseconds
Optimized Pure Java 1.1.7B 15 bits 1064 milliseconds
Optimized Pure Java 1.1.7B 16 bits 2405 milliseconds
Optimized Pure Java 1.1.7B 17 bits 5074 milliseconds

SunOS orange 5.6 Generic_105181-09 sun4u sparc SUNW,Ultra-60...Wed Aug 18 14:19:49 PDT 1999
java.specification.name = Java Platform API Specification
java.version = 1.2
java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
user.timezone = America/Los_Angeles
java.specification.version = 1.2
java.vm.vendor = Sun Microsystems Inc.
java.vm.specification.version = 1.0
user.home = /homes/jeffs
os.arch = sparc
java.awt.fonts =
java.vendor.url = http://java.sun.com/
file.encoding.pkg = sun.io
java.home = /a/plum/opt2/java/jdk1.2/jre
java.class.path = .
line.separator =

java.ext.dirs = /a/plum/opt2/java/jdk1.2/jre/lib/ext
java.io.tmpdir = /var/tmp/
os.name = SunOS
java.vendor = Sun Microsystems Inc.
java.awt.printerjob = sun.awt.motif.PSPrinterJob
java.library.path =
/sw/java/jdk1.2/bin/../jre/lib/sparc/green_threads:/sw/java/jdk1.2/bin/../jre/lib/sparc/classic:/sw/java/jdk1.2/bin/../jre/lib/sparc:/usr/local/lib:/usr/openwin/lib:/usr/lib:.
java.vm.specification.vendor = Sun Microsystems Inc.
sun.io.unicode.encoding = UnicodeBig
file.encoding = 8859_1
java.specification.vendor = Sun Microsystems Inc.
user.name = jeffs
user.language = en
java.vendor.url.bug = http://java.sun.com/cgi-bin/bugreport.cgi
java.vm.name = Classic VM
java.vm.specification.name = Java Virtual Machine Specification
java.class.version = 46.0
sun.boot.library.path = /a/plum/opt2/java/jdk1.2/jre/lib/sparc
os.version = 5.6
java.vm.info = build JDK-1.2-V, green threads, sunwjit
java.vm.version = 1.2
java.compiler = sunwjit
path.separator = :
user.dir = /a/homer/users/jeffs/jniTest/trix/simple-fft-1.5
file.separator = /
sun.boot.class.path =
/a/plum/opt2/java/jdk1.2/jre/lib/rt.jar:/a/plum/opt2/java/jdk1.2/jre/lib/i18n.jar:/a/plum/opt2/java/jdk1.2/jre/classes
Unoptimized Pure_C . 4 bits 0 milliseconds
Unoptimized Pure_C . 5 bits 0 milliseconds
Unoptimized Pure_C . 6 bits 0 milliseconds
Unoptimized Pure_C . 8 bits 0 milliseconds
Unoptimized Pure_C . 10 bits 0 milliseconds
Unoptimized Pure_C . 12 bits 3 milliseconds
Unoptimized Pure_C . 13 bits 9 milliseconds
Unoptimized Pure_C . 14 bits 20 milliseconds
Unoptimized Pure_C . 15 bits 42 milliseconds
Unoptimized Pure_C . 16 bits 98 milliseconds
Unoptimized Pure_C . 17 bits 241 milliseconds
Unoptimized Pure_C . 18 bits 539 milliseconds
Optimized Pure_C . 4 bits 0 milliseconds
Optimized Pure_C . 5 bits 0 milliseconds
Optimized Pure_C . 6 bits 0 milliseconds
Optimized Pure_C . 8 bits 0 milliseconds
Optimized Pure_C . 10 bits 1 milliseconds
Optimized Pure_C . 12 bits 5 milliseconds
Optimized Pure_C . 13 bits 11 milliseconds
Optimized Pure_C . 14 bits 25 milliseconds
Optimized Pure_C . 15 bits 58 milliseconds
Optimized Pure_C . 16 bits 133 milliseconds
Optimized Pure_C . 17 bits 320 milliseconds
Optimized Pure_C . 18 bits 652 milliseconds
Unoptimized Mixed_Java 1.2. 4 bits 1 milliseconds
Unoptimized Mixed_Java 1.2. 5 bits 0 milliseconds
Unoptimized Mixed_Java 1.2. 6 bits 0 milliseconds
Unoptimized Mixed_Java 1.2. 8 bits 1 milliseconds
Unoptimized Mixed_Java 1.2. 10 bits 3 milliseconds
Unoptimized Mixed_Java 1.2. 12 bits 10 milliseconds
Unoptimized Mixed_Java 1.2. 13 bits 21 milliseconds
Unoptimized Mixed_Java 1.2. 14 bits 44 milliseconds
Unoptimized Mixed_Java 1.2. 15 bits 90 milliseconds
Optimized Mixed_Java 1.2. 4 bits 0 milliseconds
Optimized Mixed_Java 1.2. 5 bits 0 milliseconds
Optimized Mixed_Java 1.2. 6 bits 0 milliseconds
Optimized Mixed_Java 1.2. 8 bits 0 milliseconds
Optimized Mixed_Java 1.2. 10 bits 1 milliseconds
Optimized Mixed_Java 1.2. 12 bits 5 milliseconds
Optimized Mixed_Java 1.2. 13 bits 13 milliseconds
Optimized Mixed_Java 1.2. 14 bits 25 milliseconds
Optimized Mixed_Java 1.2. 15 bits 59 milliseconds
Optimized Mixed_Java 1.2. 16 bits 146 milliseconds
Unoptimized Pure_Java 1.2. 4 bits 1 milliseconds
Unoptimized Pure_Java 1.2. 5 bits 1 milliseconds
Unoptimized Pure_Java 1.2. 6 bits 1 milliseconds
Unoptimized Pure_Java 1.2. 8 bits 3 milliseconds
Unoptimized Pure_Java 1.2. 10 bits 11 milliseconds
Unoptimized Pure_Java 1.2. 12 bits 48 milliseconds
Unoptimized Pure_Java 1.2. 13 bits 108 milliseconds
Unoptimized Pure_Java 1.2. 14 bits 229 milliseconds
Unoptimized Pure_Java 1.2. 15 bits 501 milliseconds
Unoptimized Pure_Java 1.2. 16 bits 1104 milliseconds
Unoptimized Pure_Java 1.2. 17 bits 2430 milliseconds
Optimized Pure Java 1.2 4 bits 0 milliseconds
Optimized Pure Java 1.2 5 bits 0 milliseconds
Optimized Pure Java 1.2 6 bits 0 milliseconds
Optimized Pure Java 1.2 8 bits 1 milliseconds
Optimized Pure Java 1.2 10 bits 2 milliseconds
Optimized Pure Java 1.2 12 bits 10 milliseconds
Optimized Pure Java 1.2 13 bits 22 milliseconds
Optimized Pure Java 1.2 14 bits 45 milliseconds
Optimized Pure Java 1.2 15 bits 100 milliseconds
Optimized Pure Java 1.2 16 bits 239 milliseconds
Optimized Pure Java 1.2 17 bits 518 milliseconds