Comparative Performance of Popular Object-Oriented Languages on a Trivial "Business Object" Benchmark

Performance Graph
Hardware: Dell Inspiron 5000e 800Mhz PIII 512MB PC100 RAM
Operating System: FreeBSD 5.0-CURRENT (2000-12-30)
Python: Python 1.5.2, native compilation
Ruby: Ruby 1.6.2, native compilation
Java 1.3: Linux JDK build 1.3.0, Classic VM, green threads, OS emulation
OpenJIT: OpenJIT 1.1.15, semi-native compilation

Intro

Business objects are becoming the standard approach for building enterprise software systems. Typically, a business object encapsulates state which represents a real-world business process or entity, and exposes methods on this state to other business objects or user interfaces. The public methods are responsible for enforcing rules, constraints and invariants ("business logic") which pertain to the encapsulated state.

Motivation

Python, Ruby, and Java are three common object-oriented languages with attractive features for business object development. The biggest shortcoming of all three, however, is that they are based on a "virtual machine" interpreter, which limits execution speed.

The motivation for this test was to determine the relative execution speed of the various languages and virtual machines on a mix of operations representative of business logic.

Method

The "Tower of Hanoi" problem was modeled as "business objects" consisting of members of two classes: Disc and Spindle. The standard recursive solution was redesigned to use methods which map to "real world" implementation. Constraint checking was also included, which, for example, verifies that a disc is not placed on top of a smaller disc, or that only the uppermost disc can be removed from a spindle.

These classes and methods were then implemented as similarly as possible in Python, Ruby, and Java. The user+system time was recorded for each VM with an increasing number of discs until execution time exceeded 100 seconds.

Results

The "Tower of Hanoi" problem runs in O(2n) time. The exponential increase in execution time is well-suited to a log-scale graph, which clearly illustrates VM startup overhead versus sustained execution speed.

The graph shows that both Python and Ruby have very low startup overhead, but also very low sustained execution speed. Ruby is marginally faster than Python in both respects.

The Java "Classic" VM has the lowest startup overhead of the three Java VM's, but is around six times slower in sustained execution. The OpenJIT compiler increases startup time of the Classic VM considerably, but allows the Classic VM to approach the performance of the Hotspot client VM in sustained execution. This is significant because the Hotspot virtual machines require a threaded kernel for proper operation, and are thus not suitable for general use on FreeBSD. The OpenJIT compiler is currently the fastest option available for business object development under FreeBSD.

The Hotspot server VM performs surprisingly well, almost four times faster than the Hotspot client VM, and almost twenty-five times faster than the plain Classic VM in sustained execution.

Numeric results:

Discs

Python

Ruby

Java 1.3

OpenJIT

Hotspot
(client)

Hotspot
(server)

3

0.0230

0.0160

0.3790

1.5950

0.4840

0.5740

4

0.0240

0.0170

0.3770

1.5910

0.4840

0.5770

5

0.0250

0.0190

0.3800

1.5950

0.4850

0.5760

6

0.0300

0.0210

0.3820

1.5950

0.4850

0.5780

7

0.0380

0.0280

0.3810

1.5970

0.4860

0.5780

8

0.0560

0.0410

0.3820

1.5970

0.4890

0.5790

9

0.0940

0.0690

0.3860

1.5990

0.4940

0.5810

10

0.1730

0.1280

0.3910

1.6000

0.4960

0.5870

11

0.3400

0.2510

0.4010

1.6020

0.5060

0.6400

12

0.7020

0.5230

0.4240

1.6040

0.5160

0.6630

13

1.4400

1.0700

0.4710

1.6150

0.5300

0.6780

14

2.8770

2.2270

0.5600

1.6320

0.5330

0.7010

15

5.9750

4.6040

0.7510

1.6620

0.5680

0.7160

16

12.4300

9.5210

1.1500

1.7260

0.6120

0.7310

17

25.8310

19.7040

1.9530

1.8570

0.7280

0.7610

18

53.5760

40.7300

3.6240

2.1250

0.9280

0.8300

19

111.3050

84.1170

7.0570

2.6780

1.3900

0.9860

20

230.4530

173.6720

14.1560

3.8250

2.3390

1.2800

21

--

--

28.7370

6.1580

4.2670

1.8370

22

--

--

58.8030

11.0370

8.3050

2.9130

23

--

--

120.5990

21.1150

16.7290

5.2690

24

--

--

247.0070

42.0320

33.7470

9.9800

25

--

--

--

85.1880

69.1380

19.5090

26

--

--

--

174.3310

141.6330

38.9180

27

--

--

--

--

--

78.2340

28

--

--

--

--

--

157.5720

Conclusions

The highest-performance business object platform (by a factor of 4-5) is not available for FreeBSD. It will not be available until kernel threads are available and stable in FreeBSD. Furthermore, after the arrival of kernel threads, either the Linux ABI layer will have to be upgraded to the level where it can stably run the Linux JDK with server Hotspot, or a fully-debugged native release of the 1.3 server Hotspot compiler will have to be released for FreeBSD.

Benchmark source code

Note: these are not perfect examples of aesthetic purity, nor are they intended to be.