Execution times are recorded by timing benchmark programs twice, with and without calling the entry function run. The difference in time reflects the raw execution time of the benchmark and eliminates implementation differences in program invocation that should be timed and discussed separately (e.g. initialisation, program loading, program preprocessing or even compilation). See these Perl scripts for more detail: run.pl and analyse.pl.
In the case of language implementations that do not provide facilities to create stand-alone executables for a native or virtual machine, but are to be used interactively via a read-eval-print loop, the explained differential way of timing is carried out by subtracting the time it takes to load and immediately run the benchmark program from the time it takes to only load the program.
The actual timing is provided by the Unix time function uniformly for all language implementations and benchmark programs.
Benchmarking is not yet completed. Slot access, method invocation and real programs are added soon. The total scoring should not be taken too seriously. The ranking scheme is arguable and we don't consider that programmers typically bias their way of programming towards the individual performance profile of their machine and implementation.
Please, send comments to ak1@maths.bath.ac.uk.
System Dialect Type Time Program
usr+sys relative
1 CMU CL CLtL2 native compilation 0.58 1.000000 arith0.lisp
2 Bigloo Scheme C compilation 0.65 1.120690 arith0.sch
3 ILOG Talk ISLisp C compilation 1.95 3.362069 arith0.t
4 CSL Std Lisp bytecode interpretation 2.41 4.155172 arith0.csl
5 Scheme 48 Scheme bytecode interpretation 4.58 7.896552 arith0.sch
6 youtoo EuLisp bytecode interpretation 4.85 8.362069 arith01.em
7 Bigloo Scheme direct interpretation 12.02 20.724138 arith0.sch
8 Oaklisp Scheme bytecode interpretation 12.5 21.551724 arith0.sch
9 CLisp CLtL1 bytecode interpretation 13.47 23.224138 arith0.lisp
10 VScheme Scheme bytecode interpretation 15.16 26.137931 arith0.sch
11 Euscheme EuLisp bytecode interpretation 15.62 26.931034 arith0.em
12 GCL CLtL1 C compilation 17.17 29.603448 arith0.lisp
13 MIT Scheme Scheme direct interpretation 20.54 35.413793 arith0.sch
System Dialect Type Time Program
usr+sys relative
1 Bigloo Scheme C compilation 2.06 1.000000 arith1.sch
2 CMU CL CLtL2 native compilation 2.72 1.320388 arith1.lisp
3 CSL Std Lisp bytecode interpretation 6.12 2.970874 arith1.csl
4 VScheme Scheme bytecode interpretation 6.84 3.320388 arith1.sch
5 youtoo EuLisp bytecode interpretation 13.35 6.480583 arith11.em
6 CLisp CLtL1 bytecode interpretation 15.89 7.713592 arith1.lisp
7 GCL CLtL1 C compilation 15.98 7.757282 arith1.lisp
8 MIT Scheme Scheme direct interpretation 19.15 9.296117 arith1.sch
9 Bigloo Scheme direct interpretation 26.4 12.815534 arith1.sch
10 Euscheme EuLisp bytecode interpretation 37.98 18.436893 arith1.em
11 Oaklisp Scheme bytecode interpretation 72.35 35.121359 arith1.sch
12 Scheme 48 Scheme bytecode interpretation 1306.11 634.033981 arith1.sch
13 ILOG Talk ISLisp C compilation ??? 0.000000 arith1.t
System Dialect Type Time Program
usr+sys relative
1 Bigloo Scheme C compilation 0.27 1.000000 vec.sch
2 GCL CLtL1 C compilation 0.33 1.222222 vec.lisp
3 CMU CL CLtL2 native compilation 0.36 1.333333 vec.lisp
4 youtoo EuLisp bytecode interpretation 0.82 3.037037 vec1.em
5 CLisp CLtL1 bytecode interpretation 1.37 5.074074 vec.lisp
6 Scheme 48 Scheme bytecode interpretation 1.5 5.555556 vec.sch
7 CSL Std Lisp bytecode interpretation 1.5 5.555556 vec.csl
8 ILOG Talk ISLisp C compilation 1.86 6.888889 vec.t
9 VScheme Scheme bytecode interpretation 2.3 8.518519 vec.sch
10 Oaklisp Scheme bytecode interpretation 3.67 13.592593 vec.sch
11 Bigloo Scheme direct interpretation 4.54 16.814815 vec.sch
12 MIT Scheme Scheme direct interpretation 5.47 20.259259 vec.sch
13 Euscheme EuLisp bytecode interpretation 11.77 43.592593 vec.em
System Dialect Type Time Program
usr+sys relative
1 CMU CL CLtL2 native compilation 0.85 1.000000 rec.lisp
2 Bigloo Scheme C compilation 1.78 2.094118 rec.sch
3 GCL CLtL1 C compilation 4.11 4.835294 rec.lisp
4 youtoo EuLisp bytecode interpretation 6.1 7.176471 rec1.em
5 Oaklisp Scheme bytecode interpretation 7.81 9.188235 rec.sch
6 Euscheme EuLisp bytecode interpretation 9.19 10.811765 rec.em
7 Scheme 48 Scheme bytecode interpretation 12.3 14.470588 rec.sch
8 CSL Std Lisp bytecode interpretation 12.63 14.858824 rec.csl
9 ILOG Talk ISLisp C compilation 15.42 18.141176 rec.t
10 CLisp CLtL1 bytecode interpretation 16.18 19.035294 rec.lisp
11 VScheme Scheme bytecode interpretation 23.74 27.929412 rec.sch
12 Bigloo Scheme direct interpretation 26.82 31.552941 rec.sch
13 MIT Scheme Scheme direct interpretation 37.35 43.941176 rec.sch
System Dialect Type Time Program
usr+sys relative
1 CMU CL CLtL2 native compilation 0.85 1.000000 mem.lisp
2 Bigloo Scheme C compilation 9.79 11.517647 mem.sch
3 VScheme Scheme bytecode interpretation 14.02 16.494118 mem.sch
4 CSL Std Lisp bytecode interpretation 21.47 25.258824 mem.csl
5 ILOG Talk ISLisp C compilation 27.9 32.823529 mem.t
6 Scheme 48 Scheme bytecode interpretation 33.8 39.764706 mem.sch
7 Oaklisp Scheme bytecode interpretation 35.64 41.929412 mem.sch
8 Bigloo Scheme direct interpretation 36.45 42.882353 mem.sch
9 youtoo EuLisp bytecode interpretation 45.84 53.929412 mem1.em
10 MIT Scheme Scheme direct interpretation 52.42 61.670588 mem.sch
11 CLisp CLtL1 bytecode interpretation 101.87 119.847059 mem.lisp
12 GCL CLtL1 C compilation 446.54 525.341176 mem.lisp
13 Euscheme EuLisp bytecode interpretation 1113.96 1310.541176 mem.em
System Dialect Type Time Program
usr+sys relative
1 CMU CL CLtL2 native compilation 0.2 1.000000 div.lisp
2 Bigloo Scheme C compilation 0.4 2.000000 div.sch
3 GCL CLtL1 C compilation 1.31 6.550000 div.lisp
4 CSL Std Lisp bytecode interpretation 1.35 6.750000 div.csl
5 Oaklisp Scheme bytecode interpretation 1.5 7.500000 div.sch
6 CLisp CLtL1 bytecode interpretation 1.81 9.050000 div.lisp
7 VScheme Scheme bytecode interpretation 2.4 12.000000 div.sch
8 youtoo EuLisp bytecode interpretation 2.53 12.650000 div1.em
9 Bigloo Scheme direct interpretation 4.57 22.850000 div.sch
10 Scheme 48 Scheme bytecode interpretation 4.61 23.050000 div.sch
11 ILOG Talk ISLisp C compilation 5.61 28.050000 div.t
12 MIT Scheme Scheme direct interpretation 7.38 36.900000 div.sch
13 Euscheme EuLisp bytecode interpretation 9.29 46.450000 div.em
System Dialect Type Time Program
usr+sys relative
1 CMU CL CLtL2 native compilation 0.04 1.000000 tak.lisp
2 Bigloo Scheme C compilation 0.06 1.500000 tak.sch
3 GCL CLtL1 C compilation 0.2 5.000000 tak.lisp
4 CSL Std Lisp bytecode interpretation 0.37 9.250000 tak.csl
5 Oaklisp Scheme bytecode interpretation 0.37 9.250000 tak.sch
6 youtoo EuLisp bytecode interpretation 0.39 9.750000 tak1.em
7 CLisp CLtL1 bytecode interpretation 0.4 10.000000 tak.lisp
8 Scheme 48 Scheme bytecode interpretation 0.84 21.000000 tak.sch
9 ILOG Talk ISLisp C compilation 1.01 25.250000 tak.t
10 Bigloo Scheme direct interpretation 1.36 34.000000 tak.sch
11 VScheme Scheme bytecode interpretation 1.53 38.250000 tak.sch
12 Euscheme EuLisp bytecode interpretation 2.59 64.750000 tak.em
13 MIT Scheme Scheme direct interpretation 2.86 71.500000 tak.sch
System Dialect Type Time Program
usr+sys relative
1 CMU CL CLtL2 native compilation 0.12 1.000000 takl.lisp
2 Bigloo Scheme C compilation 0.26 2.166667 takl.sch
3 GCL CLtL1 C compilation 0.34 2.833333 takl.lisp
4 CLisp CLtL1 bytecode interpretation 1.92 16.000000 takl.lisp
5 youtoo EuLisp bytecode interpretation 2.25 18.750000 takl1.em
6 CSL Std Lisp bytecode interpretation 2.42 20.166667 takl.csl
7 Oaklisp Scheme bytecode interpretation 3.08 25.666667 takl.sch
8 VScheme Scheme bytecode interpretation 4.37 36.416667 takl.sch
9 ILOG Talk ISLisp C compilation 5.08 42.333333 takl.t
10 Scheme 48 Scheme bytecode interpretation 9.92 82.666667 takl.sch
11 Bigloo Scheme direct interpretation 13.32 111.000000 takl.sch
12 MIT Scheme Scheme direct interpretation 14.07 117.250000 takl.sch
13 Euscheme EuLisp bytecode interpretation 16.75 139.583333 takl.em
System Dialect Type Time Program
usr+sys relative
1 Bigloo Scheme C compilation 0.12 1.000000 takr.sch
2 CMU CL CLtL2 native compilation 0.15 1.250000 takr.lisp
3 Oaklisp Scheme bytecode interpretation 0.4 3.333333 takr.sch
4 youtoo EuLisp bytecode interpretation 0.42 3.500000 takr1.em
5 CSL Std Lisp bytecode interpretation 0.45 3.750000 takr.csl
6 GCL CLtL1 C compilation 0.51 4.250000 takr.lisp
7 CLisp CLtL1 bytecode interpretation 0.55 4.583333 takr.lisp
8 Scheme 48 Scheme bytecode interpretation 0.95 7.916667 takr.sch
9 VScheme Scheme bytecode interpretation 1.68 14.000000 takr.sch
10 Bigloo Scheme direct interpretation 1.72 14.333333 takr.sch
11 ILOG Talk ISLisp C compilation 1.94 16.166667 takr.t
12 MIT Scheme Scheme direct interpretation 3.44 28.666667 takr.sch
13 Euscheme EuLisp bytecode interpretation 3.81 31.750000 takr.em
System Dialect Type Time Program
usr+sys relative
1 CMU CL CLtL2 native compilation 0.07 1.000000 nfib.lisp
2 Bigloo Scheme C compilation 0.13 1.857143 nfib.sch
3 youtoo EuLisp bytecode interpretation 0.52 7.428571 nfib1.em
4 CSL Std Lisp bytecode interpretation 0.56 8.000000 nfib.csl
5 Oaklisp Scheme bytecode interpretation 0.59 8.428571 nfib.sch
6 GCL CLtL1 C compilation 0.72 10.285714 nfib.lisp
7 CLisp CLtL1 bytecode interpretation 1.33 19.000000 nfib.lisp
8 ILOG Talk ISLisp C compilation 2 28.571429 nfib.t
9 Bigloo Scheme direct interpretation 2.06 29.428571 nfib.sch
10 VScheme Scheme bytecode interpretation 3.08 44.000000 nfib.sch
11 MIT Scheme Scheme direct interpretation 4.1 58.571429 nfib.sch
12 Scheme 48 Scheme bytecode interpretation 7.8 111.428571 nfib.sch
13 Euscheme EuLisp bytecode interpretation 15.02 214.571429 nfib.em
System Dialect Type Score
points relative
1 CMU CL CLtL2 native compilation 14 1.000000
2 Bigloo Scheme C compilation 17 1.214286
3 CSL Std Lisp bytecode interpretation 49 3.500000
4 youtoo EuLisp bytecode interpretation 54 3.857143
5 GCL CLtL1 C compilation 57 4.071429
6 Oaklisp Scheme bytecode interpretation 66 4.714286
7 CLisp CLtL1 bytecode interpretation 72 5.142857
8 VScheme Scheme bytecode interpretation 82 5.857143
9 Scheme 48 Scheme bytecode interpretation 84 6.000000
10 ILOG Talk ISLisp C compilation 86 6.142857
11 Bigloo Scheme direct interpretation 96 6.857143
12 MIT Scheme Scheme direct interpretation 116 8.285714
13 Euscheme EuLisp bytecode interpretation 117 8.357143
System Dialect Type Time Program
usr+sys relative
1 Allegro CL CLtL2 native compilation 0.87 1.000000 arith0.lisp
2 youtoo EuLisp bytecode interpretation 3.82 4.390805 arith01.em
System Dialect Type Time Program
usr+sys relative
1 Allegro CL CLtL2 native compilation 0.93 1.000000 arith1.lisp
2 youtoo EuLisp bytecode interpretation 8.31 8.935484 arith11.em
System Dialect Type Time Program
usr+sys relative
1 Allegro CL CLtL2 native compilation 0.22 1.000000 vec.lisp
2 youtoo EuLisp bytecode interpretation 0.58 2.636364 vec1.em
System Dialect Type Time Program
usr+sys relative
1 Allegro CL CLtL2 native compilation 0.66 1.000000 rec.lisp
2 youtoo EuLisp bytecode interpretation 4.87 7.378788 rec1.em
System Dialect Type Time Program
usr+sys relative
1 Allegro CL CLtL2 native compilation 19.17 1.000000 mem.lisp
2 youtoo EuLisp bytecode interpretation 24.74 1.290558 mem1.em
System Dialect Type Time Program
usr+sys relative
1 Allegro CL CLtL2 native compilation 0.28 1.000000 div.lisp
2 youtoo EuLisp bytecode interpretation 1.67 5.964286 div1.em
System Dialect Type Time Program
usr+sys relative
1 Allegro CL CLtL2 native compilation 0.05 1.000000 tak.lisp
2 youtoo EuLisp bytecode interpretation 0.28 5.600000 tak1.em
System Dialect Type Time Program
usr+sys relative
1 Allegro CL CLtL2 native compilation 0.1 1.000000 takl.lisp
2 youtoo EuLisp bytecode interpretation 1.62 16.200000 takl1.em
System Dialect Type Time Program
usr+sys relative
1 Allegro CL CLtL2 native compilation 0.06 1.000000 takr.lisp
2 youtoo EuLisp bytecode interpretation 0.31 5.166667 takr1.em
System Dialect Type Time Program
usr+sys relative
1 Allegro CL CLtL2 native compilation 0.06 1.000000 nfib.lisp
2 youtoo EuLisp bytecode interpretation 0.31 5.166667 nfib1.em
System Dialect Type Score
points relative
1 Allegro CL CLtL2 native compilation 10 1.000000
2 youtoo EuLisp bytecode interpretation 20 2.000000
This page was last modified on Mon Jul 21 11:16:34 BST 1997 .