The Lisp Performance Page

The Lisp Performance Page

1. Preface

Execution times are recorded by timing benchmark programs twice, with and without calling the entry function run. The difference in time reflects the raw execution time of the benchmark and eliminates implementation differences in program invocation that should be timed and discussed separately (e.g. initialisation, program loading, program preprocessing or even compilation). See these Perl scripts for more detail: run.pl and analyse.pl.

In the case of language implementations that do not provide facilities to create stand-alone executables for a native or virtual machine, but are to be used interactively via a read-eval-print loop, the explained differential way of timing is carried out by subtracting the time it takes to load and immediately run the benchmark program from the time it takes to only load the program.

The actual timing is provided by the Unix time function uniformly for all language implementations and benchmark programs.

Benchmarking is not yet completed. Slot access, method invocation and real programs are added soon. The total scoring should not be taken too seriously. The ranking scheme is arguable and we don't consider that programmers typically bias their way of programming towards the individual performance profile of their machine and implementation.

Please, send comments to ak1@maths.bath.ac.uk.

2. Platform mips


Program: arith0 Stress: integer arithmetic Platform: mips

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  CMU CL      CLtL2     native compilation       0.58     1.000000	arith0.lisp
2  Bigloo      Scheme    C compilation            0.65     1.120690	arith0.sch
3  ILOG Talk   ISLisp    C compilation            1.95     3.362069	arith0.t
4  CSL         Std Lisp  bytecode interpretation  2.41     4.155172	arith0.csl
5  Scheme 48   Scheme    bytecode interpretation  4.58     7.896552	arith0.sch
6  youtoo      EuLisp    bytecode interpretation  4.85     8.362069	arith01.em
7  Bigloo      Scheme    direct interpretation    12.02    20.724138	arith0.sch
8  Oaklisp     Scheme    bytecode interpretation  12.5     21.551724	arith0.sch
9  CLisp       CLtL1     bytecode interpretation  13.47    23.224138	arith0.lisp
10 VScheme     Scheme    bytecode interpretation  15.16    26.137931	arith0.sch
11 Euscheme    EuLisp    bytecode interpretation  15.62    26.931034	arith0.em
12 GCL         CLtL1     C compilation            17.17    29.603448	arith0.lisp
13 MIT Scheme  Scheme    direct interpretation    20.54    35.413793	arith0.sch


Program: arith1 Stress: float arithmetic Platform: mips

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Bigloo      Scheme    C compilation            2.06     1.000000	arith1.sch
2  CMU CL      CLtL2     native compilation       2.72     1.320388	arith1.lisp
3  CSL         Std Lisp  bytecode interpretation  6.12     2.970874	arith1.csl
4  VScheme     Scheme    bytecode interpretation  6.84     3.320388	arith1.sch
5  youtoo      EuLisp    bytecode interpretation  13.35    6.480583	arith11.em
6  CLisp       CLtL1     bytecode interpretation  15.89    7.713592	arith1.lisp
7  GCL         CLtL1     C compilation            15.98    7.757282	arith1.lisp
8  MIT Scheme  Scheme    direct interpretation    19.15    9.296117	arith1.sch
9  Bigloo      Scheme    direct interpretation    26.4     12.815534	arith1.sch
10 Euscheme    EuLisp    bytecode interpretation  37.98    18.436893	arith1.em
11 Oaklisp     Scheme    bytecode interpretation  72.35    35.121359	arith1.sch
12 Scheme 48   Scheme    bytecode interpretation  1306.11  634.033981	arith1.sch
13 ILOG Talk   ISLisp    C compilation            ???      0.000000	arith1.t


Program: vec Stress: vector access Platform: mips

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Bigloo      Scheme    C compilation            0.27     1.000000	vec.sch
2  GCL         CLtL1     C compilation            0.33     1.222222	vec.lisp
3  CMU CL      CLtL2     native compilation       0.36     1.333333	vec.lisp
4  youtoo      EuLisp    bytecode interpretation  0.82     3.037037	vec1.em
5  CLisp       CLtL1     bytecode interpretation  1.37     5.074074	vec.lisp
6  Scheme 48   Scheme    bytecode interpretation  1.5      5.555556	vec.sch
7  CSL         Std Lisp  bytecode interpretation  1.5      5.555556	vec.csl
8  ILOG Talk   ISLisp    C compilation            1.86     6.888889	vec.t
9  VScheme     Scheme    bytecode interpretation  2.3      8.518519	vec.sch
10 Oaklisp     Scheme    bytecode interpretation  3.67     13.592593	vec.sch
11 Bigloo      Scheme    direct interpretation    4.54     16.814815	vec.sch
12 MIT Scheme  Scheme    direct interpretation    5.47     20.259259	vec.sch
13 Euscheme    EuLisp    bytecode interpretation  11.77    43.592593	vec.em


Program: rec Stress: recursion Platform: mips

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  CMU CL      CLtL2     native compilation       0.85     1.000000	rec.lisp
2  Bigloo      Scheme    C compilation            1.78     2.094118	rec.sch
3  GCL         CLtL1     C compilation            4.11     4.835294	rec.lisp
4  youtoo      EuLisp    bytecode interpretation  6.1      7.176471	rec1.em
5  Oaklisp     Scheme    bytecode interpretation  7.81     9.188235	rec.sch
6  Euscheme    EuLisp    bytecode interpretation  9.19     10.811765	rec.em
7  Scheme 48   Scheme    bytecode interpretation  12.3     14.470588	rec.sch
8  CSL         Std Lisp  bytecode interpretation  12.63    14.858824	rec.csl
9  ILOG Talk   ISLisp    C compilation            15.42    18.141176	rec.t
10 CLisp       CLtL1     bytecode interpretation  16.18    19.035294	rec.lisp
11 VScheme     Scheme    bytecode interpretation  23.74    27.929412	rec.sch
12 Bigloo      Scheme    direct interpretation    26.82    31.552941	rec.sch
13 MIT Scheme  Scheme    direct interpretation    37.35    43.941176	rec.sch


Program: mem Stress: memory management Platform: mips

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  CMU CL      CLtL2     native compilation       0.85     1.000000	mem.lisp
2  Bigloo      Scheme    C compilation            9.79     11.517647	mem.sch
3  VScheme     Scheme    bytecode interpretation  14.02    16.494118	mem.sch
4  CSL         Std Lisp  bytecode interpretation  21.47    25.258824	mem.csl
5  ILOG Talk   ISLisp    C compilation            27.9     32.823529	mem.t
6  Scheme 48   Scheme    bytecode interpretation  33.8     39.764706	mem.sch
7  Oaklisp     Scheme    bytecode interpretation  35.64    41.929412	mem.sch
8  Bigloo      Scheme    direct interpretation    36.45    42.882353	mem.sch
9  youtoo      EuLisp    bytecode interpretation  45.84    53.929412	mem1.em
10 MIT Scheme  Scheme    direct interpretation    52.42    61.670588	mem.sch
11 CLisp       CLtL1     bytecode interpretation  101.87   119.847059	mem.lisp
12 GCL         CLtL1     C compilation            446.54   525.341176	mem.lisp
13 Euscheme    EuLisp    bytecode interpretation  1113.96  1310.541176	mem.em


Program: div Stress: list processing, recursion, iteration Platform: mips

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  CMU CL      CLtL2     native compilation       0.2      1.000000	div.lisp
2  Bigloo      Scheme    C compilation            0.4      2.000000	div.sch
3  GCL         CLtL1     C compilation            1.31     6.550000	div.lisp
4  CSL         Std Lisp  bytecode interpretation  1.35     6.750000	div.csl
5  Oaklisp     Scheme    bytecode interpretation  1.5      7.500000	div.sch
6  CLisp       CLtL1     bytecode interpretation  1.81     9.050000	div.lisp
7  VScheme     Scheme    bytecode interpretation  2.4      12.000000	div.sch
8  youtoo      EuLisp    bytecode interpretation  2.53     12.650000	div1.em
9  Bigloo      Scheme    direct interpretation    4.57     22.850000	div.sch
10 Scheme 48   Scheme    bytecode interpretation  4.61     23.050000	div.sch
11 ILOG Talk   ISLisp    C compilation            5.61     28.050000	div.t
12 MIT Scheme  Scheme    direct interpretation    7.38     36.900000	div.sch
13 Euscheme    EuLisp    bytecode interpretation  9.29     46.450000	div.em


Program: tak Stress: recursion, integer arithmetic Platform: mips

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  CMU CL      CLtL2     native compilation       0.04     1.000000	tak.lisp
2  Bigloo      Scheme    C compilation            0.06     1.500000	tak.sch
3  GCL         CLtL1     C compilation            0.2      5.000000	tak.lisp
4  CSL         Std Lisp  bytecode interpretation  0.37     9.250000	tak.csl
5  Oaklisp     Scheme    bytecode interpretation  0.37     9.250000	tak.sch
6  youtoo      EuLisp    bytecode interpretation  0.39     9.750000	tak1.em
7  CLisp       CLtL1     bytecode interpretation  0.4      10.000000	tak.lisp
8  Scheme 48   Scheme    bytecode interpretation  0.84     21.000000	tak.sch
9  ILOG Talk   ISLisp    C compilation            1.01     25.250000	tak.t
10 Bigloo      Scheme    direct interpretation    1.36     34.000000	tak.sch
11 VScheme     Scheme    bytecode interpretation  1.53     38.250000	tak.sch
12 Euscheme    EuLisp    bytecode interpretation  2.59     64.750000	tak.em
13 MIT Scheme  Scheme    direct interpretation    2.86     71.500000	tak.sch


Program: takl Stress: list processing, recursion Platform: mips

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  CMU CL      CLtL2     native compilation       0.12     1.000000	takl.lisp
2  Bigloo      Scheme    C compilation            0.26     2.166667	takl.sch
3  GCL         CLtL1     C compilation            0.34     2.833333	takl.lisp
4  CLisp       CLtL1     bytecode interpretation  1.92     16.000000	takl.lisp
5  youtoo      EuLisp    bytecode interpretation  2.25     18.750000	takl1.em
6  CSL         Std Lisp  bytecode interpretation  2.42     20.166667	takl.csl
7  Oaklisp     Scheme    bytecode interpretation  3.08     25.666667	takl.sch
8  VScheme     Scheme    bytecode interpretation  4.37     36.416667	takl.sch
9  ILOG Talk   ISLisp    C compilation            5.08     42.333333	takl.t
10 Scheme 48   Scheme    bytecode interpretation  9.92     82.666667	takl.sch
11 Bigloo      Scheme    direct interpretation    13.32    111.000000	takl.sch
12 MIT Scheme  Scheme    direct interpretation    14.07    117.250000	takl.sch
13 Euscheme    EuLisp    bytecode interpretation  16.75    139.583333	takl.em


Program: takr Stress: hardware caching, recursion, integer arithmetic Platform: mips

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Bigloo      Scheme    C compilation            0.12     1.000000	takr.sch
2  CMU CL      CLtL2     native compilation       0.15     1.250000	takr.lisp
3  Oaklisp     Scheme    bytecode interpretation  0.4      3.333333	takr.sch
4  youtoo      EuLisp    bytecode interpretation  0.42     3.500000	takr1.em
5  CSL         Std Lisp  bytecode interpretation  0.45     3.750000	takr.csl
6  GCL         CLtL1     C compilation            0.51     4.250000	takr.lisp
7  CLisp       CLtL1     bytecode interpretation  0.55     4.583333	takr.lisp
8  Scheme 48   Scheme    bytecode interpretation  0.95     7.916667	takr.sch
9  VScheme     Scheme    bytecode interpretation  1.68     14.000000	takr.sch
10 Bigloo      Scheme    direct interpretation    1.72     14.333333	takr.sch
11 ILOG Talk   ISLisp    C compilation            1.94     16.166667	takr.t
12 MIT Scheme  Scheme    direct interpretation    3.44     28.666667	takr.sch
13 Euscheme    EuLisp    bytecode interpretation  3.81     31.750000	takr.em


Program: nfib Stress: recursion, integer arithmetic Platform: mips

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  CMU CL      CLtL2     native compilation       0.07     1.000000	nfib.lisp
2  Bigloo      Scheme    C compilation            0.13     1.857143	nfib.sch
3  youtoo      EuLisp    bytecode interpretation  0.52     7.428571	nfib1.em
4  CSL         Std Lisp  bytecode interpretation  0.56     8.000000	nfib.csl
5  Oaklisp     Scheme    bytecode interpretation  0.59     8.428571	nfib.sch
6  GCL         CLtL1     C compilation            0.72     10.285714	nfib.lisp
7  CLisp       CLtL1     bytecode interpretation  1.33     19.000000	nfib.lisp
8  ILOG Talk   ISLisp    C compilation            2        28.571429	nfib.t
9  Bigloo      Scheme    direct interpretation    2.06     29.428571	nfib.sch
10 VScheme     Scheme    bytecode interpretation  3.08     44.000000	nfib.sch
11 MIT Scheme  Scheme    direct interpretation    4.1      58.571429	nfib.sch
12 Scheme 48   Scheme    bytecode interpretation  7.8      111.428571	nfib.sch
13 Euscheme    EuLisp    bytecode interpretation  15.02    214.571429	nfib.em


Total on Platform mips

   System      Dialect   Type                         Score
                                                points     relative
1  CMU CL      CLtL2     native compilation       14       1.000000
2  Bigloo      Scheme    C compilation            17       1.214286
3  CSL         Std Lisp  bytecode interpretation  49       3.500000
4  youtoo      EuLisp    bytecode interpretation  54       3.857143
5  GCL         CLtL1     C compilation            57       4.071429
6  Oaklisp     Scheme    bytecode interpretation  66       4.714286
7  CLisp       CLtL1     bytecode interpretation  72       5.142857
8  VScheme     Scheme    bytecode interpretation  82       5.857143
9  Scheme 48   Scheme    bytecode interpretation  84       6.000000
10 ILOG Talk   ISLisp    C compilation            86       6.142857
11 Bigloo      Scheme    direct interpretation    96       6.857143
12 MIT Scheme  Scheme    direct interpretation    116      8.285714
13 Euscheme    EuLisp    bytecode interpretation  117      8.357143



3. Platform i586


Program: arith0 Stress: integer arithmetic Platform: i586

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Allegro CL  CLtL2     native compilation       0.87     1.000000	arith0.lisp
2  youtoo      EuLisp    bytecode interpretation  3.82     4.390805	arith01.em


Program: arith1 Stress: float arithmetic Platform: i586

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Allegro CL  CLtL2     native compilation       0.93     1.000000	arith1.lisp
2  youtoo      EuLisp    bytecode interpretation  8.31     8.935484	arith11.em


Program: vec Stress: vector access Platform: i586

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Allegro CL  CLtL2     native compilation       0.22     1.000000	vec.lisp
2  youtoo      EuLisp    bytecode interpretation  0.58     2.636364	vec1.em


Program: rec Stress: recursion Platform: i586

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Allegro CL  CLtL2     native compilation       0.66     1.000000	rec.lisp
2  youtoo      EuLisp    bytecode interpretation  4.87     7.378788	rec1.em


Program: mem Stress: memory management Platform: i586

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Allegro CL  CLtL2     native compilation       19.17    1.000000	mem.lisp
2  youtoo      EuLisp    bytecode interpretation  24.74    1.290558	mem1.em


Program: div Stress: list processing, recursion, iteration Platform: i586

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Allegro CL  CLtL2     native compilation       0.28     1.000000	div.lisp
2  youtoo      EuLisp    bytecode interpretation  1.67     5.964286	div1.em


Program: tak Stress: recursion, integer arithmetic Platform: i586

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Allegro CL  CLtL2     native compilation       0.05     1.000000	tak.lisp
2  youtoo      EuLisp    bytecode interpretation  0.28     5.600000	tak1.em


Program: takl Stress: list processing, recursion Platform: i586

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Allegro CL  CLtL2     native compilation       0.1      1.000000	takl.lisp
2  youtoo      EuLisp    bytecode interpretation  1.62     16.200000	takl1.em


Program: takr Stress: hardware caching, recursion, integer arithmetic Platform: i586

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Allegro CL  CLtL2     native compilation       0.06     1.000000	takr.lisp
2  youtoo      EuLisp    bytecode interpretation  0.31     5.166667	takr1.em


Program: nfib Stress: recursion, integer arithmetic Platform: i586

   System      Dialect   Type                          Time             Program
                                                 usr+sys   relative
1  Allegro CL  CLtL2     native compilation       0.06     1.000000	nfib.lisp
2  youtoo      EuLisp    bytecode interpretation  0.31     5.166667	nfib1.em


Total on Platform i586

   System      Dialect   Type                         Score
                                                points     relative
1  Allegro CL  CLtL2     native compilation       10       1.000000
2  youtoo      EuLisp    bytecode interpretation  20       2.000000



This page was last modified on Mon Jul 21 11:16:34 BST 1997 .