25th March 2023, 7 min read

Performance Comparison C vs. Java vs. Javascript vs. PHP vs. Python vs. Cobol vs. Dart

1. Introduction. This post is a continuation of a previous benchmark in Performance Comparison C vs. Java vs. Javascript vs. LuaJIT vs. PyPy vs. PHP vs. Python vs. Perl. Here we compare

1. C
2. Java
3. JavaScript
4. Python
5. Cobol
   • GnuCOBOL
   • gcobol (gcc based COBOL)
6. Dart

I tried to run IBM COBOL as well, but installation as described in Installing IBM COBOL for Linux on Arch Linux failed.

Testing machine is a Ryzen 7 5700G (Cezanne) with 16 CPUs and 4672.0698 MHz max clock. Operating system is Arch Linux 6.2.6-arch1-1. Source code for the programs is in Performance Comparison C vs. Java vs. Javascript vs. LuaJIT vs. PyPy vs. PHP vs. Python vs. Perl.

2. Results. I again use the n-queens problem, i.e., how many times can n queens be put on an n x n chess board without attacking any other queen. Task at hand is to compute from n=1 to n=13 all possible combinations. For example, for C I call time xdamcnt2 13. I ran the programs multiple times and took the lowest result. Of course, all programs produced exactly the same results -- all computations are integer computations. Expected results are:

Runtimes in real and user seconds are given in below table. Times taken with time command.

3. Special modifications in COBOL program. Simon Sobisch, in private e-mail from 13-Mar-2023 to me, made below changes to the COBOL program:

1. No variable with sign any more, i.e., variable is always greater or equal zero
2. Compiler optimization flag -fno-trunc

Modifications by Simon Sobisch:

21c21
<        77 l             pic s9(8) comp-5.
---
>        77 l             pic 9(8) comp-5.
139,149c134,146
<                compute l = z - A(j)
<                if l = 0 then
<       *            exit section
<                    go to configOK-exit
<                end-if
<                if l < 0 then
<                    compute l = 0 - l
<                end-if
<                if l = k - j then
<       *            exit section
<                    go to configOK-exit
---
>                evaluate true
>                when z < A(j)
>                    move A(j) to l
>                    subtract z from l
>                when z = A(j)
>                    exit section
>                when other
>                    move z to l
>                    subtract A(j) from l
>                end-evaluate
>                add j to l
>                if l = k then
>                    exit section

4. Conclusions.

1. C is by far the fastest. Faster than Java by a factor of 1.5. Almost two times faster than Dart as its other closest competitor.
2. Java is roughly 20% faster than Javascript.
3. JavaScript and Dart have almost the same performance.
4. Cobol can be made faster than JavaScript and Dart but slightly slower than Java.
5. PHP 8 is two times faster than Python, but almost 25-times slower than Javascript.
6. PHP with JIT is almost three times faster than normal PHP.
7. PHP with JIT enabled is five times faster than Python, but nine times slower than Javascript.

5. Dart program. Dart is a programming language by Google and used for Flutter.

int abs(int x) { return  ((x >= 0) ? x : -x); }

/* Check if k-th queen is attacked by any other prior queen.
   Return nonzero if configuration is OK, zero otherwise.
*/
bool configOkay (int k, List<int> a) {
    int z = a[k];

    for (int j=1; j<k; ++j) {
        int l = z - a[j];
        if (l == 0  ||  abs(l) == k - j) return false;
    }
    return true;
}


int solve (int N, List<int> a) {  // return number of positions
    int cnt = 0;
    int k = a[1] = 1;
    int N2 = N;  //(N + 1) / 2;
    bool flag = false;

    for (;;) {
        if (configOkay(k,a)) {
            if (k < N)  { a[++k] = 1;  continue; }
            else ++cnt;
        }
        flag = false;
        do
            if (a[k] < N)  { a[k] += 1;  flag = true; break; }
        while (--k > 1);
        if (flag) continue;
        a[1] += 1;
        if (a[1] > N2) return cnt;
        a[k=2] = 1;
    }
}


void main (List<String> argv) {
    int NMAX = 100;
    List<int> a = [ for(var i=0;i<100;++i) 0 ];
    int argc = argv.length;

    print(" n-queens problem.\n"
    "   2   4    6     8      10         12           14\n"
    " 1 0 0 2 10 4 40 92 352 724 2680 14200 73712 365596\n"
    );

    int start = 1;
    int end = 0;
    if (argc >= 1)  end = int.parse(argv[0]);
    if (end <= 0  ||  end >= NMAX)  end = 10;
    if (argc >= 2) { start = end;  end = int.parse(argv[1]); }
    if (end <= 0  ||  end >= NMAX)   end = 10;

    for (int n=start; n<=end; ++n)
        print(" D(${n}) = ${solve(n,a)}");
}

Added 28-Sep-2024: Francesco Cristiano on 24-Sep-2024 wrote to me and stated that inlining the abs() function in PHP would reduce the runtime by one second. That inlining is fully justified as the C program in question also uses a macro like:

#define abs(x)   ((x >= 0) ? x : -x)

So doing the same in PHP:

if ($l == 0  ||  ($l >= 0 ? $l : -$l) == $k - $j) return 0;

But this change in the PHP program not only reduces the runtime by one second, it almost halves the runtime!

The original benchmark was done more than a year ago. The environment now has changed a bit: Linux is now 6.10.10, and PHP is 8.3.12. Old program under this changed environment:

$ time php xdamcnt2.php 13
array(2) {
  [0]=>
  string(12) "xdamcnt2.php"
  [1]=>
  string(2) "13"
}
 n-queens problem.
   2   4    6     8      10         12           14
 1 0 0 2 10 4 40 92 352 724 2680 14200 73712 365596
 D( 1) = 1
 D( 2) = 0
 D( 3) = 0
 D( 4) = 2
 D( 5) = 10
 D( 6) = 4
 D( 7) = 40
 D( 8) = 92
 D( 9) = 352
 D(10) = 724
 D(11) = 2680
 D(12) = 14200
 D(13) = 73712
        real 6.71s
        user 6.69s
        sys 0
        swapped 0
        total space 0

Inlined program:

time php xdamcnt3.php 13
array(2) {
  [0]=>
  string(12) "xdamcnt3.php"
  [1]=>
  string(2) "13"
}
 n-queens problem.
   2   4    6     8      10         12           14
 1 0 0 2 10 4 40 92 352 724 2680 14200 73712 365596
 D( 1) = 1
 D( 2) = 0
 D( 3) = 0
 D( 4) = 2
 D( 5) = 10
 D( 6) = 4
 D( 7) = 40
 D( 8) = 92
 D( 9) = 352
 D(10) = 724
 D(11) = 2680
 D(12) = 14200
 D(13) = 73712
        real 3.53s
        user 3.50s
        sys 0
        swapped 0
        total space 0

Thanks Francesco.

Added 28 Sept 2024 at 15:14, Francesco Cristiano wrote:

Hi Elmar,

Glad to have contributed to your benchmark. I have been using PHP 8-jit to implement a power-to-ammonia simulation plant (with 3 millions time-steps) all through CLI and the execution time is indeed impressive. I don't see why people keep saying that PHP is a scripting language for webserver only.

Cheers Fran

Added 02-Oct-2024, Francesco Cristiano wrote:

Hi Elmar,

I've been testing how close PHP can get to Java using your n-queen code as a benchmark, and after using some tricks to improve the PHP code I'm getting some interesting results: This is the tweaked PHP code:

function solve ( $N, &$a) {  // return number of positions
    $cnt = 0;
    $k = $a[1] = 1;
    $N2 = $N;  //(N + 1) / 2;

    for (;;) {
        $flag = false;

        //code equivalent to configOkay function
        $configOkayFlag=true;
        {
           
            $z = $a[$k];

            for ($j=1; $j<$k; ++$j) {
                $l = $z - $a[$j];
                if ($l == 0  ||  (($l >= 0) ? $l : -$l) == $k - $j) {
                    $configOkayFlag= false;
                    break;
                }
                   
            }
        }
        // end of code equivalent to configOkay function
       
        //if (configOkay ($k, $a)) {
        if ( $configOkayFlag) {
            if ($k < $N)  {
                $a[++$k] = 1;  
                continue;
            }
            else
                ++$cnt;
        }
       
        do
            if ($a[$k] < $N)  {
                $a[$k] += 1;
                $flag = true;
                break;
            }
        while (--$k > 1);

        if ($flag)
            continue;

        $a[1] += 1;
        if ($a[1] > $N2)
            return  $cnt;
        $k = 2 ;  
        $a[2] = 1;
    }
}
$get_as_float = true;
$start = microtime(true);
$a= [];
for($i=1; $i <= 13; $i++){

    $result=solve($i,$a);
   
    echo "i=$i - number of combinations: $result\n";
  
}
$time_elapsed_secs = microtime(true) - $start;
echo "time_elapsed_secs:$time_elapsed_secs \n";

And the equivalent Java code is the following:

class bench {

public static long solve (int N, int[] a) {  // return number of positions
        long cnt = 0;
        int k = a[1] = 1;
        int N2 = N;  //(N + 1) / 2;
        boolean flag;

        for (;;) {
            flag = false;

            //code equivalent to configOkay function
            boolean configOkayFlag=true;
             {
                int z = a[k];

                for (int j=1; j<k; ++j) {
                    int l = z - a[j];
                    if (l == 0  || ((l >= 0) ? l : -l) == k - j) {
                        configOkayFlag= false;
                        break;
                    }
                   
                }
            }
            // end of  code equivalent to configOkay function

            //if (configOkay(k,a)) {
            if ( configOkayFlag) {
                if (k < N)  { a[++k] = 1;  continue; }
                else ++cnt;
            }
            do
                if (a[k] < N)  { a[k] += 1;  flag = true; break; }
            while (--k > 1);
            if (flag) continue;
            a[1] += 1;
            if (a[1] > N2) return cnt;
            k = 2;  a[2] = 1;
        }
    }

public static void main(String[] args){
long startTime = System.currentTimeMillis();

int[] a = new int[73713];

        for(int i=1; i<=13; i++){
            System.out.println("i="+i+ " - number of combinations: "+ bench.solve(i, a));
        }
        long stopTime = System.currentTimeMillis();
        long elapsedTime = stopTime - startTime;
      
        System.out.println("Total time: " + elapsedTime +"ms");
}
}

These are the results I'm getting:

c:\php8.4\php.exe -f bench.php
i=1 - number of combinations: 1
i=2 - number of combinations: 0
i=3 - number of combinations: 0
i=4 - number of combinations: 2
i=5 - number of combinations: 10
i=6 - number of combinations: 4
i=7 - number of combinations: 40
i=8 - number of combinations: 92
i=9 - number of combinations: 352
i=10 - number of combinations: 724
i=11 - number of combinations: 2680
i=12 - number of combinations: 14200
i=13 - number of combinations: 73712
time_elapsed_secs:1.577357

java bench
i=1 - number of combinations: 1
i=2 - number of combinations: 0
i=3 - number of combinations: 0
i=4 - number of combinations: 2
i=5 - number of combinations: 10
i=6 - number of combinations: 4
i=7 - number of combinations: 40
i=8 - number of combinations: 92
i=9 - number of combinations: 352
i=10 - number of combinations: 724
i=11 - number of combinations: 2680
i=12 - number of combinations: 14200
i=13 - number of combinations: 73712
Total time: 875ms

Not bad, PHP is only taking 1.8 times the execution time taken by Java.

Cheers Fran