Benchmark

In this article we carry out a benchmark of a real world business problem that is solved using Java, C# and COBOL. We wanted to ascertain whether or not gnuCOBOL could solve the business problem as well as more modern object oriented languages.

We used a benchmark created by Mike Cowlishaw, which was devised to investigate the balance between Input and Output (I/O) time and compute calculation time in a simple batch program that might be found in a billing application at a telephone company.

The benchmark reads a large input file containing a list of telephone call durations, (each in seconds). For each call, a charging rate is chosen and the price calculated and rounded to hundredths. Taxes are then applied depending on the type of call and the total cost is aggregated and written out to an output file for each telephone call. Finally, running totals of total cost and taxes are displayed at the end of the benchmark for auditing purposes:

We set up directories for each of the implementations on our windows laptop C drive:

We then downloaded each of the implementations and compiled them in each of the directories. We downloaded the large data set called expon180.1e6 and upzipped a copy into each of the directories. We then ran the benchmark. So let’s see how java got on:

It took 1.377 microseconds to process each line of the file. Given there are one million lines in the file, that’s a batch job that took 1.377 seconds to complete. Now, we forgot to mention, this is running on an old Hewlett Packard laptop with an old Intel Core i7-2630QM CPU, so any compute instance in the cloud would process this batch job much more quickly. Still, waiting around for 1.377 seconds for your report to complete isn’t the end of the world!

Next up is C# using the open source mono distribution:

Slightly quicker, but not a lot in it to be honest!

Finally, let’s give gnuCOBOL a test drive:

Now, it took a lot longer than Java or C# weighing in at a hefty 8.75 seconds. Epic fail you might think? Well let’s look at the report it produced:

It’s a business friendly report that shows the call duration in seconds, the rate, price, taxes and total cost of the call. The COBOL version is doing a lot more write operations than the other two programs. It would be interesting to adapt the Java and C# versions to produce the same report as the COBOL version so that we can really benchmark these languages. Also the COBOL version applies bankers rounding to the figures, hence the difference in the aggregates.

However, we want to draw your attention to another aspect of this solution to real world business problems and that is code base maintenance. If we had to make an amendment to the code base, the COBOL version is much more readable as it’s pretty much written in English:

Compared to the object oriented code:

Furthermore, there are plenty of these COBOL batch jobs currently being run on mainframes and midrange computers and rather the rewrite them in Java or C#, most organisations would be better off migrating them to gnuCOBOL and training a new generation of COBOL developers. I am sure that’s the advice that Grace Hopper would be giving if she were alive today!