SPEC invalidates SPECjbb 2013 benchmark scores due to bug

The Standard Performance Evaluation Corporation, or SPEC, has discovered a flaw in its SPECjbb 2013 benchmark software. The error may result in a SPECjbb score that is too high. The benchmark suite scores have been invalidated by SPEC for this reason.

The SPECjbb 2013 benchmark software is in use for testing servers. To do this, the performance of Java code is measured. The software has been on the market since January last year, but developer SPEC reports that it has found a disturbing flaw in the benchmark software. The bug in the so-called partial purchase request test item can result in a score that is too high, making fair comparisons impossible.

SPEC has decided to stop selling SPECjbb 2013 after discovering the bug. It is also no longer possible to upload benchmark scores and existing scores have been marked as unreliable. SPEC says it is working on a bug fix. Licensees of the SPECjbb 2013 suite will receive a free update in due course.