The Standard Performance Evaluation Corporation (SPEC), which tasks itself with maintaining standardized benchmarks for computing components, has invalidated more than 2,600 results for Intel Xeon chips.
The non-profit consortium revealed that it would no longer publish SPEC CPU 2017 results for Intel PCs running a specific version of the Intel compiler.
The decision stems from concerns over what the consortium describes as a targeted optimization for a particular workload. In other words, SPEC is accusing Intel of tweaking its products to obtain better benchmark results.
2,600 Intel benchmarks invalidated
SPEC CPU 2017 is a widely used benchmark for high-end servers, data centers and workstations, testing performance across a total of 43 benchmarks. The controversy relates to the Intel oneAPI DPC++/C++ Compiler, which has been allegedly optimized for the 523.xalancbmk_r / 623.xalancbmk_s benchmarks.
A disclaimer is now attached to more than 2,600 results: “The compiler used for this result was performing a compilation that specifically improves the performance of the 523.xalancbmk_r / 623.xalancbmk_s benchmarks using a priori knowledge,” thus invalidating the results.
Although benchmark figures are generally non-reflective of real-world conditions, testing is designed to be as life-like as possible, so optimizations designed to cater to specific benchmarks can result in skewed (and unrealistic) results.
According to a Phoronix report, 2022.0 to 2023.0 models are affected, leaving models either side of this date range unaffected by the disclaimer notices. The article continues: “The SPEC CPU specific optimization could result in a 9% overall SPECint speed uplift and around 4% for the SPECint rate.”
TechRadar Pro has reached out to SPEC and Intel for any further information, but neither responded immediately.
More from TechRadar Pro
- Check out the best processors for your PC
- We’ve rounded up a list of the best small business servers
- Will Intel’s cheapest 14th-gen CPU be worth buying? Price leak gives us a hint