With 6 scores posted for the PIC32 starter kit, I thought we should check out the results to understand the variance.The scores submitted by using 3 different compilers: MPLAB C32 1.0 (2007) , MPLAB C 1.05 (2009), Sourcery G++4.3 (2009).
Since the microcontroller is not using cache, the memory to cpu frequency is going to make a difference.
So let us take frequency into account:
| Freq | Compiler | /Mhz | Mark |
| 30MHz | GCC 4.3.2 -O3 | 2.599 | 77.973 |
| 30MHz | MPLAB C v1.05 -O3 | 2.426 | 72.78 |
| 72MHz | MPLAB C32 v1.00 -O2 | 1.712 | 123.264 |
| 72MHz | MPLAB C32 v1.00 -O3 | 1.899 | 136.728 |
| 80MHz | GCC 4.3.2 -O3 | 2.297 | 183.762 |
| 80MHz | MPLAB C v1.05 -O3 | 2.040 | 163.234 |
We can see that the compilers from 2009 are both pretty close. There is no data in the submission on the number of wait states used for the flash with the MPLAB compiler, but Konstantin Yurkevich of Rovalant Inc sent us the project and port he used when submitting the scores (which you can find in the CoreMark download section).
We can also see that even with the same compiler, there is a difference of ~10% between different optimization levels (-O2 to -O3).
The compiler from 2007 creates code that is ~20% slower then the best submitted score, showing that the compiler has matured in 2 years (I would hope it would). More importantly, it shows that the compiler and compiler flags used must be included along with the score, as without that information the score is not reliable or repeatable!