While compilers can find more efficient ways of implementing those computations, the computations cannot be done at compile time, and thus actual operation cannot be “optimized away”. Dhrystone for example was split to multiple files since the advent of the technique called inlining allowed compilers to avoid performing the very tasks the benchmark was trying to analyze. Since compilers can now analyze the program even when it is split to multiple files, this cure did not work…

In fact, many modern architectures rely on the compiler to create code that can make efficient use of hardware resources, and restricting the compiler from optimizing is not a reasonable restriction. CoreMark does not attempt to force a specific number of branches or loads, rather only that all computations are performed at run time. Complex architectures are going to find different ways of doing those computations (e.g. SIMD, predicated execution and more), while simple architectures are going to perform the operations head on. As long as the operation is performed, the benchmark is useful to analyze the performance potential of the core.

[NOTE: To put this blog into context, refer to the scores submitted on April 12 for  Intel Atom N450, VIA Nano L3050, AMD Mobile Athlon XP-M (Barton), and Freescale i.MX515 at http://coremark.org/benchmark/index.php?pg=benchmark)

Why did you use FORK rather then PTHREADS?

Answer: I used the same set of CoreMark compiler flags and settings as I had come up with several months  ago after experimenting on an AMD Phenom II system. However, these settings came directly from the best performing Intel system on CoreMark website. Of course, forking is the most reliable way of getting multicore scaling across a broad range of platforms, which is my goal. Threading can lead to headaches in those situations.

*NOTE from Shay Gal-On: CoreMark actually contains 3 separate methods to use concurrency. All of them have been thoroughly validated, so feel free to pick any method you wish and tell us why you picked it…

Why did you choose to oversubscribe the cores with 4 threads?

Answer: I wanted to use exactly the same settings across all platforms under test. The only platform in my ARM versus x86 report to benefit from forking was the Intel Atom; if all CPUs had been single-core without HyperThreading, I would have only used one thread.  As you know, threading/forking places greater pressure on  caches and memory and I wanted to keep this consistent across all systems under test.

* We like this approach, though what is a fair comparison? As mentioned in previous posts, CoreMark is not really focused on multiple cores. We highly recommend using EEMBC MultiBench when trying to evaluate performance in a MultiCore environment.

Why did you under-clock the N450 to 1GHz when it normally runs at 1.66GHz? Did it change the core/bus ratio?

Answer: The ARM Cortex-A8 system ran at 800MHz and I wanted to have all of the platforms operate at this speed for a fair IPC performance comparison.

Unfortunately, the Atom N450 can only be downclocked to 1GHz, so that’s what I was forced to use.  Only the multiplier was changed on the Atom and the VIA Nano L3050; adjusting the bus clock / system clock should always be avoided in performance benchmarking for many, many obvious reasons.

* For processors with even a small cache, CoreMark will operate entirely inside the cache, and will not be affected by memory performance at all. This is a critical concern for small embedded devices though, as mentioned in a previous blog post.

All in all, nice to see that industry analysts are picking up CoreMark and using it to test processors at any level. Check the news section to see the latest…

The memory to CPU frequency ratio is a common limitation, and various technological solutions are available. Cache is one answer, but expensive in terms of silicon area and resulting cost for the end product, especially critical in low-end microcontrollers. Other solutions may have wide reads (e.g. NXP ARM7 parts read 128 bits at a time) which will speed up execution of serial blocks of code, or more advanced techniques such as the “Enhanced Flash Memory Accelerator”  (see NXP LPC1759 CPU).

In general, performance will not increase linearly with frequency if the code and/or data the program needs resides in memory that cannot scale at the same ratio. This will be true in benchmarks and in ‘real’ life. Does it matter to your application?

http://www.youtube.com/watch?v=njBGUzAExo4

Every programmer needs a code editor, and once you get used to the quirks of an editor, it is hard to switch. Many still use EMACS or even VI – on some embedded platforms I use VI since it is easily available on even the most minimal Linux distribution.

For a graphical environment though, I like more convenience that comes with a GUI but am not willing to give up on some useful features:
- Syntax highlighting
- Regular expressions for find/replace
- Auto code layout (smart tabs, brace matching and their ilk)
- Browse/tags database support
- Performance (time to initial open, time to load/edit/search large files)
- Customization
- Macros (Define commands and key sequences)
- Bindings (the ability to customize any key combo to an editor command)

Other nice to have features:
- Code completion
- Column editing mode
- Tabs for documents
- Integration with external commands
- Code folding
- Integrated file explorer
- Integrated source control (SVN)
- Sessions (open up with whatever was in the editor when it was closed)
- Hex edit

Anything else is a bonus I will gladly take but am not willing to give up
convenience or speed.

I count 60 editors on http://en.wikipedia.org/wiki/Comparison_of_text_editors#Programming_features, how do you choose the right one for you? I try to check what is available and try 3 new editors once a year.Sometimes I will even switch…

Currently on Windows I am using Notepad++. Originally I was drawn by the fact this had most of what I wanted and I had access to the source to make a few modifications. Since then this editor has matured and I no longer have a private version compiled from the source.

Things I wish were different but not enough to fix the source code:
- Regex for find/replace could be better (e.g. just use Perl Regex)
- Really long lines don’t display correctly (compressed JavaScript)

Other editors I have used in the past and switched from:
- Slickedit (awesome editor, but cost issues when I switched to a new
company)
- Editplus (was not maintained for a long while. Looks like it is actively
developed again, may need to check it out)
- Ultraedit (somehow it just did not measure up and I ended up dropping it after a mere 3 months of use)
- XEmacs (EMACS is extremely powerful, but the human interface of the GUI version doesn’t cut it)
- PSpad (performance issues caused me to drop this one)

Which 3 editors do you think I should try in 2010?

Realistically though, all of those data types are commonly used in most C code. The compiler is in charge of making the best use of the processor resources, and making sure the end result is correct.

CoreMark intrinsically uses  several integer data types:

8b – used as data for the state machine (mostly in compares).

16b – used as data and status info during the list processing (read/write and bit manipulation) as well as input data for the matrix operations (computations), and for crc.

32b – used as result data type for matrix multiply operations.

This introduces a mix of the common programming data types and covers core integer functionality in the processor.

Since the microcontroller is not using cache, the memory to cpu frequency is going to make a difference.

So let us take frequency into account:

We can see that the compilers from 2009 are both pretty close. There is no data in the submission on the number of wait states used for the flash with the MPLAB compiler, but Konstantin Yurkevich of Rovalant Inc sent us the project and port he used when submitting the scores (which you can find in the CoreMark download section).

We can also see that even with the same compiler, there is a difference of ~10% between different optimization levels (-O2 to -O3).

The compiler from 2007 creates code that is ~20% slower then the best submitted score, showing that the compiler has matured in 2 years (I would hope it would). More importantly, it shows that the compiler and compiler flags used must be included along with the score, as without that information the score is not reliable or repeatable!

Running on a Linux platform with a native toolchain:

Open up linux/core_portme.h (or linux64/core_portme.h on a 64b platform), and check all predefined macros (for example endianness) for your platform.

Make Results are in run1.log and run2.log.

For a cross hosted toolchain, edit the file linux/core_portme.mak. You need to define how to load the binary to the target and how to run it. If the target supports ssh, just un-comment the LOAD and RUN sample using ssh.

What if there is no OS running on the board?

1. Create a new folder.

2. Copy the barebones content to the new folder.

3. Edit the 3 files in the new folder, adding functionality for timing and output, and setting toolchain options.

* You may want to use basic sample code for UART (or debugger output) and timer for your board. Sample code is usually available with your embedded toolchain or kit.

> make

CoreMark aims at measuring core efficiency, and nothing else. Multiple copies of coremark would (assuming decent scheduling by the underlying scheduler or OS) each operate on a single core. There is no interaction between multiple copies of CoreMark, and synchronization only occurs at the end of the run. Thus we would expect execution of multiple copies to simply result in linear speedup – and indeed the posted scores show linear speedup (or as close as makes no difference). When comparing CoreMark scores, make sure to check the compiler flags and parallel options.

For true analysis of multi-core devices, system factors such as cache coherency and bus arbitrations mechanisms also need to be considered, as does the efficiency of synchronization primitives and the underlying scheduler (or OS). Head to the EEMBC website and check out MultiBench for more comprehensive analysis of multi-core systems.

E5405 Speedup

]]> http://www.eembc.org/wordpress/?feed=rss2&p=48 3 http://www.eembc.org/wordpress/?p=32 http://www.eembc.org/wordpress/?p=32#comments Sat, 06 Jun 2009 19:52:27 +0000 Markus Levy http://www.coremark.org/wordpress/?p=32 Although the current run rules for CoreMark don’t allow messing with the code, it’s an interesting exercise to see how much concurrency can be extracted from this relatively sequential benchmark code. CriticalBlue took this on with its Prism tool, writing up an interesting article on the process and results. Check it out