Industry-Standard Benchmarks for Embedded Systems
EEMBC, an industry alliance, develops benchmarks to help system designers select the optimal processors and understand the performance and energy characteristics of their systems. EEMBC has benchmark suites targeting cloud and big data, mobile devices (for phones and tablets), networking, ultra-low power microcontrollers, the Internet of Things (IoT), digital media, automotive, and other application areas. EEMBC also has benchmarks for general-purpose performance analysis including CoreMark, MultiBench (multicore), and FPMark (floating-point).
coremark

CoreMark FAQ

CoreMark-Pro is Now Available. Check it out!

Expand All | Collapse All | Click on arrow to expand individual FAQ

CoreMark Technical Fundamentals
What is CoreMark?
As its name implies, CoreMark is a simple, yet sophisticated, benchmark that is designed specifically to test the functionality of a processor core. CoreMark is not system dependent, therefore it functions the same regardless of the platform (e.g. big/little endian, high-end or low-end processor). Running CoreMark produces a single-number score allowing users to make quick comparisons between processors.
Why is CoreMark needed now?
Simply put, it’s time to eliminate the use of Dhrystone. Processors and associated systems are getting increasingly complex requiring increasingly complex benchmarks to analyze. The current and future EEMBC benchmarks are aimed at specific embedded market segments and are very successful at approximating real-world performance of embedded devices. However, there is also a need for a widely-available, generic benchmark specifically targeted at the processor core.
What are the basic features and characteristics of CoreMark?
CoreMark is comprised of small and easy to understand ANSI C code with a realistic mixture of read/write operations, integer operations, and control operations. CoreMark has a total binary size of no more then 16K using gcc on an x86 machine (this small size makes it more convenient to run using simulation tools).
Unlike EEMBC’s primary benchmark suites, CoreMark is not based on any real application, but the workload is actually comprised of several commonly used algorithms that include matrix manipulation (to allow for the use of MAC and common math operations), linked list manipulation (to exercise the common use of pointers), state machine operation (common use of data dependent branches), and Cyclic Redundancy Check (CRC is a very common function used in embedded).
What functionality of a processor core does CoreMark actually test?
Due to the focused nature of CoreMark, it will be useful for testing a processor’s pipeline operation, memory (or cache if any) access, and handling of integer operations.
NOTE: To thoroughly comprehend the capability of a processor’s core and an associated compiler, EEMBC recommends that you also utilize the EEMBC suites including AutoBench, DENBench, Networking, OABench, TeleBench, and (if applicable) MultiBench,.
How does CoreMark's small size help me test a platform’s memory subsystem?
The small size of CoreMark allows it to easily fit in a processor’s cache. One of the goals of CoreMark is to make it suitable for testing on a very wide range of processors. Some low-end microcontrollers don’t even have caches, let alone large amounts of system memory. Testing a platform’s memory subsystem requires more substantial benchmarks, such as those EEMBC provides with its many different benchmark suites that are aimed at testing the entire processor platform.
Is it possible for a compiler to optimize away the CoreMark workload?
While compilers may find more efficient ways of processing the workloads contained in CoreMark, the work itself cannot be optimized away. Furthermore, CoreMark does not use special libraries that can be artificially manipulated, and it was specifically designed not to make any library calls from within the timed portion of the benchmark.
What are the similarities and differences between CoreMark and Dhrystone?
Both CoreMark and Dhrystone are free, small, easily portable, available for practically any processor, and display a single number benchmark score. Despite these similarities, Dhrystone has several serious drawbacks which can be summarized into several categories:
  • Major portions of Dhrystone are susceptible to a compiler’s ability to optimize the work away; as an upshot, this is more a compiler benchmark then a hardware benchmark. This also makes it very difficult to compare results when different compilers/flags are used.
  • Library calls are made within the timed portion of Dhrystone. Typically, those library calls consume the majority of the time consumed by the benchmark. Since the library code is not part of the benchmark, it is difficult to compare results if different libraries are used
  • The Dhrystone code is completely synthetic and does not mimic any behavior that can be expected in a real application
  • The description is very vague and ambiguous on how the Dhrystone code should be run and on how results should be reported. This makes it very difficult to use results for a meaningful comparison, unless the exact same Dhrystone binary is used to make a comparison within a specific CPU family
  • There is no official source where you can get Dhrystone code so different, and often undisclosed, versions are in use (1.1, 2.0, 2.1)
  • Guidelines exist on how to run Dhrystone but they are not universally known, and since results are not certified or verified, they are not enforced
  • There is no standardization on how Dhrystone results should be reported, with various formats in use (DMIPS, Dhrystones per second, DMIPS/MHz)
What factors are involved in improving a device
Compilers can and do improve, and if CoreMark pushes compiler vendors to take better advantage of the existing platform hardware, this is a wonderful development. Multiply-Accumulate (MAC) and SIMD operations can make a difference if the compiler is able to utilize them for about a 1/3 of the CoreMark workload. Specifically, the matrix kernel can use any hardware you throw at it for improving ILP. Other improvements can be made by the compiler being able to lay out the code to take advantage of the branch prediction facilities of the processor for the state machine kernel. CoreMark does not have some of the structures that can benefit from more advanced compiler techniques such as loop fusion, loop skewing etc. And in hind sight, maybe we should have put some of those structures to push compiler vendors to integrate these technologies into their tools. It was built with real data structures in mind though, and not aiming at particular algorithms or implementation, exactly so as not to become just a compiler benchmark.
Will CoreMark work for analyzing multicore performance?

You can run multiple instantiations of CoreMark (a rate-type benchmark) and the source code includes a flag that will allow you to compile in code that supports a ‘create and destroy’ context functionality (where a context can be either threads or processes with shared memory or with sockets). Regardless, it’s expected that adding more cores will yield an almost linear performance behavior. For analyzing multicore performance, we highly recommend that you use the EEMBC MultiBench software.

Power and Energy Measurement
Similar to EEMBC
Power consumption is a very important parameter for analyzing a processor
Can CoreMark be used for estimation of power consumption?

CoreMark can be used as a standard workload for purposes of estimation of power consumption. You can use the functions start_time() and stop_time() to signal your measurement facility (either trigger if running on simulator, or use a gpio pin and EEMBC EnergyBench) and measure power/energy for the duration of the test. You may also be interested in http://www.eembc.org/benchmark/ulp_sl.php for purposes of estimating energy consumption.

Comparing CoreMark to EEMBC Application Suites
How does CoreMark differ from previous EEMBC benchmarks?
Current (and future) EEMBC benchmarks will typically target specific application areas, whereas CoreMark is a generic and simple benchmark, producing a single number and completely focusing on a core
Is CoreMark similar to other EEMBC benchmarks?
All EEMBC benchmarks, including CoreMark, are defined by committee and are thoroughly tested on a wide variety of platforms prior to release. Also, similar to other EEMBC benchmarks, the portability of CoreMark is also a key requirement. Unlike the primary EEMBC benchmarks, CoreMark is strictly focused on analyzing the core’s performance.
Under what circumstances should I use EEMBC's full-featured benchmarks rather than CoreMark?
CoreMark does not replace or supersede any of EEMBC’s application suites. CoreMark provides a very basic starting point for measuring a processor’s core performance, and is also applicable for a wide range of processors ranging from 8-bit microcontrollers to high-end 32-bit devices and architectures. While the small size of CoreMark makes it a practical test for processor cores, EEMBC’s full-featured application benchmarks are much better suited for testing a processor’s capability in a real application. Furthermore, processors are becoming increasingly complex and one core-based benchmark is insufficient for a comprehensive analysis.
CoreMark Logistics
How long does it take to set up and run CoreMark
CoreMark is easily portable to a wide range of microcontrollers and microprocessors. Therefore, if you have support for the basic functionality (i.e. clock(), GetSystemTimeAsFileTime(), lock_gettime()), it could take as little as 5 minutes. Otherwise, you need to specify how timing is to be accomplished in the porting layer. Furthermore, CoreMark includes full documentation to support your porting efforts and comprehend the run guidelines.
Under the standard EEMBC rules, all scores must be certified prior to publication. Why is this requirement changed for distribution of CoreMark scores?
The simplicity of CoreMark (i.e. on par with Dhrystone) makes it more suitable for self-verification of results and helps eliminate many common benchmarking errors encountered with the use of Dhrystone. Furthermore, the wide accessibility of this benchmark makes it easy to accomplish peer reviews and help ensure benchmark score reliability. However, to guarantee the repeatability and quality of the CoreMark scores, EEMBC also provide certifications on CoreMark for EEMBC members.
How does EEMBC publish scores for CoreMark?
EEMBC has set up a specific website (www.coremark.org) for the distribution of CoreMark and the publication of scores. All registered CoreMark users are encouraged to enter their scores and platform configurations on this website.