Certified Performance Analysis for Embedded Systems Designers

Autumn 2008

EEMBC Calendar

Embedded World 2009
March 3-5, 2009
Nürnberg, Germany

EEMBC and the Multicore Association will present an extended seminar on Analysis and Implementation of Multicore Hardware and Software.

Multicore Expo
March 16-19, 2009
Santa Clara, California

EEMBC will be presenting and exhibiting at the Multicore Expo, a leading-edge technology conference and exhibition dedicated to delivering practical solutions for designs involving embedded multicore, multimedia, and graphics.

The EEMBC Web site features a new menu on the Manage Benchmarks page that allows members to import benchmark scores directly from EEMBC Benchmark Software Run Logs. From there you can upload the log files, verify the data, and import the scores — which should save everyone a lot of time and ensure that the data entered is accurate. Access to this page is also available from the benchmark entry form. Also from the Manage Benchmark page, members can now view a report comparing their own benchmark scores with those obtained by EEEMBC Technology Center.

Letter from the President


When I worked with the embedded processor industry to start EEMBC 11 years ago, little did I imagine that the Dhrystone MIPS benchmark tool would survive into the 21st century. Yet here we are at the end of 2008 and still Dhrystone refuses to wither away. It is alive and well on the datasheets and powerpoint presentations of embedded processor vendors, even though nearly everyone in the embedded industry agrees that its usefulness as a tool to predict processor performance is quite limited, subject to easy manipulation, and lacking any framework for score verification.

On the other hand, it was never really correct to envision EEMBC as a "replacement" for Dhrystone any more than it’s appropriate to see a bottle of your favorite Bordeaux as a "replacement" for a can of Bud Light. The creation of EEMBC was intended to take industry-standard benchmarking to a level beyond where it had ever been before. Not only did this imply a higher level of nuance and complexity, it also had everything to do with ensuring that results were repeatable and that no scores would be published without certification.

The initial EEMBC benchmark suites, designed to represent real-world automotive, digital imaging, networking, telecom, and office automation applications, represented a quantum leap in sophistication over DMIPS. Subsequent generations of suites have added more application-focused detail and rigor while enabling meaningful benchmarking of multicore devices and opening the door to scenario-based benchmarking. It's difficult to imagine anything more removed from DMIPS.

Yet just as Bud Light serves a certain need, so has DMIPS managed to survive, in large part because it's small, easily portable, versatile, and free. In fact these are not undesirable attributes for a benchmark, particularly one that addresses the 16- and 32-bit devices that still account for a very large portion of the embedded market. But why should the industry have to settle for DMIPS? Surely the industry would be better served by a more sophisticated brew that shares DMIPS' positive attributes while providing for a standardized test framework and verifiability. This is a question to which EEMBC's members and Technology Center are currently devoting much thought, and we look forward to sharing with you our conclusions in the coming months.

Markus Levy
EEMBC President

Pipelining with MultiBench
By Shay Gal-On, Director of Software Engineering

There are three main types of concurrency that can be utilized with multicore technology: task-level decomposition, data decomposition, and pipelining (or functional decomposition). The first version of MultiBench, EEMBC’s multicore benchmark software, covered task level decomposition with multiple different work items operating in parallel, as well as data decomposition using special algorithms that were coded to use threads explicitly and in such a way as to allow any number of threads to work on the problem. Using these common multicore programming paradigms, EEMBC established several figures of merit (or "marks") based on a combination of the workloads and different options allowed for workload set:

  • ParallelMark – This mark covers data decomposition, and allows only one copy of an algorithm, but the number of workers working on the algorithm in parallel is flexible.
  • MultiMark – This mark covers another common use of multicore where multiple copies of the same algorithm are used concurrently.
  • MixMark – This mark covers the usage model where multiple tasks are being performed in parallel, but there is no synchronization between them.

. . . click to read more . . .

New Benchmark Scores
Freescale i.MX31-532MHz
Software: ARM Realview Compilation Tools v3.1 build 569
Hardware/Production Silicon

AutoBench 1.1 (out-of-the-box)
ConsumerBench 1.1(out-of-the-box)
OABench 1.1 (out-of-the-box)
TeleBench 1.1 (out-of-the-box)

News Briefs
EEMBC is pleased to welcome Samsung Electronics as its newest board member.

LynuxWorks and VirtualLogix are the newest members of EEMBC's Hypervisor Subcommittee, which is developing benchmarks that will show the impact of using a hypervisor as a virtualization platform. EEMBC welcomes inquiries from companies that are interested in contributing to the development of these new benchmarks by becoming a member of the EEMBC Board of Directors or its hypervisor subcommittee. More information is available on the EEMBC HyperBench page.

EEMBC President Markus Levy is a guest editor for the May/June 2009 special issue of IEEE Micro to be devoted to embedded multicore processors and systems. Tom Conte, professor in the College of Computing of the Georgia Institute of Technology, will co-edit the forthcoming issue. The call for papers deadline is December 19, 2008. Topics of interest include, but are not limited to, processors and SoCs, load balancing and application partitioning, development tools to enable efficient multicore programming, legacy code migration, analysis and implementation technologies to exploit system parallelism, operating systems, debugging, power management, inter-core communication mechanisms, resource management, and hypervisors and virtual machine monitors. Further information is available on the IEEE micro page.

Recent EEMBC contributed articles include "Measuring Multicore Performance" by Markus Levy and Shay Gal-On, which appeared in the November 2008 issue of the distinguished IEEE journal Computer. To order a reprint for $19 visit the IEEE Journal page.

If you do not wish to receive e-mail from EEMBC, you can un-subscribe. EEMBC sends no more than one e-mail per month to registered users at www.eembc.org. Continuing your subscription ensures you'll be notified when new scores and other important announcements are available.