The Embedded Microprocessor Benchmark Consortium (EEMBC) develops benchmark software to help system designers select the optimal processors, and benchmark tools to help consumers and IT professionals select the appropriate smart phones/tablets and networking firewall appliances. EEMBC organizes its benchmark suites targeting Automotive, Digital Media, Java, Multicore Processors, Networking, Office Automation, Signal Processing, Smartphones/Tablets and Browsers.

EEMBC FAQ

Expand All | Collapse All | Click on arrow to expand individual FAQ

About EEMBC
What is EEMBC's mission?
The Embedded Microprocessor Benchmark Consortium (EEMBC - pronounced embassy) is a non-profit organization that works collaboratively with member companies to develop performance benchmarks that target key applications of embedded systems. EEMBC activities include the EEMBC Technology Center, which delivers technical support to members and licensees, certifies benchmark scores, and oversees the development of benchmark suites.
Why aren't benchmark scores available for all embedded processors?
For all processor benchmarks, except CoreMark, members must agree to publish their benchmark scores. If a processor score is not available on the EEMBC website, you may request that score on line or directly with the corresponding processor vendor. Customer requests for scores on specific processors are a key mechanism by which more scores are published, and EEMBC encourages you to request certified scores for any processor you are considering for a design.
What steps does EEMBC take to ensure that the published benchmark results are accurate and honest?
Only processor benchmark scores that have been verified by the EEMBC Technology Center may be published following a rigorous testing process to ensure that benchmark scores are repeatable, obtained fairly, and according to the rules of EEMBC. The EEMBC Technology Center is also responsible for verifying the exact environment under which each test was run; this includes processor frequency, wait states, and compiler version.
EEMBC Membership
How do I qualify to be an EEMBC member?

You must be a a System Developer, Designer and/or manufacturer of embedded microprocessors, microcontrollers, or DSPs, or could benefit from the use of the EEMBC Benchmarks on your embedded microprocessors, microcontrollers, and/or DSPs, and/or embedded systems.

What are the main steps in becoming a member of EEMBC?
Fill out the Request Form
Is my whole company included in our EEMBC membership? Can more than one person in a member company access the benchmark source code?
Once you have become an EEMBC member, as many people in your company as you like may download the EEMBC benchmark suites, but each such individual must be advised of the EEMBC rules and the provisions of the EEMBC Member License Agreement. Moreover, each person accessing the members-only area of the EEMBC Web site to download benchmarks or obtain other information must register individually. You are also responsible for seeing that any employees who leave your company do not take the benchmarks outside of your company.
What sort of person should be the representative from our company to the EEMBC Board of Directors?
EEMBC Board of Directors meetings combine both technical and marketing-oriented discussions, and our member representatives include individuals whose roles in their respective companies are purely technical, purely marketing, and everything in between. In general, the most technical discussions in EEMBC take place on the working group level by teleconference and web-meetings outside of Board meetings.
Do we own the benchmark source code once we have become a member?
No, EEMBC owns the benchmark suites; your membership makes you a licensee.
What are the fees for joining EEMBC?
Contact Markus Levy for more information.
How are membership dues spent?
Your member dues go towards the development, maintenance, and marketing of the benchmarks as well as funding certifications and technical support from the EEMBC Technology Center.
Can I just audit EEMBC activities?
No, you must be an EEMBC member.
EEMBC Benchmarks
Does EEMBC have benchmarks for DSPs?
Yes. Our Digital Entertainment and Telecom benchmark suites contain a number of kernels and applications that are highly relevant for DSPs. Furthermore, EEMBC
EEMBC has such a variety of benchmarks, how do I determine which one to use?
Focus on those parameters that most closely match your application needs. A designer typically uses the benchmark scores as a criterion to pick the best processor for the application. For example, if the application does many table look-ups, then that parameter (in the Automotive/industrial benchmark suite) is most important and the designer will like give the most weight to that benchmark. On the other hand, if the application is a modem, then the benchmark scores in the Telecomm suite become the most important parameters.
What process does EEMBC use to select specific benchmark algorithms?
Algorithms are selected by discussion, consensus, and ultimately a vote of the member companies. Non-members are also invited to suggest benchmark algorithms that you would like to see the consortium develop.
Does EEMBC perform board-level or system-level benchmarking?
The Version 1.0 benchmarks are processor benchmarks. However, the board and/or system can make a significant difference in processor performance (for example, the memory subsystem is one aspect that can dictate performance). EEMBC
Understanding EEMBC Benchmark Scores
What scoring methods does EEMBC use?
For Processor benchmarks, EEMBC allows two scoring methods: Out-of-the-box and Full-Fury. Out-of-the-box is where you use the compiler of your choice along with any compiler switches, but you cannot modify the original EEMBC source code. Full-Fury is the optimized version where you can hand tune the code to take advantage of any peripherals or hardware accelerators, special libraries, or assembly language. Several members have provided different levels of optimized scores.
In out-of-the-box benchmarking, a vendor can use any compiler, but cannot alter the source code in any way. Does the compiler have a significant effect on the benchmark score, even in out-of-the-box scores?
Absolutely. Feedback from several processor vendors indicates that they have seen performance differences upwards of 40% simply as a result of using a different compiler. This is why vendors such as Green Hills and IAR are actively involved in the EEMBC process.
What do the out of box scores reveal about compiler performance and/or processor performance?
Over time, as more processor vendors submit scores from a variety of compilers running on the same processor, this will provide very valuable information. Also, if you held the compiler family constant and varied the processor, this would be a great way to compare a processor's C friendliness.
Must a compiler vendor be an EEMBC member to have scores posted on the EEMBC web site?
A compiler vendor's results on the web site are not related to whether or not that vendor is an EEMBC member. The processor vendor member typically makes that decision, and they usually publish scores from the compiler that yields the best scores.
EEMBC benchmarks attempt to resemble real-world applications by allowing members to optimize the source code to take advantage of their architectures. How do you define the optimization guidelines so customers can still make apples to apples comparis
EEMBC-allowed optimizations are decided on a benchmark-by-benchmark basis. There are some fundamental restrictions in place to ensure fair comparison. EEMBC allows optimizations that change the source code to take advantage of libraries or hardware accelerators. These restrictions will allow you to demonstrate an architecture's capability and facilitate practical comparisons. You can also use intrinsics ( i.e. C code modification), or change the algorithm in some cases. For example, if EEMBC has a radix-4 implementation of an FFT, we will allow it to change to radix-2. So it is still an FFT, but on some DSPs that makes a big difference.
The Certification Process
What are the main steps in obtaining certified benchmark scores for my processor?

1. Log in to EEMBC member section and 'Make a Certification Reservation'.

2. Build and run the benchmark(s) on your platform.

3. Send your platform and tools to EEMBC Technology Center.

4. Enter your benchmark data in the online disclosure report (in member section)

Are any additional tools needed to run the benchmarks?
No. For the processor benchmarks, you need to download the benchmark and port these to your board or platform. For a fee, the EEMBC Technology Center can do this for you.
What information goes into the Disclosure Workbook?
The Disclosure Workbook contains all the information necessary to recreate the benchmark environment. In addition to all the benchmark scores, the Disclosure Workbook contains board-level information (memory configuration, etc.), compiler information (settings, libraries, etc.), and processor information (core and bus speeds, cache sizes, on-chip peripherals, etc.). It also contains details on the types of optimizations that were used for a Full-Fury score. After certification is complete, and the scores are published, the associated Disclosure Workbook is publicly available alongside the benchmark scores.
What does the certification process cost?
As an EEMBC member, you are entitled to certifications per year (certain restrictions apply).
Application-Specific Topics
Why does EEMBC have an FFT filter benchmark in both the Telecommunications and Automotive/Industrial benchmarks?
FFTs are used more often in Telecomm applications, but they are becoming increasingly important in automotive and industrial applications. Sensors used in engine knock detection, vehicle stability control, occupant safety systems and flow control, all require sophisticated filtering of the incoming signal to determine what is really happening among a lot of noise.
Why don't the Telecomm benchmarks use floating point?
All of the kernels (Bit Allocation, AutoCorrelation, FFT/iFFT, Viterbi, Biquad IIR, Convolutional Encoder) in the suite are intended to be run (for comparison purposes) using 16-bit fixed point math. There are FP versions of some of the kernels which were used to generate the golden data file sets. At this time, we have no plans to extend the "Out of the Box" code to require/use FP math. You may re-code the kernels, using FP for an optimized version of the benchmark if you wish. At this time, we do not have access to any floating point data sets. The data sets which accompany the AutoCorr kernel are in 16-bit (1.15) fixed point format (that is: 1 sign bit, 15 bits of data) to reduce the precision of the data, you can remove (or zero) bits from the LSBs of each number.