Industry-Standard Benchmarks for Embedded Systems
EEMBC, an industry alliance, develops benchmarks to help system designers select the optimal processors and understand the performance and energy characteristics of their systems. EEMBC has benchmark suites targeting mobile devices (for phones and tablets), networking, ultra-low power microcontrollers, the Internet of Things (IoT), digital media, automotive, and other application areas. EEMBC also has benchmarks for general-purpose performance analysis including CoreMark, MultiBench (multicore), and FPMark (floating-point).
Machine Learning Benchmark Suite - An EEMBC® Benchmark

This EEMBC benchmark suite will use real-world workloads to identify the performance potential and power efficiency of processor cores used for accelerating machine-learning jobs on clients such as virtual assistants, smartphones, and IoT devices.

Call for Participation

According to recent press coverage1, more than 10 processor cores built to accelerate machine learning tasks on virtual assistants and IoT devices are competing for spots in SoCs, but the industry is still waiting for benchmarks that can show which of these chips delivers the best combination of performance, power, and die area.

EEMBC is currently seeking members for a new working group that will develop Machine Learning benchmarks that will serve as a vendor-neutral industry standard for measuring the performance and power consumption of cores running learning inference models on IoT edge devices. Examples of clients where these cores are used include Amazon Alexa, Apple’s Siri, and Google Cortana. The new EEMBC suite will thus open up a new area of performance measurement that until now has been neglected in favor of benchmarks that focus mainly on training processes in the cloud.

Examples of Potential Benchmark Targets
  • Almotive Alware
  • Cadence Vision P6
  • Cambricon CPU
  • Ceva NeuPro
  • Imagination PowerVR 2NX
  • Nvidia NVDLA
  • Synopsys EV64
  • VeriSilicon VIP
  • Videantis v-MP6000
Working Group Participants as of May 11, 2018
  • Analog Devices
  • ARM
  • AuZone
  • Flex
  • Green Hills Software
  • Intel
  • Nvidia
  • NXP Semiconductors
  • Samsung
  • STMicroelectronics
  • Synopsys
  • Texas Instruments
Working Group Status
  • Currently defining the first proof of concept
  • Join the EEMBC Machine Learning working group to help ensure a meaningful and fair representation for your company’s products. Email EEMBC for more information.

Chairperson

  • Ramesh Jaladi, Intel

[1] “ARM under attack in AI,” Electronic Engineering Times, April 10, 2018; “Mobile AI Race Unfolds at MWC,” Electronic Engineering Times, February 26, 2018.

Options for using EEMBC® Benchmark Software

  1. License one or more benchmark suites
  2. Join the EEMBC Board of Directors
  3. Join application-focused EEMBC subcommittee(s)

Request information on becoming a member or licensing benchmark software

Benefits of licensing and membership are summarized in the table below.

Benefit Licensee Board or Member
Access to benchmarks
X
X
Use benchmarks to test processor/system products
X
X
Ability to tune your system for best performance
X
X
Influence selection and design of next generation benchmarks
X
X
Ability to publish or disclose your scores
X
X
Free certifications
-
X
Early access to next-generation benchmarks
-
X
Recognition as a supporting member of an industry-standard organization
-
X
Use certified scores in marketing and advertising promotions
-
X
Network with other industry leaders from partner and competitor companies
-
X