Machine Learning Benchmark Suite

This EEMBC benchmark suite will use real-world workloads to identify the performance potential and power efficiency of processor cores used for accelerating machine-learning jobs on clients such as virtual assistants, smartphones, and IoT devices.

Call for Participation

According to recent press coverage1, more than 10 processor cores built to accelerate machine learning tasks on virtual assistants and IoT devices are competing for spots in SoCs, but the industry is still waiting for benchmarks that can show which of these chips delivers the best combination of performance, power, and die area.

EEMBC is currently seeking members for a new working group that will develop Machine Learning benchmarks that will serve as a vendor-neutral industry standard for measuring the performance and power consumption of cores running learning inference models on IoT edge devices. Examples of clients where these cores are used include Amazon Alexa, Apple’s Siri, and Google Cortana. The new EEMBC suite will thus open up a new area of performance measurement that until now has been neglected in favor of benchmarks that focus mainly on training processes in the cloud.

A progress report as of September 2018 is available here.

Examples of Potential Benchmark Targets

Working Group Participants as of June 2018

Working Group Status

Chairperson

[1] “ARM under attack in AI,” Electronic Engineering Times, April 10, 2018; “Mobile AI Race Unfolds at MWC,” Electronic Engineering Times, February 26, 2018.


Copyright EEMBC © 1997-2018

Note: This website only works on browsers that support ES6, such as Edge, Chrome, Firefox, Safari; IE11 and earlier are not supported.