Industry-Standard Benchmarks for Embedded Systems
EEMBC, an industry alliance, develops benchmarks to help system designers select the optimal processors and understand the performance and energy characteristics of their systems. EEMBC has benchmark suites targeting cloud and big data, mobile devices (for phones and tablets), networking, ultra-low power microcontrollers, the Internet of Things (IoT), digital media, automotive, and other application areas. EEMBC also has benchmarks for general-purpose performance analysis including CoreMark, MultiBench (multicore), and FPMark (floating-point).

IoT-GATEWAY


An EEMBC Benchmark 

EEMBC is developing an IoT-Gateway benchmark suite to measure how gateways perform in specific IoT use cases. Because IoT use cases are incredibly diverse and involve combinations of hardware and software, the industry needs multiple benchmarks based on specific application profiles. Following EEMBC's long-standing tradition, we will provide purchasing managers and solution developers with accurate, reliable information and tools that allow them to quickly and equitably compare the efficiency of system solutions targeted at IoT-Gateway applications.


IoT-Gateway – Near the Edge

As the IoT matures, more capabilities are being realized near “the edge” instead of the cloud. Edge environments are more resource-constrained and the need arises for IoT gateways which can execute these capabilities. However, the needed capabilities can vary wildly between applications, so it is difficult to define an IoT gateway. For example, factory automation and retail advertisement both require an IoT gateway, yet the final solution drastically differs. The market is expected to standardize around certain criteria, therefore, gateways can generally be defined as the first point of contact for edge nodes and sensors. They usually incorporate physical I/O, data aggregation, processing, monitoring, control, and security. Trying to characterize how gateways operate within certain IoT scenarios, their relative performance compared to other gateways are just some of the unknowns of this emerging market. The need arises for a way to provide standardized metrics for gateway performance characterization, and the vision of the EEMBC IoT-Gateway working group is to solve this problem by developing the first ever IoT gateway benchmark.


Key Characteristics of the EEMBC IoT-Gateway Benchmark
  • Standardizes assumptions about gateway operational conditions to ensure meaningful comparisons between gateway products
  • Utilizes a distributed approach with client-server interactions and workloads generated across multiple physical ports
  • Tests the system as a whole including processor, physical and wireless interfaces (e.g. WiFi, Bluetooth), and operating system
  • Eliminates the guesswork about each gateway’s potential performance for things such as sensor fusion, type of processing workloads, and how much data traffic to manage
  • Supports flexible plug-in and workload generation ability to also serve as a general IoT test and stress tool

Working Group Status
  • The working group is currently developing a proof of concept that is focused on transport layer security, controllable payload size and rate, and uses a NULL function for optimized throughput and latency.
  • Estimated completion time for phase 1 is Q1 2017
  • Join the EEMBC IoT-Gateway benchmark working group to a) help determine the exact details of this IoT benchmark; b) ensure the benchmarks are representative of all possible scenarios; c) ensure that you don’t have a competitive disadvantage. Email EEMBC for more information.

 

Chairperson

 

 

Options for Using EEMBC® Benchmark Software
1. License one or more benchmark suites
2. Join the EEMBC Board of Directors
3. Join application-focused EEMBC subcommittee(s)
4. Special licensing program for faculty members at colleges and universities.

Request information on becoming a member or licensing benchmark software

Benefits of licensing and membership are summarized in the table below.

Benefit Licensee Board or Member
Access to benchmarks
X
X
Use benchmarks to test processor/system products
X
X
Ability to tune your system for best performance
X
X
Influence selection and design of next generation benchmarks
X
X
Ability to publish or disclose your scores
-
X
Free certifications
-
X
Early access to next-generation benchmarks
-
X
Recognition as a supporting member of an industry-standard organization
-
X
Use certified scores in marketing and advertising promotions
-
X
Network with other industry leaders from partner and competitor companies
-
X