EEMBC® IoTMark™ is an objective, standardized benchmarking framework for measuring the energy efficiency of internet of things (IoT) edge nodes. We define an edge node, or "thing" of the IoT, as a platform with three primary parts: a sensor, a processor, and a radio interface. IoTMark determines the combined energy consumption of these elements as a way of helping designers choose the best microcontroller, RF components, and communications protocol for their application.Due to their ubiquity and size, IoT products have extraordinarily low power requirements. IoTMark helps developers understand how hardware choice impacts their battery life.
Battery life is often one of the most important factors in the design of an IoT edge node since many such platforms are deployed in settings where grid power is unavailable and human intervention to change batteries must be kept to a minimum. The processors in such applications, such as an array of solar panels in the desert, are typically relaying fixed amounts of data from the environment to an internet gateway at real-world speeds. As a result, maximizing processor performance is a much lower priority than maximizing battery run times.
There are two suites included in IoTMark. The first is IoTMark-BLE, a Bluetooth Low Energy (BLE) benchmark. The second, tentatively called IoTMark-Wi-Fi, will focus on Wi-Fi and is currently under development.
The IoTMark-BLE benchmark profile models a real IoT edge node consisting of an I2C sensor and a BLE radio through sleep, advertise, and connected-mode operation. By running the benchmark, designers can better understand trade-offs between energy consumption and the selected connection interval, frequency, and sensor characteristics (such as buffering), among others. For example, the adjacent graph compares the energy cost of shorter and longer connection intervals for a group of three devices.
The IotMark-BLE profile establishes a clearly defined communications path between an emulated sensor, the edge node processor and an emulated gateway as shown in the diagram below. The benchmark measures the energy required to power the edge node platform and to run the tests fed by the benchmark. The IO Manager emulates the sensor in a real application and feeds data to the device under test (DUT). A verification test at the output of the radio ensures that the device actually read the sensor data.
The IoTMark-BLE user interacts with the DUT using the interface shown below, which allows a number of tightly defined parameters to be set, such as the platform voltage, how many bytes are read in the I2C phase, the speed of the I2C bus, the number of bytes the gateway sends to the device, and the advertise and connect transmit power. Default values are provided to enable apples-to-apples comparisons between DUTs, but the user can vary these to turn IoTMark-BLE into an analysis tool that shows the sensitivity of a design to each of the included parameters.
The IoTMark-BLE host UI contains a number of controls for interacting with the IoTConnect framework.
The physical measurement apparatus consists of the readily available, low-cost components shown below. At the center of the energy measurement capability is the STMicroelectronics PowerShield, which can measure energy down 100nJ.
The IoTConnect framework creates an extensible framework for probing an embedded system. Block diagram (left), actual implementation (right). Click to enlarge.
Thanks to a member of our working group, we now have a set of Gerber files for an IoTMark shield. This shield eliminates the need for a breadboard and reduces the wiring clutter between the IO Manager, DUT and EMON. It plugs directly into the IO Manager as shown below. Further, by removing jumper J9 on the EMON and (and connecting EMON_TS to D7), the three boards can be sandwiched together to save even more space. Download the Gerber files and review the schematic PDF. This shield can also be used with ULPMark and SecureMark-TLS benchmarks.
NEW! You may now order these from OSHPark with this link:
Low-power Wi-Fi has evolved significantly over the past few years, and is rapidly becoming a viable solution in the edge-device space due to its maturity. This benchmark will explore the energy efficiency of various components related to Wi-Fi include both WPA and TLS secure connections, wireless transfers at both the TCP and UDP interface, and the impact of using application protocols like MQTTS. In addition, we are upgrading our IoTConnect framework to support high currents with devices such as the Keysight N6705 with the 6781 and 6731 power-supply modules for certification, as well as a more affordable solution using a user-supplied power-supply and the Joulescope energy monitor for general benchmark analysis.
Currently we are wrapping up the Layer 1 & 2 components of the benchmark and moving into the application layers. We've made significant progress developing a highly-programmable access point builing on the Raspberry Pi platform. Anyone familiar with 802.11 knows it can be a complex protocol: parts of the specification are ambiguous, which means implementations vary among vendors. Since repeatability is a fundamental component of all EEMBC benchmarks, these vendor variations create inconsistencies, and the IoTMark-Wi-Fi benchmark must guarantee a constrained, repeatable environment. To accomplish this, we’ve developed custom firmware for our own AP built on the Raspberry Pi platform using the hostapd service. By locking down the Broadcom driver version in the Pi’s image, and by strictly configuring hostapd (and tweaking the source code), the benchmark asserts greater control over the content of the frames and how the AP manages stations. The result is a well-behaved benchmark environment, with control over fragmentation, DTIM, channel hopping, and anything else in the 802.11 link-layer that might introduce surprises.
The working group is currently developing a Wi-Fi benchmark for IoT and is in the product definition phase. There is still room for new members to join and participate.
Copyright © EEMBC
Note: This website only works on browsers that support ES6, such as Edge, Chrome, Firefox, Safari; IE11 and earlier are not supported.