EEMBC Announces First Public Details of New Java Benchmarks

New Standards Are Industry’s First Aimed at J2ME Mobile Java Devices

EL DORADO HILLS, Calif.—February 17, 2003—EEMBC, the Embedded Microprocessor Benchmark Consortium, today disclosed the components of its new Java™ benchmark suite, which provides the industry’s first mechanism for evaluating the performance of embedded Java implementations according to objective and certifiable standards.

Selected to be representative of real-world Java applications, and for their resistance to manipulation, kernels for the first EEMBC® Java benchmark suite include photo decoding, a computation-intensive chess game, an Internet-usage benchmark for stressing CLDC threading, a cryptographic package oriented toward M-commerce, and an object-oriented benchmark to exercise object creation and garbage collection.

Used together, the components of the suite will broadly exercise the Java API while avoiding the limitations of synthetic benchmarks.

The new EEMBC Java benchmark suite is focused on devices running on J2ME, the Java standard for mobile devices. Because handheld systems use smaller processors and typically offer fewer than 256 kbytes of memory, a benchmark that specifically measures Java Virtual Machine (JVM) performance in these systems is crucial. Until now, however, Java benchmarks have been suitable only for measuring the performance of JVMs in PCs and servers. They thus assume Java APIs and capabilities (such as floating-point math) that are not available on the J2ME platform.

EEMBC designed the first EEMBC J2ME benchmarks for the Connected Limited Devices Configuration (CLDC), which is the set of classes that underlies all other J2ME profiles, including the Mobile Information Device Profile (MIDP). Platforms tested against the EEMBC J2ME benchmark must comply with the Java Technology Compatibility Kit (TCK) for CLDC to qualify for certification and publication of EEMBC benchmark scores.

“Mobile implementations will drive growth for Java in 2003, so EEMBC’s timing with these new Java benchmarks couldn’t be better,” said Will Strauss, president of Forward Concepts. “Java-acceleration technologies are proliferating to meet the needs of mobile applications, and the EEMBC benchmarks provide a quantitative means to weigh their merits. Embedded code within the EEMBC benchmarks works as a sort of watermark to prevent ‘spoofing,’ and this ensures that results are genuine and not bypassing critical operations to show better speed.”

The new EEMBC Java suite was developed by a group within the consortium whose members are the industry leaders who produce Java hardware- and software-acceleration technologies. Many of these companies compete head-on in the J2ME space but under the EEMBC umbrella have banded together to create an industry standard by which the performance of their products can be measured. Participants in the effort included representatives of Aplix, ARM, IBM, Infineon, Insignia, Intel, Motorola, Sun, Symbian, and Tao Group.

“This is the first time that OEMs implementing Java in their mobile embedded devices will be able to compare competing technologies objectively,” said Rod Crawford, EEMBC Java Subcommittee chairman and Java debug product manager at ARM. “The next step will include the certification and publication of benchmark scores from the competing companies. I also plan to lead the Java Subcommittee in the development of next-generation benchmarks that will evolve with the needs of the industry.”

To ensure that its Java and other application-focused benchmarks are applied fairly, EEMBC requires that the results of benchmark tests be repeated and certified by the EEMBC Certification Laboratories (ECL) as a condition of publication. ECL certification ensures that scores are obtained fairly and according to EEMBC rules.

“The members of EEMBC’s Java Subcommittee have undergone a diligent, multi-month process to ensure that the new Java benchmark suite is representative of real-world applications and workloads in the mobile device space,” said Markus Levy, EEMBC president. “Benchmark scores based on this suite will be tremendously important for system developers as a way to decide what is the most appropriate Java solution for their products.”

Scores based on the EEMBC Java benchmarks will be available for free on the EEMBC Web site at www.eembc.org.

EEMBC is a registered trademark of the Embedded Microprocessor Benchmark Consortium. All other trademarks referred to herein are the property of their respective owners.