Know How MLPerf Benchmark can be Beneficial for AI

MLPerf Benchmark can add value to AI



Most people when making capital investments in equipment are worried of three things: performance, price and total cost of the ownership. So it is vital to set a benchmark on which you can depend to understand the performance without a sort of pricing since it is impossible to calculate total cost of ownership. This is the same case for advanced systems that use AI applications like for cargo aircraft, an electronic locomotive etc. 

And this where the MLPerf benchmark suite steps in. It’s been just three years that researchers and engineers from Baidu, Harvard University, Google, Stanford University and University of California Berkeley have been administered by the MLCommons consortium that was formed in December 2020. And then it has become a quick suite to demonstrate the performance of their AI systems. 

MLCommons has nearly 70 member organizations that include researchers at top universities, cloud builders and semiconductor makers trying to prove that they have the best compute engines for AI. The MLPerf benchmark suite started out with ML training tests in 2018 and followed by benchmarks aimed at running training workloads on classical supercomputers focusing on HPC modeling and simulation. Later in 2019, MLPerf released a set of ML inference benchmarks, expanding upon the mobile devices too. ‘

When Dell and Hewlett Packard Enterprises are exploding this is because of AI applications making them the fastest growing products in the company’s three decades in the server business. The MLPerf benchmark suite creates a virtuous cycle, driving hardware and software engineers to create and design their systems for diverse AI algorithms that support image recognition, and recommendation engines. And the companies exploring customer demands and coming up with AI systems will have an edge over the MLPerf tests rankings. 

For all this having a suite of benchmarks is mandatory. The MLPerf v0.7 training benchmark results were in the last July of 2020 where Alibaba, Dell, Fujitsu, Google, Inspur, Intel, Nvidia, SIAT, and Tencent have submitted their benchmark results for their systems where the rankings goes as: 1 Inspur, 2 NVIDIA, 3 NVIDIA, 4 Alibaba, 5 Fujitsu, 6 NVIDIA, 7 DellEMC, 8 DellEMC, 9 Intel. 

Over years, MLCommons have been able to plot the performance that jumps with CPU, GPU, FPGA and custom ASIC devices that can be used across various devices for both training and interference workloads. 

Since Inspur tops the ranking, what it does for customers for real workloads and for its own MLPerf tests is that it has a unique design and software that turns capabilities and customers to turn towards and see how their AI application and workflows can relate. It has done MLperf optimizations that have open sourced the code to GitHub to help customers achieve AI performance faster on their applications. The other thing Inspur can leverage is on its supply chain and scale of its server business that can be cost effective for AI systems. And the best thing you could do is to know the power that AI systems consume working as power is a factor in AI system architecture.