人工智能与神经网络BenchMark--MLPerf

What is MLPerf

MLPerf is a benchmark suite that is used to evaluate training and inference performance of on-premises and cloud platforms. MLPerf is intended as an independent, objective performance yardstick for software frameworks, hardware platforms, and cloud platforms for machine learning.

The goal of MLPerf is to give developers a way to evaluate hardware architectures and the wide range of advancing machine learning frameworks.

MLPerf Training benchmarks

The suite measures the time it takes to train machine learning models to a target level of accuracy.

MLPerf Inference benchmarks

The suite measure how quickly a trained neural network can perform inference tasks on new data.

The MLPerf inference benchmark is intended as an objective way to measure inference performance in both the data center and the edge. Each benchmark has four measurement scenarios: server, offline, single-stream, and multi-stream. The following figure depicts the basic structure of the MLPerf Inference benchmark and the order that data flows through the system:

 Server and offline scenarios are most relevant for data center use cases.

while single-stream and multi-stream scenarios evaluate the workloads of edge devices. 

MLPerf Results Show Advances in Machine Learning Inference Performance and Efficiency | MLCommons

Today, MLCommons®, an open engineering consortium, released new results for three MLPerf™ benchmark suites - Inference v2.0, Mobile v2.0, and Tiny v0.7. These three benchmark suites measure the performance of inference - applying a trained machine learning model to new data. Inference enables adding intelligence to a wide range of applications and systems. Collectively, these benchmark suites scale from ultra-low power devices that draw just a few microwatts all the way up to the most powerful datacenter computing platforms. The latest MLPerf results demonstrate wide industry participation, an emphasis on energy efficiency, and up to 3.3X greater performance ultimately paving the way for more capable intelligent systems to benefit society at large.

The MLPerf Mobile benchmark suite targets smartphones, tablets, notebooks, and other client systems with the latest submissions hig