SimBench : a portable benchmarking methodology for full-system simulators
Date
13/07/2017Metadata
Show full item recordAbstract
Full-system simulators are increasingly finding their way into the consumer space for the purposes of backwards compatibility and hardware emulation (e.g. for games consoles). For such compute-intensive applications simulation performance is paramount. In this paper we argue that existing benchmarksuites such as SPEC CPU2006, originally designed for architecture and compiler performance evaluation, are not well suited for the identification of performance bottlenecks in full-system simulators. While their large, complex workloads provide an indication as to the performance of the simulator on ‘real-world’ workloads, this does not give any indication of why a particular simulator might run an application faster or slower than another. In this paper we present SimBench, an extensive suite of targeted micro-benchmarks designed to run bare-metal on a fullsystem simulator. SimBench exercises dynamic binary translation (DBT) performance, interrupt and exception handling, memoryaccess performance, I/O and other performance-sensitive areas. SimBench is cross-platform benchmarking framework and can be retargeted to new architectures with minimal effort. For several simulators, including QEMU, Gem5 and SimIt-ARM, and targeting ARM and Intel x86 architectures, we demonstrate that SimBench is capable of accurately pinpointing and explaining real-world performance anomalies, which are largely obfuscated by existing application-oriented benchmarks.
Citation
Wagstaff , H , Bodin , B , Spink , T & Franke , B 2017 , SimBench : a portable benchmarking methodology for full-system simulators . in 2017 IEEE International Symposium on Performance Analysis of Systems and Software . , 7975293 , Institute of Electrical and Electronics Engineers (IEEE) , 2017 IEEE International Symposium on Performance Analysis of Systems and Software , United States , 24/04/17 . https://doi.org/10.1109/ISPASS.2017.7975293 conference
Publication
2017 IEEE International Symposium on Performance Analysis of Systems and Software
Type
Conference item
Rights
Copyright © 2017 IEEE. This work has been made available online in accordance with publisher policies or with permission. Permission for further reuse of this content should be sought from the publisher or the rights holder. This is the author created accepted manuscript following peer review and may differ slightly from the final published version. The final published version of this work is available at https://doi.org/10.1109/ISPASS.2017.7975293.
Description
We acknowledge funding by the EPSRC grant PAMELA EP/K008730/1.Collections
Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.