St Andrews Research Repository

St Andrews University Home
View Item 
  •   St Andrews Research Repository
  • University of St Andrews Research
  • University of St Andrews Research
  • University of St Andrews Research
  • View Item
  •   St Andrews Research Repository
  • University of St Andrews Research
  • University of St Andrews Research
  • University of St Andrews Research
  • View Item
  •   St Andrews Research Repository
  • University of St Andrews Research
  • University of St Andrews Research
  • University of St Andrews Research
  • View Item
  • Login
JavaScript is disabled for your browser. Some features of this site may not work without it.

Plug and Play Bench : simplifying big data benchmarking using containers

Thumbnail
View/Open
plug_play_bench.pdf (267.7Kb)
Date
11/12/2017
Author
Ceesay, Sheriffo
Barker, Adam David
Varghese, Blesson
Keywords
QA75 Electronic computers. Computer science
DAS
Metadata
Show full item record
Altmetrics Handle Statistics
Altmetrics DOI Statistics
Abstract
The recent boom of big data, coupled with the challenges of its processing and storage gave rise to the development of distributed data processing and storage paradigms like MapReduce, Spark, and NoSQL databases. With the advent of cloud computing, processing and storing such massive datasets on clusters of machines is now feasible with ease. However, there are limited tools and approaches, which users can rely on to gauge and comprehend the performance of their big data applications deployed locally on clusters, or in the cloud. Researchers have started exploring this area by providing benchmarking suites suitable for big data applications. However, many of these tools are fragmented, complex to deploy and manage, and do not provide transparency with respect to the monetary cost of benchmarking an application. In this paper, we present Plug And Play Bench (PAPB1): aninfrastructure aware abstraction built to integrate and simplifythe deployment of big data benchmarking tools on clusters of machines. PAPB automates the tedious process of installing, configuring and executing common big data benchmark work-loads by containerising the tools and settings based on the underlying cluster deployment framework. Our proof of concept implementation utilises HiBench as the benchmark suite, HDP as the cluster deployment framework and Azure as the cloud platform. The paper further illustrates the inclusion of cost metrics based on the underlying Microsoft Azure cloud platform.
Citation
Ceesay , S , Barker , A D & Varghese , B 2017 , Plug and Play Bench : simplifying big data benchmarking using containers . in J-Y Nie , Z Obradovic , T Suzumura , R Ghosh , R Nambiar , C Wang , H Zang , R Baeza-Yates , X Hu , J Kepner , A Cuzzocrea , J Tang & M Toyoda (eds) , Proceedings 2017 IEEE International Conference on Big Data (IEEE BigData 2017) . , 8258249 , IEEE Computer Society , pp. 2821-2828 , Workshop on Benchmarking, Performance Tuning and Optimization for Big Data Applications (BPOD) , Boston , Massachusetts , United States , 11/12/17 . https://doi.org/10.1109/BigData.2017.8258249
 
workshop
 
Publication
Proceedings 2017 IEEE International Conference on Big Data (IEEE BigData 2017)
DOI
https://doi.org/10.1109/BigData.2017.8258249
Type
Conference item
Rights
© IEEE, 2017. This work has been made available online in accordance with the publisher’s policies. This is the author created, accepted version manuscript following peer review and may differ slightly from the final published version. The final published version of this work is available at https://doi.org/10.1109/BigData.2017.8258249
Description
This research was supported by a Microsoft Azure Award.
Collections
  • Computer Science Research
  • University of St Andrews Research
URI
http://hdl.handle.net/10023/12315

Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

Advanced Search

Browse

All of RepositoryCommunities & CollectionsBy Issue DateNamesTitlesSubjectsClassificationTypeFunderThis CollectionBy Issue DateNamesTitlesSubjectsClassificationTypeFunder

My Account

Login

Open Access

To find out how you can benefit from open access to research, see our library web pages and Open Access blog. For open access help contact: openaccess@st-andrews.ac.uk.

Accessibility

Read our Accessibility statement.

How to submit research papers

The full text of research papers can be submitted to the repository via Pure, the University's research information system. For help see our guide: How to deposit in Pure.

Electronic thesis deposit

Help with deposit.

Repository help

For repository help contact: Digital-Repository@st-andrews.ac.uk.

Give Feedback

Cookie policy

This site may use cookies. Please see Terms and Conditions.

Usage statistics

COUNTER-compliant statistics on downloads from the repository are available from the IRUS-UK Service. Contact us for information.

© University of St Andrews Library

University of St Andrews is a charity registered in Scotland, No SC013532.

  • Facebook
  • Twitter