St Andrews Research Repository

St Andrews University Home
View Item 
  •   St Andrews Research Repository
  • University of St Andrews Research
  • University of St Andrews Research
  • University of St Andrews Research
  • View Item
  •   St Andrews Research Repository
  • University of St Andrews Research
  • University of St Andrews Research
  • University of St Andrews Research
  • View Item
  •   St Andrews Research Repository
  • University of St Andrews Research
  • University of St Andrews Research
  • University of St Andrews Research
  • View Item
  • Register / Login
JavaScript is disabled for your browser. Some features of this site may not work without it.

Benchmarking and performance modelling of MapReduce communication pattern

Thumbnail
View/Open
PID6195863.pdf (530.6Kb)
Date
27/01/2020
Author
Ceesay, Sheriffo
Barker, Adam David
Lin, Yuhui
Funder
EPSRC
Grant ID
EP/R010528/1
Keywords
Communication Pattern
Big Data
MapReduce
Modelling
QA75 Electronic computers. Computer science
3rd-NDAS
Metadata
Show full item record
Abstract
Understanding and predicting the performance of big data applications running in the cloud or on-premises could help minimise the overall cost of operations and provide opportunities in efforts to identify performance bottlenecks. The complexity of the low-level internals of big data frameworks and the ubiquity of application and workload configuration parameters makes it challenging and expensive to come up with comprehensive performance modelling solutions. In this paper, instead of focusing on a wide range of configurable parameters, we studied the low-level internals of the MapReduce communication pattern and used a minimal set of performance drivers to develop a set of phase level parametric models for approximating the execution time of a given application on a given cluster. Models can be used to infer the performance of unseen applications and approximate their performance when an arbitrary dataset is used as input. Our approach is validated by running empirical experiments in two setups. On average, the error rate in both setups is ±10% from the measured values.
Citation
Ceesay , S , Barker , A D & Lin , Y 2020 , Benchmarking and performance modelling of MapReduce communication pattern . in J Chen & L T Yang (eds) , Proceedings 2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom 2019) . , 8968864 , IEEE International Conference on Cloud Computing Technology and Science , IEEE Computer Society , pp. 127-134 , 2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom) , Sydney , New South Wales , Australia , 11/12/19 . https://doi.org/10.1109/CloudCom.2019.00029
 
conference
 
Publication
Proceedings 2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom 2019)
DOI
https://doi.org/10.1109/CloudCom.2019.00029
ISSN
2330-2186
Type
Conference item
Rights
Copyright © 2019 IEEE. This work has been made available online in accordance with publisher policies or with permission. Permission for further reuse of this content should be sought from the publisher or the rights holder. This is the author created accepted manuscript following peer review and may differ slightly from the final published version. The final published version of this work is available at https://doi.org/10.1109/CloudCom.2019.00029
Description
Funding: UK EPSRC EP/R010528/1 and IsDB
Collections
  • University of St Andrews Research
URI
http://hdl.handle.net/10023/19480

Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

Advanced Search

Browse

All of RepositoryCommunities & CollectionsBy Issue DateNamesTitlesSubjectsClassificationTypeFunderThis CollectionBy Issue DateNamesTitlesSubjectsClassificationTypeFunder

My Account

Login

Open Access

To find out how you can benefit from open access to research, see our library web pages and Open Access blog. For open access help contact: openaccess@st-andrews.ac.uk.

Accessibility

Read our Accessibility statement.

How to submit research papers

The full text of research papers can be submitted to the repository via Pure, the University's research information system. For help see our guide: How to deposit in Pure.

Electronic thesis deposit

Help with deposit.

Repository help

For repository help contact: Digital-Repository@st-andrews.ac.uk.

Give Feedback

Cookie policy

This site may use cookies. Please see Terms and Conditions.

Usage statistics

COUNTER-compliant statistics on downloads from the repository are available from the IRUS-UK Service. Contact us for information.

© University of St Andrews Library

University of St Andrews is a charity registered in Scotland, No SC013532.

  • Facebook
  • Twitter