Load balancing of irregular parallel applications on heterogeneous computing environments
Abstract
Large-scale heterogeneous distributed computing environments (such as Computational
Grids and Clouds) offer the promise of access to a vast amount of computing
resources at a relatively low cost. In order to ease the application development and
deployment on such complex environments, high-level parallel programming languages
exist that need to be supported by sophisticated runtime systems. One of the main
problems that these runtime systems need to address is dynamic load balancing that
ensures that no resources in the environment are underutilised or overloaded with
work.
This thesis deals with the problem of obtaining good speedups for irregular applications
on heterogeneous distributed computing environments. It focuses on workstealing
techniques that can be used for load balancing during the execution of irregular
applications. It specifically addresses two problems that arise during work-stealing:
where thieves should look for work during the application execution and how victims
should respond to steal attempts.
In particular, we describe and implement a new Feudal Stealing algorithm and
also we describe and implement new granularity-driven task selection policies in the
SCALES simulator, which is a work-stealing simulator developed for this thesis. In addition,
we present the comprehensive evaluation of the Feudal Stealing algorithm and
the granularity-driven task selection policies using the simulations of a large class of
regular and irregular parallel applications on a wide range of computing environments.
We show how the Feudal Stealing algorithm and the granularity-driven task selection
policies bring significant improvements in speedups of irregular applications, compared
to the state-of-the-art work-stealing algorithms. Furthermore, we also present the implementation
of the task selection policies in the Grid-GUM runtime system [AZ06]
for Glasgow Parallel Haskell (GpH) [THLPJ98], in addition to the implementation in
SCALES, and we also present the evaluation of this implementation on a large set of
synthetic applications.
Type
Thesis, PhD Doctor of Philosophy
Collections
Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.