Towards semi-automatic data-type translation for parallelism in Erlang

Abstract

As part of our ongoing research programme into programmer-in-the-loop parallelisation, we are studying the problem of introducing alternative data structures to support parallelism. Automated support for data structure transformations makes it easier to produce the best parallelisation for some given program, or even to make parallelisation feasible. We use a refactoring approach to choose and introduce these transformations for specific algorithmic skeletons, structured forms of parallelism that capture common patterns of parallelism. Our approach integrates with the Wrangler refactoring tool for Erlang, and uses the advanced Skel [4] skeleton library for Erlang. This library has previously been shown to give good parallelisations for a number of applications, including a multi-agent system [1] where we have achieved speedups of up to 142.44 on a 61-core machine with 244 threads. We have investigated three widely-used Erlang data structures: lists, binary structures and ETS (Erlang Term Storage) tables. In general, we have found that ETS tables deliver the best parallel performance for the examples that we have considered. However, our results show that simple lists may deliver similar performance to the use of ETS tables, and better performance than using binary structures. This means that we cannot blindly choose to implement a single optimisation as part of the compilation process. Our approach also allows the use of new (possibly user-defined) data structures and other transformations in future, giving a high level of flexibility and generality.