All weather scheduling : towards effective scheduling across the edge, fog and cloud

Abstract

The cloud plays a crucial part in the deployment of many applications today. However, alternative processing layers such as the edge and the fog are increasingly being used in order to meet applications' low-latency requirements, save bandwidth or reduce monetary cost. The rise in edge/fog use means that schedulers that can successfully schedule applications and their tasks across the edge/fog/cloud are a growing area of research interest. However, schedulers to date that have been proposed are limited in their functionality, not being adaptive (in terms of scheduling policies) to changing cluster state, consider applications only as a whole and not their individual task requirements, have been validated on a minimal number of applications, or require significant a-priori knowledge by the end-user of their applications' characteristics and its affinity to resources in their cluster.

We present the Weather-Adaptive-Scheduler or WASCH, a novel, data-driven scheduler for the edge/fog/cloud that achieves minimal task makespan for an applications' tasks. WASCH first profiles applications that a user wants to schedule across a cluster of edge/fog/cloud resources, using a set of representative tasks for that application. This data (application/task/node performance metrics) is used by WASCH to build a regression model. This model is then used to predict which resource in the cluster will deliver the minimal makespan of new end-user tasks. WASCH in a new contribution to the state of the art, does all of this automatically without any need for end-user a-priori knowledge of the cluster, the characteristics of an application or affinity of the application to the various resources. Via empirical evaluation of WASCH using 14 diverse, real-life applications, we show that WASCH successfully achieves minimal task makespan for those 14 applications' tasks when compared to two state of the art schedulers only taking an average of 6ms per prediction per resource.