Conception of a Load Balancing Strategy for CLOAK-Reduce

EasyChair Preprint 6487

Abstract

Distributed systems are highly heterogeneous, dynamic and unstable. It is therefore realistic to expect that some resources will fail during use. To overcome these problems and achieve better performance, it is necessary to implement load balancing algorithms that are adapted to any situation where some nodes are overloaded while others are less so or are even idle.

Load balancing between JobManager and JobManagers candidates, and between JobManagers of the same scheduler or load balancing between Schedulers, implies that additional loads are only done hierarchically. 

In this paper, we propose a two-level dynamic, hierarchical and decentralised  load balancing strategy focusing on three performance indicators namely: response time, process latency and running time of MapReduce jobs.

The first level of load balancing is intra-scheduler, in order to avoid the use of the large-scale communication network, and the second level of load balancing is inter-scheduler, for load regulation of our whole system.
The proposed solution provides a better optimisation of the load balancing process and an improvement of the task mean response time with minimal communication.

Keyphrases: Big Data, CLOAK-Reduce, Load Balancing, distributed processing, task allocation

@booklet{EasyChair:6487,
  author    = {Diarra Mamadou and Tiendrebeogo B. Telesphore},
  title     = {Conception of a Load Balancing Strategy for CLOAK-Reduce},
  howpublished = {EasyChair Preprint 6487},
  year      = {EasyChair, 2021}}