Mosharaf Chowdhury

Efficient Coflow Scheduling Without Prior Knowledge

Mosharaf Chowdhury, Ion Stoica
Appears in: 
CCR August 2015

Inter-coflow scheduling improves application-level communication performance in data-parallel clusters. However, existing efficient schedulers require a priori coflow information and ignore cluster dynamics like pipelining, task failures, and speculative executions, which limit their applicability. Schedulers without prior knowledge compromise on performance to avoid head-of-line blocking. In this paper, we present Aalo that strikes a balance and efficiently schedules coflows without prior knowledge.

Efficient coflow scheduling with Varys

Mosharaf Chowdhury, Yuan Zhong, Ion Stoica
Appears in: 
CCR August 2014

Communication in data-parallel applications often involves a collection of parallel flows. Traditional techniques to optimize flowlevel metrics do not perform well in optimizing such collections, because the network is largely agnostic to application-level requirements. The recently proposed coflow abstraction bridges this gap and creates new opportunities for network scheduling. In this paper, we address inter-coflow scheduling for two different objectives: decreasing communication time of data-intensive jobs and guaranteeing predictable communication time.

Syndicate content