Preemption for Hadoop

We deliver a patched version of Hadoop 0.21 which enables efficient task-level preemption.

The characteristics of this software are summarized in Deliverable 3.2 of the BigFoot project (cf.

Compiling and Installing

This patched version of Hadoop is compiled and installed in the same way as upstream Hadoop version (cf. hadoop/GettingStartedWithHadoop). The source code is included in this repository.

Using Suspension

This patched version of Hadoop exposes new options to the command line. To suspend a task, it is sufficient to call the command hadoop job -suspend-task <taskid>

Tasks can be resumed via the command hadoop job -resume-task <taskid>

Schedulers can exploit the same functionality via the suspendTaskAttempt and resumeTaskAttempt methods present in the TaskInProgress class.