[SPARK-20087][CORE] Attach accumulators / metrics to 'TaskKilled' end reason
authorXianjin YE <advancedxy@gmail.com>
Tue, 22 May 2018 13:02:17 +0000 (21:02 +0800)
committerWenchen Fan <wenchen@databricks.com>
Tue, 22 May 2018 13:02:17 +0000 (21:02 +0800)
commit82fb5bfa770b0325d4f377dd38d89869007c6111
tree71a5a2dc77e4ca6cb253c17dd8f3d9792d7f724e
parent952e4d1c830c4eb3dfd522be3d292dd02d8c9065
[SPARK-20087][CORE] Attach accumulators / metrics to 'TaskKilled' end reason

## What changes were proposed in this pull request?
The ultimate goal is for listeners to onTaskEnd to receive metrics when a task is killed intentionally, since the data is currently just thrown away. This is already done for ExceptionFailure, so this just copies the same approach.

## How was this patch tested?
Updated existing tests.

This is a rework of https://github.com/apache/spark/pull/17422, all credits should go to noodle-fb

Author: Xianjin YE <advancedxy@gmail.com>
Author: Charles Lewis <noodle@fb.com>

Closes #21165 from advancedxy/SPARK-20087.
core/src/main/scala/org/apache/spark/TaskEndReason.scala
core/src/main/scala/org/apache/spark/executor/Executor.scala
core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala
core/src/main/scala/org/apache/spark/util/JsonProtocol.scala
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala
project/MimaExcludes.scala