[SPARK-21475][Core]Revert "[SPARK-21475][CORE] Use NIO's Files API to replace FileInp...
authorShixiong Zhu <zsxwing@gmail.com>
Sat, 30 Dec 2017 06:33:29 +0000 (22:33 -0800)
committerShixiong Zhu <zsxwing@gmail.com>
Sat, 30 Dec 2017 06:33:29 +0000 (22:33 -0800)
commit14c4a62c126ac5e542f7b82565b5f0fcf3e7fa5a
tree81bd6e828cfacef2771dcda2b4522d96075da994
parentf2b3525c17d660cf6f082bbafea8632615b4f58e
[SPARK-21475][Core]Revert "[SPARK-21475][CORE] Use NIO's Files API to replace FileInputStream/FileOutputStream in some critical paths"

## What changes were proposed in this pull request?

This reverts commit 5fd0294ff8960982cfb3b901f84bc91a9f51bf28 because of a huge performance regression.
I manually fixed a minor conflict in `OneForOneBlockFetcher.java`.

`Files.newInputStream` returns `sun.nio.ch.ChannelInputStream`. `ChannelInputStream` doesn't override `InputStream.skip`, so it's using the default `InputStream.skip` which just consumes and discards data. This causes a huge performance regression when reading shuffle files.

## How was this patch tested?

Jenkins

Author: Shixiong Zhu <zsxwing@gmail.com>

Closes #20119 from zsxwing/revert-SPARK-21475.
common/network-common/src/main/java/org/apache/spark/network/buffer/FileSegmentManagedBuffer.java
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ShuffleIndexInformation.java
core/src/main/java/org/apache/spark/shuffle/sort/BypassMergeSortShuffleWriter.java
core/src/main/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriter.java
core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala