tez.git
11 months agoUpload CHANGES.txt for imcompatible changes branch-0.9.0 rel/release-0.9.0 release-0.9.0-rc1
Zhiyuan Yang [Mon, 17 Jul 2017 22:47:36 +0000 (15:47 -0700)] 
Upload CHANGES.txt for imcompatible changes

11 months agoincorporate new changes in CHANGES file release-0.9.0-rc0
Zhiyuan Yang [Fri, 14 Jul 2017 23:19:00 +0000 (16:19 -0700)] 
incorporate new changes in CHANGES file

11 months agoincorporate new changes in CHANGES file
Zhiyuan Yang [Fri, 14 Jul 2017 18:00:35 +0000 (11:00 -0700)] 
incorporate new changes in CHANGES file

11 months agoChange version to 0.9.0, create CHANGES file
Zhiyuan Yang [Thu, 6 Jul 2017 18:35:11 +0000 (11:35 -0700)] 
Change version to 0.9.0, create CHANGES file

11 months agoTEZ-3798. Remove duplicate package-info.java (zhiyuany)
Zhiyuan Yang [Fri, 14 Jul 2017 23:10:28 +0000 (16:10 -0700)] 
TEZ-3798. Remove duplicate package-info.java (zhiyuany)

11 months agoTEZ-3792. RootInputVertexManager doesn't drain queued source task completed events...
Jonathan Eagles [Fri, 14 Jul 2017 15:42:23 +0000 (10:42 -0500)] 
TEZ-3792. RootInputVertexManager doesn't drain queued source task completed events (Eric Badger via jeagles)

11 months agoTEZ-3795. Vertex state machine can throw InvalidStateTransitonException from TERMINAT...
Jonathan Eagles [Fri, 14 Jul 2017 15:29:52 +0000 (10:29 -0500)] 
TEZ-3795. Vertex state machine can throw InvalidStateTransitonException from TERMINATING state (Kuhu Shukla via jeagles)

11 months agoTEZ-3794. tez-tools: swimlane does not recognize HistoryEventHandler.criticalEvents...
Rajesh Balamohan [Wed, 12 Jul 2017 21:55:08 +0000 (03:25 +0530)] 
TEZ-3794. tez-tools: swimlane does not recognize HistoryEventHandler.criticalEvents based logs (rbalamohan)

11 months agoTEZ-3791. Failed/Killed task can throw InvalidStateTransitonException when a new...
Jonathan Eagles [Tue, 11 Jul 2017 20:29:09 +0000 (15:29 -0500)] 
TEZ-3791. Failed/Killed task can throw InvalidStateTransitonException when a new attempt is launched (Kuhu Shukla via jeagles)

11 months agoTEZ-3787. Remove Tez UI build and rebuild errors and warning due to yarn install...
Jonathan Eagles [Mon, 10 Jul 2017 15:03:06 +0000 (10:03 -0500)] 
TEZ-3787. Remove Tez UI build and rebuild errors and warning due to yarn install and ember-truth-helpers (jeagles)

11 months agoTEZ-3786. Fix Tez UI test failures after TEZ-3775 (jeagles)
Jonathan Eagles [Mon, 10 Jul 2017 14:58:19 +0000 (09:58 -0500)] 
TEZ-3786. Fix Tez UI test failures after TEZ-3775 (jeagles)

11 months agoTEZ-3784. Submitting very large DAG throws com.google.protobuf.CodedInputStream excep...
Rajesh Balamohan [Sat, 8 Jul 2017 03:10:27 +0000 (08:40 +0530)] 
TEZ-3784. Submitting very large DAG throws com.google.protobuf.CodedInputStream exception (rbalamohan)

11 months agoTEZ-3785. Add Zhiyuan Yang's public key to KEYS (zhiyuany)
Zhiyuan Yang [Wed, 5 Jul 2017 20:15:47 +0000 (13:15 -0700)] 
TEZ-3785. Add Zhiyuan Yang's public key to KEYS (zhiyuany)

11 months agoTEZ-3274. Vertex with MRInput and broadcast input does not respect slow start (Eric...
Jonathan Eagles [Fri, 30 Jun 2017 20:09:10 +0000 (15:09 -0500)] 
TEZ-3274. Vertex with MRInput and broadcast input does not respect slow start (Eric Badger via jeagles)

11 months agoTEZ-3775. Tez UI: Show DAG context in document title (jeagles)
Jonathan Eagles [Fri, 30 Jun 2017 14:33:15 +0000 (09:33 -0500)] 
TEZ-3775. Tez UI: Show DAG context in document title (jeagles)

11 months agoTEZ-3605. Detect and prune empty partitions for the Ordered case (kshukla)
Kuhu Shukla [Wed, 28 Jun 2017 17:47:37 +0000 (12:47 -0500)] 
TEZ-3605. Detect and prune empty partitions for the Ordered case (kshukla)

11 months agoTEZ-3771. Tez UI: WASB/ADLS counters should be listed on the Tez UI (sree)
Sreenath Somarajapuram [Wed, 28 Jun 2017 14:17:15 +0000 (19:47 +0530)] 
TEZ-3771. Tez UI: WASB/ADLS counters should be listed on the Tez UI (sree)

11 months agoTEZ-3769. Unordered: Fix wrong stats being sent out in the last event, when final...
Rajesh Balamohan [Wed, 28 Jun 2017 08:07:09 +0000 (13:37 +0530)] 
TEZ-3769. Unordered: Fix wrong stats being sent out in the last event, when final merge is disabled (rbalamohan)

11 months agoTEZ-3777. Avoid buffer copies by passing RLE flag to TezMerger from PipelinedSorter...
Rajesh Balamohan [Wed, 28 Jun 2017 07:19:04 +0000 (12:49 +0530)] 
TEZ-3777. Avoid buffer copies by passing RLE flag to TezMerger from PipelinedSorter (rbalamohan)

11 months agoTEZ-3767. Shuffle should not report error to AM during inputContext.killSelf(). ...
Rajesh Balamohan [Wed, 28 Jun 2017 00:54:29 +0000 (06:24 +0530)] 
TEZ-3767. Shuffle should not report error to AM during inputContext.killSelf(). (rbalamohan)

11 months agoTEZ-3778. Remove SecurityInfo from tez-auxservices shaded jar (Kuhu Shukla via jeagles)
Jonathan Eagles [Tue, 27 Jun 2017 19:43:17 +0000 (14:43 -0500)] 
TEZ-3778. Remove SecurityInfo from tez-auxservices shaded jar (Kuhu Shukla via jeagles)

11 months agoTEZ-3762. When final merge is disabled in unordered case, it should create index...
Zhiyuan Yang [Mon, 26 Jun 2017 22:21:46 +0000 (15:21 -0700)] 
TEZ-3762. When final merge is disabled in unordered case, it should create index file instead of relying on cache (zhiyuany)

12 months agoTEZ-3761. NPE in Fetcher under load (jeagles)
Jonathan Eagles [Fri, 23 Jun 2017 14:56:00 +0000 (09:56 -0500)] 
TEZ-3761. NPE in Fetcher under load (jeagles)

12 months agoTEZ-3758. Vertex can hang in RUNNING state when two task attempts finish very closely...
Jonathan Eagles [Thu, 22 Jun 2017 15:44:08 +0000 (15:44 +0000)] 
TEZ-3758. Vertex can hang in RUNNING state when two task attempts finish very closely and have retroactive failures (Kuhu Shukla via jeagles)

12 months agoTEZ-3768. Test timeout value for TestShuffleHandlerJobs is low (Kuhu Shukla via jeagles)
Jonathan Eagles [Thu, 22 Jun 2017 15:16:24 +0000 (10:16 -0500)] 
TEZ-3768. Test timeout value for TestShuffleHandlerJobs is low (Kuhu Shukla via jeagles)

12 months agoTEZ-3693. ControlledClock is not used. Contributed by Muhammad Samir Khan
Jason Lowe [Tue, 20 Jun 2017 14:45:01 +0000 (09:45 -0500)] 
TEZ-3693. ControlledClock is not used. Contributed by Muhammad Samir Khan

12 months agoTEZ-3766. Tez Aux-services : Clean up shaded jar to not include default config xml...
Jonathan Eagles [Mon, 19 Jun 2017 23:48:15 +0000 (18:48 -0500)] 
TEZ-3766. Tez Aux-services : Clean up shaded jar to not include default config xml files and yarn-client pieces (Kuhu Shukla via jeagles)

12 months agoTEZ-3760. Tez AUX Services: Shading needs to filter SIG files with -Pazure builds...
Kuhu Shukla [Mon, 19 Jun 2017 19:59:08 +0000 (14:59 -0500)] 
TEZ-3760. Tez AUX Services: Shading needs to filter SIG files with -Pazure builds (kshukla)

12 months agoTEZ-3741. Tez outputs should free memory when closed (Jason Lowe via kshukla)
Kuhu Shukla [Mon, 19 Jun 2017 13:38:27 +0000 (08:38 -0500)] 
TEZ-3741. Tez outputs should free memory when closed (Jason Lowe via kshukla)

12 months agoTEZ-3698: UnorderedKV writer should be able to honor tez.runtime.enable.final-merge...
Rajesh Balamohan [Tue, 13 Jun 2017 03:06:43 +0000 (08:36 +0530)] 
TEZ-3698: UnorderedKV writer should be able to honor tez.runtime.enable.final-merge.in.output without pipelinedshuffle (rbalamohan)

12 months agoTEZ-3732. Reduce Object size of InputAttemptIdentifier and MapOutput for large jobs...
Kuhu Shukla [Fri, 2 Jun 2017 22:43:54 +0000 (17:43 -0500)] 
TEZ-3732. Reduce Object size of InputAttemptIdentifier and MapOutput for large jobs (Jonathan Eagles via kshukla)

12 months agoTEZ-3745. Change master to required java 8 (Siddharth Seth via jeagles)
Jonathan Eagles [Fri, 2 Jun 2017 16:05:28 +0000 (11:05 -0500)] 
TEZ-3745. Change master to required java 8 (Siddharth Seth via jeagles)

12 months agoTEZ-3701. UnorderedPartitionedKVWriter - issues with parallel Deflater usage, synchro...
Rajesh Balamohan [Fri, 2 Jun 2017 01:58:58 +0000 (07:28 +0530)] 
TEZ-3701. UnorderedPartitionedKVWriter - issues with parallel Deflater usage, synchronousqueue in threadpool (rbalamohan)

12 months agoTEZ-3750. Add TEZ_RUNTIME_UNORDERED_PARTITIONED_KVWRITER_BUFFER_MERGE_PERCENT to...
Harish JP [Fri, 2 Jun 2017 01:34:06 +0000 (07:04 +0530)] 
TEZ-3750. Add TEZ_RUNTIME_UNORDERED_PARTITIONED_KVWRITER_BUFFER_MERGE_PERCENT to UnorderedPartitionedKVOutput. (harishjp)

12 months agoTEZ-3749. Get map and reduce task memory from JobConf. (harishjp)
Harish JP [Fri, 2 Jun 2017 01:31:15 +0000 (07:01 +0530)] 
TEZ-3749. Get map and reduce task memory from JobConf. (harishjp)

12 months agoTEZ-3748. TaskAttemptImpl State Machine Invalid event: TA_SUBMITTED at KILL_IN_PROGRE...
Jonathan Eagles [Thu, 1 Jun 2017 01:38:59 +0000 (20:38 -0500)] 
TEZ-3748. TaskAttemptImpl State Machine Invalid event: TA_SUBMITTED at KILL_IN_PROGRESS (jeagles)

12 months agoTEZ-3739. Fair CartesianProduct doesn't works well with huge difference in output...
Zhiyuan Yang [Wed, 31 May 2017 20:49:40 +0000 (13:49 -0700)] 
TEZ-3739. Fair CartesianProduct doesn't works well with huge difference in output size (zhiyuany)

12 months agoTEZ-3713. Allow dag level deletion in cases where containers are reused (Kuhu Shukla...
Jonathan Eagles [Wed, 31 May 2017 19:54:37 +0000 (14:54 -0500)] 
TEZ-3713. Allow dag level deletion in cases where containers are reused (Kuhu Shukla via jeagles)

12 months agoTEZ-3714. Tez UI: Hive Queries page: Use Dag ID and App ID if they are published...
Sreenath Somarajapuram [Wed, 31 May 2017 19:30:13 +0000 (01:00 +0530)] 
TEZ-3714. Tez UI: Hive Queries page: Use Dag ID and App ID if they are published form Hive side. (sree)

12 months agoTEZ-3747. TezConstants.TEZ_SHUFFLE_HANDLER_SERVICE_ID is referenced in Hive (kshukla)
Kuhu Shukla [Wed, 31 May 2017 17:27:20 +0000 (12:27 -0500)] 
TEZ-3747. TezConstants.TEZ_SHUFFLE_HANDLER_SERVICE_ID is referenced in Hive (kshukla)

12 months agoTEZ-3712. Use Local FileContext for deleting dag level directories (Kuhu Shukla via...
Jonathan Eagles [Tue, 30 May 2017 20:42:04 +0000 (15:42 -0500)] 
TEZ-3712. Use Local FileContext for deleting dag level directories (Kuhu Shukla via jeagles)

12 months agoTEZ-3633. Implement keep-alive timeout in tez shuffle handler (jeagles)
Jonathan Eagles [Tue, 30 May 2017 19:22:24 +0000 (14:22 -0500)] 
TEZ-3633. Implement keep-alive timeout in tez shuffle handler (jeagles)

12 months agoTEZ-3736. SubmittedDAGs is always 0 in Resource Manager UI. Contributed by Artem...
Siddharth Seth [Tue, 30 May 2017 18:05:48 +0000 (11:05 -0700)] 
TEZ-3736. SubmittedDAGs is always 0 in Resource Manager UI. Contributed by Artem Velykorodnyi.

12 months agoTEZ-3744. Fix findbugs warnings after TEZ-3334 merge (Kuhu Shukla via jeagles)
Jonathan Eagles [Thu, 25 May 2017 20:40:59 +0000 (15:40 -0500)] 
TEZ-3744. Fix findbugs warnings after TEZ-3334 merge (Kuhu Shukla via jeagles)

12 months agoTEZ-3743. TestTaskCommunicatorContextImpl throws NullPointerException after TEZ-3334...
Kuhu Shukla [Thu, 25 May 2017 19:23:53 +0000 (14:23 -0500)] 
TEZ-3743. TestTaskCommunicatorContextImpl throws NullPointerException after TEZ-3334 merge (Jonathan Eagles via kshukla)

12 months agoTEZ-3742. Fix AMContainerHelpers#createCommonContainerLaunchContext to not pass local...
Jonathan Eagles [Thu, 25 May 2017 15:48:49 +0000 (10:48 -0500)] 
TEZ-3742. Fix AMContainerHelpers#createCommonContainerLaunchContext to not pass localResources. (Kuhu Shukla via jeagles)

13 months agoTEZ-3740. Clean up TEZ-3334-CHANGES.txt (Jonathan Eagles via kshukla)
Kuhu Shukla [Wed, 24 May 2017 21:38:53 +0000 (16:38 -0500)] 
TEZ-3740. Clean up TEZ-3334-CHANGES.txt (Jonathan Eagles via kshukla)

13 months agoTEZ-3735. Test failures in TestTaskAttempt and TestAMContainerMap (Kuhu Shukla via...
Jonathan Eagles [Wed, 24 May 2017 21:20:18 +0000 (16:20 -0500)] 
TEZ-3735. Test failures in TestTaskAttempt and TestAMContainerMap (Kuhu Shukla via jeagles)

13 months agoMerge branch 'master' into TEZ-3334-MERGE1
Jonathan Eagles [Wed, 24 May 2017 20:56:02 +0000 (15:56 -0500)] 
Merge branch 'master' into TEZ-3334-MERGE1

13 months agoTEZ-3737. FairCartesianProductVertexMananger used incorrect #partition (zhiyuany)
Zhiyuan Yang [Wed, 24 May 2017 19:33:03 +0000 (12:33 -0700)] 
TEZ-3737. FairCartesianProductVertexMananger used incorrect #partition (zhiyuany)

13 months agoTEZ-3691. Setup fetchers to use shared executor. (harishjp)
Harish JP [Wed, 24 May 2017 06:26:09 +0000 (11:56 +0530)] 
TEZ-3691. Setup fetchers to use shared executor. (harishjp)

13 months agoTEZ-3007. Use AppFinalState.ENDED when unregistering with the RM in session mode...
Harish JP [Wed, 24 May 2017 06:09:20 +0000 (11:39 +0530)] 
TEZ-3007. Use AppFinalState.ENDED when unregistering with the RM in session mode (harishjp)

13 months agoTEZ-3334 merge conflict
Jonathan Eagles [Fri, 19 May 2017 20:04:29 +0000 (15:04 -0500)] 
TEZ-3334 merge conflict

13 months agoMerge branch 'master' into TEZ-3334-MERGE1
Jonathan Eagles [Fri, 19 May 2017 19:57:08 +0000 (14:57 -0500)] 
Merge branch 'master' into TEZ-3334-MERGE1

13 months agoMerge branch 'TEZ-3334' into TEZ-3334-MERGE1
Jonathan Eagles [Fri, 19 May 2017 19:56:37 +0000 (14:56 -0500)] 
Merge branch 'TEZ-3334' into TEZ-3334-MERGE1

13 months agoTEZ-3662. Vertex Duration in 0.9 Tez UI regression from 0.7 (jeagles)
Jonathan Eagles [Fri, 19 May 2017 19:28:49 +0000 (14:28 -0500)] 
TEZ-3662. Vertex Duration in 0.9 Tez UI regression from 0.7 (jeagles)

13 months agoTEZ-3730. Lower logging level in UnorderedPartitionedKVWriter. Contributed by Prasan...
Siddharth Seth [Fri, 19 May 2017 17:34:38 +0000 (10:34 -0700)] 
TEZ-3730. Lower logging level in UnorderedPartitionedKVWriter.  Contributed by Prasanth Jayachandran.

13 months agoTEZ-3726. Clean up DeletionTracker's reflection instantiation and provide ContainerLa... TEZ-3334
Kuhu Shukla [Wed, 17 May 2017 15:28:44 +0000 (10:28 -0500)] 
TEZ-3726. Clean up DeletionTracker's reflection instantiation and provide ContainerLauncher with dagComplete() functionality (kshukla)

13 months agoTEZ-3723. TezIndexRecord#hasData() returns true for empty index record in the Unorder...
Kuhu Shukla [Tue, 16 May 2017 21:25:33 +0000 (16:25 -0500)] 
TEZ-3723. TezIndexRecord#hasData() returns true for empty index record in the Unordered case (kshukla)

13 months agoTEZ-3731. Add committer kshukla to Tez Team List (kshukla)
Kuhu Shukla [Tue, 16 May 2017 20:45:39 +0000 (15:45 -0500)] 
TEZ-3731. Add committer kshukla to Tez Team List (kshukla)

13 months agoTEZ-3725. Cleanup http connections and other unnecessary fields in DAG Deletion track...
Kuhu Shukla [Tue, 16 May 2017 17:40:56 +0000 (12:40 -0500)] 
TEZ-3725. Cleanup http connections and other unnecessary fields in DAG Deletion tracker classes. (kshukla)

13 months agoTEZ-3709. TezMerger is slow for high number of segments (jeagles)
Jonathan Eagles [Tue, 16 May 2017 15:16:13 +0000 (10:16 -0500)] 
TEZ-3709. TezMerger is slow for high number of segments (jeagles)

13 months agoTEZ-3716. Allow attempt retries to be treated the same as the first
Siddharth Seth [Sun, 14 May 2017 17:27:01 +0000 (10:27 -0700)] 
TEZ-3716. Allow attempt retries to be treated the same as the first
attempt. (sseth)

13 months agoTEZ-3715. Differentiate between TaskAttempt submission and TaskAttempt
Siddharth Seth [Sat, 13 May 2017 00:05:47 +0000 (17:05 -0700)] 
TEZ-3715. Differentiate between TaskAttempt submission and TaskAttempt
started. (sseth)

13 months agoTEZ-3719. DAGImpl.computeProgress slows down dispatcher and ipc threads (Gopal V...
Jonathan Eagles [Fri, 12 May 2017 14:51:00 +0000 (09:51 -0500)] 
TEZ-3719. DAGImpl.computeProgress slows down dispatcher and ipc threads (Gopal V via jeagles)

13 months agoTEZ-3708. Improve parallelism and auto grouping of unpartitioned cartesian product...
Zhiyuan Yang [Thu, 11 May 2017 22:19:39 +0000 (15:19 -0700)] 
TEZ-3708. Improve parallelism and auto grouping of unpartitioned cartesian product (zhiyuany)

13 months agoTEZ-3717. tez-yarn-timeline-history-with-fs does not build with hadoop-2.8. (harishjp)
Harish JP [Wed, 10 May 2017 04:02:23 +0000 (09:32 +0530)] 
TEZ-3717. tez-yarn-timeline-history-with-fs does not build with hadoop-2.8. (harishjp)

13 months agoTEZ-3700 addendum. Consumer attempt should kill itself instead of
Siddharth Seth [Mon, 8 May 2017 18:35:24 +0000 (11:35 -0700)] 
TEZ-3700 addendum. Consumer attempt should kill itself instead of
failing during validation checks with final merge avoidance (rbalamohan)

13 months agoTEZ-3697. Adding #output_record in vertex manager event payload (zhiyuany)
Zhiyuan Yang [Mon, 8 May 2017 07:18:14 +0000 (00:18 -0700)] 
TEZ-3697. Adding #output_record in vertex manager event payload (zhiyuany)

13 months agoTEZ-3673. Allocate smaller buffers in UnorderedPartitionedKVWriter. (harishjp)
Harish JP [Fri, 5 May 2017 10:12:42 +0000 (15:42 +0530)] 
TEZ-3673. Allocate smaller buffers in UnorderedPartitionedKVWriter. (harishjp)

13 months agoMerge remote-tracking branch 'origin/master' into TEZ-3334-MERGE1
Jonathan Eagles [Thu, 4 May 2017 21:40:16 +0000 (16:40 -0500)] 
Merge remote-tracking branch 'origin/master' into TEZ-3334-MERGE1

13 months agoMerge remote-tracking branch 'origin/TEZ-3334' into TEZ-3334-MERGE1
Jonathan Eagles [Thu, 4 May 2017 20:52:23 +0000 (15:52 -0500)] 
Merge remote-tracking branch 'origin/TEZ-3334' into TEZ-3334-MERGE1

13 months agoTEZ-3705. Modify DeletionTracker and deletion threads to be initialized only if enabl...
Jonathan Eagles [Thu, 4 May 2017 20:33:23 +0000 (15:33 -0500)] 
TEZ-3705. Modify DeletionTracker and deletion threads to be initialized only if enabled for tez_shuffle (Kuhu Shukla via jeagles)

13 months agoTEZ-3696. Jobs can hang when both concurrency and speculation are
Siddharth Seth [Thu, 4 May 2017 18:35:05 +0000 (11:35 -0700)] 
TEZ-3696. Jobs can hang when both concurrency and speculation are
enabled. Contributed by Eric Badger.

13 months agoTEZ-3707. TezSharedExecutor race condition in awaitTermination vs isTerminated. ...
Harish JP [Thu, 4 May 2017 03:54:39 +0000 (09:24 +0530)] 
TEZ-3707. TezSharedExecutor race condition in awaitTermination vs isTerminated. (harishjp)

13 months agoTEZ-3700. Consumer attempt should kill itself instead of failing during validation...
Rajesh Balamohan [Wed, 3 May 2017 00:42:51 +0000 (06:12 +0530)] 
TEZ-3700. Consumer attempt should kill itself instead of failing during validation checks with final merge avoidance (rbalamohan)

13 months agoTEZ-3703. Use a sha comparison to compare vertex and dag resources, if
Siddharth Seth [Mon, 1 May 2017 18:58:35 +0000 (11:58 -0700)] 
TEZ-3703. Use a sha comparison to compare vertex and dag resources, if
there is a mismatch. (sseth)

13 months agoTEZ-3702. Tez shuffle jar includes service loader entry for ClientProtocolProvider...
Jason Lowe [Thu, 27 Apr 2017 18:49:16 +0000 (13:49 -0500)] 
TEZ-3702. Tez shuffle jar includes service loader entry for ClientProtocolProvider but not the corresponding class (jlowe)

13 months agoTEZ-3690. addendum patch. Tez on hadoop 3 build failed due to hdfs client/server...
Zhiyuan Yang [Thu, 27 Apr 2017 07:02:31 +0000 (00:02 -0700)] 
TEZ-3690. addendum patch. Tez on hadoop 3 build failed due to hdfs client/server jar separation (zhiyuany)

13 months agoTEZ-3699. For large dataset, pipelined shuffle throws exceptions in consumer side...
Rajesh Balamohan [Wed, 26 Apr 2017 23:43:33 +0000 (05:13 +0530)] 
TEZ-3699. For large dataset, pipelined shuffle throws exceptions in consumer side for UnorderedPartitioned edge (rbalamohan)

13 months agoTEZ-3685. ShuffleHandler completedInputSet off-by-one error (jeagles)
Jonathan Eagles [Tue, 25 Apr 2017 20:34:59 +0000 (15:34 -0500)] 
TEZ-3685. ShuffleHandler completedInputSet off-by-one error (jeagles)

13 months agoTEZ-3695. TestTezSharedExecutor fails sporadically. (jlowe)
Jason Lowe [Tue, 25 Apr 2017 14:55:34 +0000 (09:55 -0500)] 
TEZ-3695. TestTezSharedExecutor fails sporadically. (jlowe)

14 months agoTEZ-2049. Remove YARN references from Tez AsyncDispatcher (zhiyuany)
Zhiyuan Yang [Mon, 24 Apr 2017 23:58:42 +0000 (16:58 -0700)] 
TEZ-2049. Remove YARN references from Tez AsyncDispatcher (zhiyuany)

14 months agoTEZ-3675. Handle changes to ResourceCalculatorProcessTree in YARN-3427 for Hadoop...
Zhiyuan Yang [Fri, 21 Apr 2017 22:13:08 +0000 (15:13 -0700)] 
TEZ-3675. Handle changes to ResourceCalculatorProcessTree in YARN-3427 for Hadoop 3.x (zhiyuany)

14 months agoTEZ-3690. Tez on hadoop 3 build failed due to hdfs client/server jar separation ...
Zhiyuan Yang [Fri, 21 Apr 2017 22:10:44 +0000 (15:10 -0700)] 
TEZ-3690. Tez on hadoop 3 build failed due to hdfs client/server jar separation (Junping Du via zhiyuany)

14 months agoTEZ-3689. Change minimum hadoop version to 2.7.0. (sseth)
Siddharth Seth [Thu, 20 Apr 2017 23:31:57 +0000 (16:31 -0700)] 
TEZ-3689. Change minimum hadoop version to 2.7.0. (sseth)

14 months agoTEZ-3687. Code smell in DAGStatus and VertexStatus equals implementation (Akira Ajisa...
Zhiyuan Yang [Thu, 20 Apr 2017 19:57:12 +0000 (12:57 -0700)] 
TEZ-3687. Code smell in DAGStatus and VertexStatus equals implementation (Akira Ajisaka via zhiyuany)

14 months agoTEZ-3611. Create lightweight summary events for ATS. (harishjp)
Harish JP [Thu, 20 Apr 2017 10:12:16 +0000 (15:42 +0530)] 
TEZ-3611. Create lightweight summary events for ATS. (harishjp)

14 months agoTEZ-3631. Tez UI: TEZ_DAG_EXTRA_INFO compatibility changes - Makes All DAGs page...
Sreenath Somarajapuram [Thu, 20 Apr 2017 09:57:19 +0000 (15:27 +0530)] 
TEZ-3631. Tez UI: TEZ_DAG_EXTRA_INFO compatibility changes - Makes All DAGs page faster (sree)

14 months agoTEZ-3680. Optimizations to UnorderedPartitionedKVWriter (rbalamohan)
Rajesh Balamohan [Thu, 20 Apr 2017 09:19:20 +0000 (14:49 +0530)] 
TEZ-3680. Optimizations to UnorderedPartitionedKVWriter (rbalamohan)

14 months agoTEZ-1187. Add a framework ExecutorService which shares threads (harishjp)
Harish JP [Thu, 20 Apr 2017 09:15:44 +0000 (14:45 +0530)] 
TEZ-1187. Add a framework ExecutorService which shares threads (harishjp)

14 months agoTEZ-3616. TestMergeManager#testLocalDiskMergeMultipleTasks fails intermittently ...
Harish JP [Wed, 19 Apr 2017 15:44:01 +0000 (21:14 +0530)] 
TEZ-3616. TestMergeManager#testLocalDiskMergeMultipleTasks fails intermittently (Fei Hui via harishjp)

14 months agoTEZ-3654. Make CartesianProduct edge work with GroupInputEdge (zhiyuany)
Zhiyuan Yang [Mon, 10 Apr 2017 18:18:33 +0000 (11:18 -0700)] 
TEZ-3654. Make CartesianProduct edge work with GroupInputEdge (zhiyuany)

14 months agoTEZ-3285. Tez UI: Lock down dependency versions (sree)
Sreenath Somarajapuram [Mon, 10 Apr 2017 09:37:41 +0000 (15:07 +0530)] 
TEZ-3285. Tez UI: Lock down dependency versions (sree)

14 months agoTEZ-3684. Incorporate first pass non-essential TEZ-3334 pre-merge feedback (jeagles)
Jonathan Eagles [Thu, 6 Apr 2017 18:06:31 +0000 (13:06 -0500)] 
TEZ-3684. Incorporate first pass non-essential TEZ-3334 pre-merge feedback (jeagles)

14 months agoTEZ-3683. LocalContainerLauncher#shouldDelete member variable is not used (Kuhu Shukl...
Jonathan Eagles [Thu, 6 Apr 2017 14:22:26 +0000 (09:22 -0500)] 
TEZ-3683. LocalContainerLauncher#shouldDelete member variable is not used (Kuhu Shukla via jeagles)

14 months agoTEZ-3682. Pass parameters instead of configuration for changes to support tez shuffle...
Jonathan Eagles [Thu, 6 Apr 2017 14:14:49 +0000 (09:14 -0500)] 
TEZ-3682. Pass parameters instead of configuration for changes to support tez shuffle handler (jeagles)

14 months agoTEZ-3681. Improve UI error message while trying to sort running DAGs with Auto Refres...
Sreenath Somarajapuram [Wed, 5 Apr 2017 15:36:06 +0000 (21:06 +0530)] 
TEZ-3681. Improve UI error message while trying to sort running DAGs with Auto Refresh enabled (Eric Badger via sree)

14 months agoTEZ-3652. Remove ShuffleClientMetrics (zhiyuany)
Zhiyuan Yang [Mon, 3 Apr 2017 18:21:17 +0000 (11:21 -0700)] 
TEZ-3652. Remove ShuffleClientMetrics (zhiyuany)

14 months agoRevert "TEZ-3642. Remove ShuffleClientMetrics (zhiyuany)"
Zhiyuan Yang [Mon, 3 Apr 2017 18:18:42 +0000 (11:18 -0700)] 
Revert "TEZ-3642. Remove ShuffleClientMetrics (zhiyuany)"

This reverts commit f7f60385cf5d8d66a1ee02e64d1e671bf1ad8771.