SAMZA-1434: Fix issues found in Hadoop 0.14.0
authorXinyu Liu <xinyuliu.us@gmail.com>
Fri, 29 Sep 2017 22:05:55 +0000 (15:05 -0700)
committerXinyu Liu <xiliu@xiliu-ld1.linkedin.biz>
Fri, 29 Sep 2017 22:05:55 +0000 (15:05 -0700)
commita1f01444ec12f49684213cc69b1cce16ff0f8232
tree888542467ca80011e8b598cc6d710bbf3cdecabd
parent2819cbc7691b569b3eef66d702746d9e34b3e745
SAMZA-1434: Fix issues found in Hadoop

Fix the following bugs found when running Samza on hadoop:

1. Hdfs allows output partitions to be 0 (empty folder)
2. Add null check for the changelog topic generation
3. Call getStreamSpec() instead of using streamSpec member in StreamEdge. This is due to getStreamSpec will do more transformation.
4. Bound the auto-generated intermediate topic partition by a certain count (256).

Author: Xinyu Liu <xiliu@xiliu-ld1.linkedin.biz>

Reviewers: Jagadish Venkatraman <jagadish@apache.org>

Closes #307 from xinyuiscool/SAMZA-1434
samza-api/src/main/java/org/apache/samza/system/StreamSpec.java
samza-core/src/main/java/org/apache/samza/config/JavaStorageConfig.java
samza-core/src/main/java/org/apache/samza/execution/ExecutionPlanner.java
samza-core/src/main/java/org/apache/samza/execution/StreamEdge.java