samza.git
15 hours agoUpdated API documentation for high and low level APIs. master
Prateek Maheshwari [Wed, 14 Nov 2018 02:17:17 +0000 (18:17 -0800)] 
Updated API documentation for high and low level APIs.

vjagadish1989 nickpan47 Please take a look.

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Reviewers: Jagadish<jagadish@apache.org>

Closes #802 from prateekm/api-docs

45 hours agoSAMZA-1986: Samza-sql: Use system name along with stream name for streamId
Aditya Toomula [Mon, 12 Nov 2018 19:42:51 +0000 (11:42 -0800)] 
SAMZA-1986: Samza-sql: Use system name along with stream name for streamId

Author: Aditya Toomula <atoomula@linkedin.com>

Reviewers: Srinivasulu Punuru <spunuru@linkedin.com>

Closes #800 from atoomula/streamid

46 hours agoSAMZA-1988: Properly suffix modules with direct Scala dependencies with the Scala...
Daniel Nishimura [Mon, 12 Nov 2018 19:11:28 +0000 (11:11 -0800)] 
SAMZA-1988: Properly suffix modules with direct Scala dependencies with the Scala version.

List of modules without a Scala version suffix that have direct Scala dependencies and the direct Scala API calls in each module are in the JIRA ticket: https://issues.apache.org/jira/browse/SAMZA-1988

Author: Daniel Nishimura <dnishimura@linkedin.com>

Reviewers: Sanil Jain <snjain@linkedin.com>

Closes #801 from dnishimura/samza-1988-scala-version-suffixes

4 days agoSAMZA-1978: Use samza offset reset value in kafka consumer
Boris S [Sat, 10 Nov 2018 00:23:05 +0000 (16:23 -0800)] 
SAMZA-1978: Use samza offset reset value in kafka consumer

Author: Boris S <bshkolnik@linkedin.com>
Author: Boris S <boryas@apache.org>
Author: Boris Shkolnik <bshkolni@linkedin.com>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #753 from sborya/UseSamazResetInKafka

5 days agoSAMZA-1616: Samza-Sql - Support remote table for stream-table join
Aditya Toomula [Fri, 9 Nov 2018 16:30:59 +0000 (08:30 -0800)] 
SAMZA-1616: Samza-Sql - Support remote table for stream-table join

Author: Aditya Toomula <atoomula@linkedin.com>

Reviewers: Srinivasulu Punuru <spunuru@linkedin.com>

Closes #794 from atoomula/remote

5 days agoSAMZA-1981: Consolidate table descriptors to samza-api
Wei Song [Thu, 8 Nov 2018 22:04:28 +0000 (14:04 -0800)] 
SAMZA-1981: Consolidate table descriptors to samza-api

As per subject, table descriptors moved are
 - LocalTableDescriptor
 - RemoteTableDescriptor
 - HybridTableDescriptor
 - GuavaCacheTableDescriptor
 - CachingTableDescriptor

Author: Wei Song <wsong@linkedin.com>

Reviewers: Prateek Maheshwari <pmaheshwari@linkedin.com>

Closes #799 from weisong44/SAMZA-1981

6 days agoSAMZA-1980: Rename LocalStoreBackedTable to LocalTable
Wei Song [Wed, 7 Nov 2018 22:46:17 +0000 (14:46 -0800)] 
SAMZA-1980: Rename LocalStoreBackedTable to LocalTable

As per subject, this is to keep naming of local tables consistent with other table types.

Author: Wei Song <wsong@linkedin.com>

Reviewers: Prateek Maheshwari <pmaheshwari@linkedin.com>

Closes #798 from weisong44/SAMZA-1980 and squashes the following commits:

51c17ff4 [Wei Song] Renamed LocalStoreBackedTable to LocalTable
9c121207 [Wei Song] Merge remote-tracking branch 'upstream/master'
89bfc14c [Wei Song] Merge remote-tracking branch 'upstream/master'
a53e5628 [Wei Song] SAMZA-1964 Make getTableSpec() in RemoteTableDescriptor reentrant
c9e8bf7c [Wei Song] Merge remote-tracking branch 'upstream/master'
7c777fec [Wei Song] Merge remote-tracking branch 'upstream/master'
a06e8ec2 [Wei Song] Merge remote-tracking branch 'upstream/master'
2c679c39 [Wei Song] Merge remote-tracking branch 'upstream/master'
a56c28dc [Wei Song] Merge remote-tracking branch 'upstream/master'
097958c8 [Wei Song] Merge remote-tracking branch 'upstream/master'
05822f0a [Wei Song] Merge remote-tracking branch 'upstream/master'
f7480505 [Wei Song] Merge remote-tracking branch 'upstream/master'
7706ab1f [Wei Song] Merge remote-tracking branch 'upstream/master'
f5731b10 [Wei Song] Merge remote-tracking branch 'upstream/master'
1e5de45a [Wei Song] Merge remote-tracking branch 'upstream/master'
c85604e0 [Wei Song] Merge remote-tracking branch 'upstream/master'
242d8442 [Wei Song] Merge remote-tracking branch 'upstream/master'
ec7d8409 [Wei Song] Merge remote-tracking branch 'upstream/master'
e19b4dc9 [Wei Song] Merge remote-tracking branch 'upstream/master'
8ee78441 [Wei Song] Merge remote-tracking branch 'upstream/master'
1c6a2eae [Wei Song] Merge remote-tracking branch 'upstream/master'
a6c94add [Wei Song] Merge remote-tracking branch 'upstream/master'
41299b5b [Wei Song] Merge remote-tracking branch 'upstream/master'
239a0950 [Wei Song] Merge remote-tracking branch 'upstream/master'
eca00204 [Wei Song] Merge remote-tracking branch 'upstream/master'
51562391 [Wei Song] Merge remote-tracking branch 'upstream/master'
de708f5e [Wei Song] Merge remote-tracking branch 'upstream/master'
df2f8d7b [Wei Song] Merge remote-tracking branch 'upstream/master'
f28b491d [Wei Song] Merge remote-tracking branch 'upstream/master'
4782c61d [Wei Song] Merge remote-tracking branch 'upstream/master'
0440f75f [Wei Song] Merge remote-tracking branch 'upstream/master'
aae0f380 [Wei Song] Merge remote-tracking branch 'upstream/master'
a15a7c9a [Wei Song] Merge remote-tracking branch 'upstream/master'
5cbf9af9 [Wei Song] Merge remote-tracking branch 'upstream/master'
3f7ed71f [Wei Song] Added self to committer list

6 days agoSAMZA-1921: upgrade to use the latest java AdminClient.
Boris S [Wed, 7 Nov 2018 22:39:59 +0000 (14:39 -0800)] 
SAMZA-1921: upgrade to use the latest java AdminClient.

In this PR, I've refactored create/clear streams methods.

Author: Boris S <bshkolnik@linkedin.com>
Author: Boris S <boryas@apache.org>
Author: Boris Shkolnik <bshkolni@linkedin.com>
Author: svenkata <svenkataraman@linkedin.com>

Reviewers: Shanthoosh Venkataraman <svenkataraman@linkedin.com>

Closes #789 from sborya/JavaAdminClient

11 days agoCleanup docs for HDFS connector
Jagadish [Sat, 3 Nov 2018 00:35:20 +0000 (17:35 -0700)] 
Cleanup docs for HDFS connector

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #793 from vjagadish1989/website-reorg30

12 days agoSAMZA-1952: StreamPartitionCountMonitor for standalone.
Shanthoosh Venkataraman [Fri, 2 Nov 2018 16:38:30 +0000 (09:38 -0700)] 
SAMZA-1952: StreamPartitionCountMonitor for standalone.

This patch adds the capability to detect the partition change of the input streams of a stateless standalone jobs and trigger a re-balancing phase(which will essentially account for new partitions from input stream and distribute it to the live processors of the group).

Existing partition count detection of input streams is broken in yarn for stateful jobs. This will be addressed for both yarn and standalone as a part of #622

Author: Shanthoosh Venkataraman <spvenkat@usc.edu>

Reviewers: Boris Shkolnik <boryas@apache.org>

Closes #726 from shanthoosh/stream_partition_count_monitor_for_standalone

13 days agoy
Boris S [Wed, 31 Oct 2018 21:22:04 +0000 (14:22 -0700)] 
y

Author: Boris S <boryas@apache.org>
Author: Boris S <bshkolnik@linkedin.com>
Author: Boris Shkolnik <bshkolni@linkedin.com>

Reviewers: Ray Matharu <rmatharu@linkedin.com>

Closes #779 from sborya/RemoveGetKafkaSystemConsumerConfig

13 days agoSAMZA-1943 Remove ExtendedSystemAdmin and deprecated getNewestOffsets method.
Boris S [Wed, 31 Oct 2018 20:45:28 +0000 (13:45 -0700)] 
SAMZA-1943 Remove ExtendedSystemAdmin and deprecated getNewestOffsets method.

Author: Boris S <boryas@apache.org>
Author: Boris S <bshkolnik@linkedin.com>
Author: Boris Shkolnik <bshkolni@linkedin.com>

Reviewers: Bharath Kumarasubramanian <bkumarasubramanian@linkedin.com>

Closes #782 from sborya/removeExtendedSystemAdmin

13 days agoJavadoc cleanup for new Application, Descriptor, Context and Table APIs - Part 2
Prateek Maheshwari [Wed, 31 Oct 2018 20:36:54 +0000 (13:36 -0700)] 
Javadoc cleanup for new Application, Descriptor, Context and Table APIs - Part 2

Currently, we don't allow imports for use only in javadocs. This requires using FQNs in link tags, which is not very readable. Checkstyle's UnusedImport rule has an option to allow imports for use in javadoc comments (processJavadocs=true, should be read as "check javadocs for import usage == true").

AFAICT, there's no good way to change the check's properties within a submodule. This PR adds both versions (strict and relaxed) to the checkstyle, and disables the strict validation for samza-api only.

This PR also updates the javadocs to use the class names with imports.

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Reviewers: Cameron Lee <calee@linkedin.com>

Closes #760 from prateekm/javadoc-cleanup

13 days agoSAMZA-1970: Support for physical names in InMemorySystem
Sanil15 [Wed, 31 Oct 2018 19:41:40 +0000 (12:41 -0700)] 
SAMZA-1970: Support for physical names in InMemorySystem

if super is not there, java compiles this to this.withPhysicalName which results in StackOverflows

Author: Sanil15 <sanil.jain15@gmail.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #788 from Sanil15/SAMZA-1970-edit

13 days agoSAMZA-1971: Fix NPE in partition key computation for InMemorySystemProducer
bharathkk [Wed, 31 Oct 2018 19:41:20 +0000 (12:41 -0700)] 
SAMZA-1971: Fix NPE in partition key computation for InMemorySystemProducer

Author: bharathkk <codin.martial@gmail.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #786 from bharathkk/fix-inmemory-partitionkey-npe

2 weeks agoClose iterators to time-series store on deletes
Jagadish [Wed, 31 Oct 2018 01:48:40 +0000 (18:48 -0700)] 
Close iterators to time-series store on deletes

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #787 from vjagadish1989/website-reorg29

2 weeks agoUpdated RELEASE instructions.
Prateek Maheshwari [Tue, 30 Oct 2018 23:29:53 +0000 (16:29 -0700)] 
Updated RELEASE instructions.

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Reviewers: Jagadish <jagadish@apache.org>

Closes #783 from prateekm/release-docs

2 weeks agoAdding release notes for Samza 1.0
rmatharu@linkedin.com [Tue, 30 Oct 2018 22:54:25 +0000 (15:54 -0700)] 
Adding release notes for Samza 1.0

<img width="1280" alt="screen shot 2018-10-26 at 7 25 58 pm" src="https://user-images.githubusercontent.com/40646191/47598776-11921000-d956-11e8-8863-ce245e1d0cb2.png">
<img width="1280" alt="screen shot 2018-10-26 at 7 26 06 pm" src="https://user-images.githubusercontent.com/40646191/47598777-11921000-d956-11e8-884b-3b5acbdbfb94.png">
<img width="1280" alt="screen shot 2018-10-26 at 7 26 08 pm" src="https://user-images.githubusercontent.com/40646191/47598778-11921000-d956-11e8-9428-8a8f11f3d1e8.png">
<img width="1280" alt="screen shot 2018-10-26 at 7 26 10 pm" src="https://user-images.githubusercontent.com/40646191/47598779-11921000-d956-11e8-99ee-9c77349417f2.png">
<img width="1280" alt="screen shot 2018-10-26 at 7 26 13 pm" src="https://user-images.githubusercontent.com/40646191/47598780-122aa680-d956-11e8-9fbc-98527b099a6a.png">
<img width="1280" alt="screen shot 2018-10-26 at 7 26 17 pm" src="https://user-images.githubusercontent.com/40646191/47598781-122aa680-d956-11e8-8c81-b113fbddbd8a.png">

Author: rmatharu@linkedin.com <rmatharu@linkedin.com>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #776 from rmatharu/releasenotes

2 weeks agoFix to make Samza SQL applications work after the Runner refactoring
Srinivasulu Punuru [Tue, 30 Oct 2018 20:33:33 +0000 (13:33 -0700)] 
Fix to make Samza SQL applications work after the Runner refactoring

With recent change in Samza, Constructor signature for ApplicationRunner has changed. But the SamzaSQLApplicationRunner was not updated with the new signature. This is to fix the signature of the constructor for SamzaSQLApplicationRunner with the updated signature.

Author: Srinivasulu Punuru <spunuru@linkedin.com>

Reviewers: Aditya Toomula <atoomula@linkedin.com>

Closes #784 from srinipunuru/sql-app-fix.1

2 weeks agoatoomula and prateekm FYI..
Jagadish [Tue, 30 Oct 2018 19:11:56 +0000 (12:11 -0700)] 
atoomula and prateekm FYI..

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #785 from vjagadish1989/website-reorg28

2 weeks agoSAMZA-1968: Samza-sql - Change Calcite sql type for samza sql rel message __key__...
Aditya Toomula [Tue, 30 Oct 2018 18:25:00 +0000 (11:25 -0700)] 
SAMZA-1968: Samza-sql - Change Calcite sql type for samza sql rel message __key__ to accept any format

Author: Aditya Toomula <atoomula@linkedin.com>

Reviewers: Srinivasulu Punuru <spunuru@linkedin.com>

Closes #774 from atoomula/keyformat

2 weeks agoAdd physical name support for InMemoryStreamDescriptors
Sanil15 [Tue, 30 Oct 2018 02:52:04 +0000 (19:52 -0700)] 
Add physical name support for InMemoryStreamDescriptors

Author: Sanil15 <sanil.jain15@gmail.com>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #781 from Sanil15/SAMZA-1970

2 weeks agoCleanup the EventHubs connector section. Use System/Stream Descriptors
Jagadish [Mon, 29 Oct 2018 23:31:11 +0000 (16:31 -0700)] 
Cleanup the EventHubs connector section. Use System/Stream Descriptors

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #780 from vjagadish1989/website-reorg27

2 weeks agoCleanup the connectors-overview and Kafka-connector sections. Use System/StreamDescri...
Jagadish [Sun, 28 Oct 2018 18:29:07 +0000 (11:29 -0700)] 
Cleanup the connectors-overview and Kafka-connector sections. Use System/StreamDescriptors

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish <jagadish@apache.org>

Closes #778 from vjagadish1989/website-reorg26

2 weeks agoDOCS: Clean-up the section on YARN deployments
Jagadish [Sat, 27 Oct 2018 18:18:08 +0000 (11:18 -0700)] 
DOCS: Clean-up the section on YARN deployments

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #777 from vjagadish1989/website-reorg25

2 weeks agoSAMZA-1967: Tests failing when Job uses any serde other than NoOp
Sanil15 [Sat, 27 Oct 2018 00:51:58 +0000 (17:51 -0700)] 
SAMZA-1967: Tests failing when Job uses any serde other than NoOp

Context: Serde is configured in JobNodeConfigurationGenerator and any StreamDescriptor#toConfig does not generate key and msg serde configs

Problem: Tests failing when Job uses any serde other than NoOp, since ApplicationDescriptor serdes take precedence in absence of any user-supplied configs

Solution: Passing null msg and key serde configs in userConfigs for StreamDescriptors ensures ApplicationDescriptor generated serde configs don't take precedence

prateekm rmatharu please take a look

Author: Sanil15 <sanil.jain15@gmail.com>

Reviewers: Prateek M<pmaheshw@linkedin.com>

Closes #764 from Sanil15/SAMZA-1967

2 weeks agoPrint the logical plan during query planning
Srinivasulu Punuru [Thu, 25 Oct 2018 21:32:38 +0000 (14:32 -0700)] 
Print the logical plan during query planning

Minor fix to print the logical plan.

Author: Srinivasulu Punuru <spunuru@linkedin.com>

Reviewers: Aditya Toomula <atoomula@linkedin.com>

Closes #763 from srinipunuru/print.1

2 weeks agoSAMZA-1903: Samza-sql - Fix stream-table join to work with udfs
Aditya Toomula [Thu, 25 Oct 2018 21:28:19 +0000 (14:28 -0700)] 
SAMZA-1903: Samza-sql - Fix stream-table join to work with udfs

Author: Aditya Toomula <atoomula@linkedin.com>

Reviewers: Srinivasulu Punuru <spunuru@linkedin.com>

Closes #762 from atoomula/udf

2 weeks agoSAMZA-1927: Samza-sql - always repartition the stream denoted by stream-table join.
Aditya Toomula [Thu, 25 Oct 2018 21:16:24 +0000 (14:16 -0700)] 
SAMZA-1927: Samza-sql - always repartition the stream denoted by stream-table join.

Author: Aditya Toomula <atoomula@linkedin.com>

Reviewers: Srinivasulu Punuru <spunuru@linkedin.com>

Closes #676 from atoomula/dsl3 and squashes the following commits:

e86ef83c [Aditya Toomula] Adding metadatastream prefix config. This will be used to reset both the intermediate streams and changelogstore streams by changing the prefix name.
14450264 [Aditya Toomula] Adding metadatastream prefix config. This will be used to reset both the intermediate streams and changelogstore streams by changing the prefix name.
c3289673 [Aditya Toomula] Adding changelogstreamname prefix config. This will be used to reset the state by changing the prefix name.
804b07a1 [Aditya Toomula] SAMZA-1927: Samza-sql - always repartition the stream denoted by stream-table join.

2 weeks agoClean up the deployment options section.
Jagadish [Wed, 24 Oct 2018 22:20:02 +0000 (15:20 -0700)] 
Clean up the deployment options section.

- Reword for consistency, style, tone

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #761 from vjagadish1989/website-reorg24

3 weeks agoClean-up the Quick-Start and Code-Examples pages; Re-organize content
Jagadish [Wed, 24 Oct 2018 06:39:54 +0000 (23:39 -0700)] 
Clean-up the Quick-Start and Code-Examples pages; Re-organize content

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #759 from vjagadish1989/website-reorg23

3 weeks agoTests for deprecated Kafka Consumer
Boris S [Wed, 24 Oct 2018 01:11:43 +0000 (18:11 -0700)] 
Tests for deprecated Kafka Consumer

Tested with running with system set to org.apache.samza.system.kafka_deprecated.KafkaSystemFactory.

Author: Boris S <bshkolnik@linkedin.com>
Author: Boris S <boryas@apache.org>
Author: Boris Shkolnik <bshkolni@linkedin.com>

Reviewers: Cameron Lee <calee@linkedin.com>

Closes #755 from sborya/OldKafkaConsumer

3 weeks agoMinor instruction changes
rmatharu@linkedin.com [Tue, 23 Oct 2018 17:31:20 +0000 (10:31 -0700)] 
Minor instruction changes

Author: rmatharu@linkedin.com <rmatharu@linkedin.com>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #758 from rmatharu/doc

3 weeks agoFix samza-test checkstyle errors.
Shanthoosh Venkataraman [Tue, 23 Oct 2018 15:43:37 +0000 (08:43 -0700)] 
Fix samza-test checkstyle errors.

Author: Shanthoosh Venkataraman <svenkata@linkedin.com>

Reviewers: Yi Pan <nickpan47@gmail.com>

Closes #757 from shanthoosh/master

3 weeks agoChanges to Standalone integration tests
rmatharu@linkedin.com [Tue, 23 Oct 2018 02:04:58 +0000 (19:04 -0700)] 
Changes to Standalone integration tests

* Removing task.inputs since its not required in the new Samza 1.0 API
* Fixing input-output sys bug
* Fixing jobModel zk path bug -- camel case.

Author: rmatharu@linkedin.com <rmatharu@linkedin.com>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #756 from rmatharu/standalonetests

3 weeks agoUpdate hardcoded versions for the 1.0.0 release.
Prateek Maheshwari [Tue, 23 Oct 2018 00:55:06 +0000 (17:55 -0700)] 
Update hardcoded versions for the 1.0.0 release.

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #754 from prateekm/version-updates

3 weeks agoAdding fix to integration test
rmatharu@linkedin.com [Mon, 22 Oct 2018 23:29:34 +0000 (16:29 -0700)] 
Adding fix to integration test

ClientHelper changed the log-print, which the integration test relies on.

Author: rmatharu@linkedin.com <rmatharu@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #752 from rmatharu/integ-test-fix

3 weeks agoSAMZA-1901: Implementation of Samza SQL Shell
Weiqing Yang [Mon, 22 Oct 2018 16:51:40 +0000 (09:51 -0700)] 
SAMZA-1901: Implementation of Samza SQL Shell

## What changes were proposed in this pull request?
This PR is to implement Samza SQL shell. The document about the shell was attached [here](https://issues.apache.org/jira/browse/SAMZA-1901).

## How was this patch tested?
1. Add unit tests
2. Run the shell with use cases mentioned in the attached document under https://issues.apache.org/jira/browse/SAMZA-1901

Author: Weiqing Yang <yangweiqing001@gmail.com>

Reviewers: Srinivasulu Punuru <spunuru@linkedin.com>, Aditya Toomula <atoomula@linkedin.com>

Closes #654 from weiqingy/samza-shell

3 weeks agoFix styling for LinkedIn logo; Add support for loading logos from URLs
Jagadish [Sun, 21 Oct 2018 19:22:16 +0000 (12:22 -0700)] 
Fix styling for LinkedIn logo; Add support for loading logos from URLs

The current styling for logos:
- Trims the logos by overlaying a circular foreground
- Auto-fetches images from Clearbit's logo API, which returns LinkedIn logo with pointed corners

Branding guidelines dictate that
  - We don't apply any foreground
  - We leave the corners in their perfectly rounded-form

To conform with them, we will
- Remove CSS settings for "background-radius"
- Add support for loading images from custom URLs other than clearbit

**Before:**
On image load:
<img width="133" alt="li-circular" src="https://user-images.githubusercontent.com/18561937/47271223-81a22f80-d52b-11e8-853b-22bd550c623a.png">
On mouse-over:
<img width="148" alt="li-rectangular" src="https://user-images.githubusercontent.com/18561937/47271222-81a22f80-d52b-11e8-8e40-0c87e7499052.png">

**After:**
<img width="161" alt="li-correct" src="https://user-images.githubusercontent.com/18561937/47271225-8830a700-d52b-11e8-97d0-d985027af436.png">

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #750 from vjagadish1989/website-reorg22

3 weeks agoClean up website docs for Samza 1.0 release
Jagadish [Sun, 21 Oct 2018 17:52:56 +0000 (10:52 -0700)] 
Clean up website docs for Samza 1.0 release

- Fix titles for blog posts
- Fix a rendering issue with hyperlinks on the front-page
- Add links from homepage sections to relevant documentation pages

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #749 from vjagadish1989/website-reorg21

3 weeks agoAdding Samza 1.0 API docs
Yi Pan (Data Infrastructure) [Fri, 19 Oct 2018 21:13:04 +0000 (14:13 -0700)] 
Adding Samza 1.0 API docs

jagadish-v0 prateekm please take a look. Thanks!

Author: Yi Pan (Data Infrastructure) <nickpan47@gmail.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #746 from nickpan47/api-docs

3 weeks agoEventhubs 1.0 tutorials cleanup
Daniel Chen [Fri, 19 Oct 2018 19:49:24 +0000 (12:49 -0700)] 
Eventhubs 1.0 tutorials cleanup

Misc Event hubs cleanup for the 1.0 release:
- Updated hello-samza tutorial documentation
- Removed unused depricated documentation (all content is not in `samza-event-hubs-standalone.md` or `connectors/eventhubs.md`
- Fixed some integration tests
- Added Event Hubs tutorial in the tutorials `index.md`

Screenshots:
<img width="1680" alt="screen shot 2018-10-16 at 11 56 51 pm" src="https://user-images.githubusercontent.com/29577458/47067812-62a84400-d19f-11e8-9511-2a3e652f4b75.png">
<img width="1680" alt="screen shot 2018-10-16 at 11 57 09 pm" src="https://user-images.githubusercontent.com/29577458/47067822-66d46180-d19f-11e8-8121-0f6fe09cbce1.png">
<img width="1680" alt="screen shot 2018-10-16 at 11 57 19 pm" src="https://user-images.githubusercontent.com/29577458/47067826-6936bb80-d19f-11e8-80a1-0e94e8459468.png">
<img width="1680" alt="screen shot 2018-10-16 at 11 57 31 pm" src="https://user-images.githubusercontent.com/29577458/47067828-6a67e880-d19f-11e8-9828-b672d0b3a1a0.png">

vjagadish prateekm please review

Author: Daniel Chen <dchen1@linkedin.com>

Reviewers: Jagadish Venkatraman <vjagadish1989@gmail.com>

Closes #736 from dxichen/eventhubs-tutorial-cleanup

3 weeks agoSAMZA-1961: Doc improvements for meetups/blogs, adding abstracts
Sanil15 [Fri, 19 Oct 2018 19:48:13 +0000 (12:48 -0700)] 
SAMZA-1961: Doc improvements for meetups/blogs, adding abstracts

vjagadish please have a look

Author: Sanil15 <sanil.jain15@gmail.com>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #739 from Sanil15/SAMZA-1961

3 weeks agoSAMZA-1964: Make getTableSpec() in RemoteTableDescriptor reentrant
Wei Song [Fri, 19 Oct 2018 19:32:26 +0000 (12:32 -0700)] 
SAMZA-1964: Make getTableSpec() in RemoteTableDescriptor reentrant

As per subject

Author: Wei Song <wsong@linkedin.com>

Reviewers: Xinyu Liu <xinyu@apache.org>

Closes #747 from weisong44/SAMZA-1964

3 weeks agoMinor: Fix a typo in architecture documentation
Jagadish [Fri, 19 Oct 2018 01:36:54 +0000 (18:36 -0700)] 
Minor: Fix a typo in architecture documentation

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #745 from vjagadish1989/website-reorg20

3 weeks agoDelete flickity.js since it's on GPL, a license which is incompatible with Apache
Jagadish [Fri, 19 Oct 2018 01:09:19 +0000 (18:09 -0700)] 
Delete flickity.js since it's on GPL, a license which is incompatible with Apache

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #744 from vjagadish1989/website-reorg19

3 weeks agoDo not try to generate configurations based on descriptors for LegacyTaskApplication...
Prateek Maheshwari [Fri, 19 Oct 2018 00:13:41 +0000 (17:13 -0700)] 
Do not try to generate configurations based on descriptors for LegacyTaskApplication in JobPlanner

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Reviewers: Ray Mathuru <rmathuru@linkedin.com>, Jagadish Venkatraman <vjagadish1989@linkedin.com>, Sanil Jain <snjain@linkedin.com>

Closes #743 from prateekm/app-mode-fix

3 weeks agoChanged TaskApplicationDescriptor APIs to be fluent and consistent with StreamApplica...
Prateek Maheshwari [Thu, 18 Oct 2018 23:16:29 +0000 (16:16 -0700)] 
Changed TaskApplicationDescriptor APIs to be fluent and consistent with StreamApplicationDescriptor API.

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Reviewers: Yi Pan <nickpan47@gmail.com>, Ray Matharu <rmatharu@linkedin.com>

Closes #741 from prateekm/task-app-api

3 weeks agoSAMZA-1953: Remove TableDescriptorsProvider interface
Aditya Toomula [Thu, 18 Oct 2018 22:21:16 +0000 (15:21 -0700)] 
SAMZA-1953: Remove TableDescriptorsProvider interface

Author: Aditya Toomula <atoomula@linkedin.com>

Reviewers: Prateek Maheshwari <pmaheshawri@apache.org>

Closes #742 from atoomula/tablefixes1

3 weeks agoMinor: fixing javadocs in StreamProcessor
Cameron Lee [Thu, 18 Oct 2018 17:44:41 +0000 (10:44 -0700)] 
Minor: fixing javadocs in StreamProcessor

Author: Cameron Lee <calee@linkedin.com>

Reviewers: Shanthoosh Venkatraman <svenkatr@linkedin.com>

Closes #734 from cameronlee314/doc_fixes

3 weeks agoKeep a version of deprecated KafkaSystemConsumer
Boris S [Thu, 18 Oct 2018 16:56:01 +0000 (09:56 -0700)] 
Keep a version of deprecated KafkaSystemConsumer

Author: Boris S <boryas@apache.org>
Author: Boris S <bshkolnik@linkedin.com>
Author: Boris Shkolnik <bshkolni@linkedin.com>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #740 from sborya/OldKafkaConsumer

3 weeks agoJavadoc cleanup for new Application, Descriptor, Context and Table APIs.
Prateek Maheshwari [Thu, 18 Oct 2018 06:58:16 +0000 (23:58 -0700)] 
Javadoc cleanup for new Application, Descriptor, Context and Table APIs.

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Reviewers: Cameron Lee<calee@linkedin.com>

Closes #737 from prateekm/javadoc-cleanup

3 weeks agoSAMZA-1888: Kafka consumer improvements
Boris S [Wed, 17 Oct 2018 23:38:19 +0000 (16:38 -0700)] 
SAMZA-1888: Kafka consumer improvements

Author: Boris S <bshkolnik@linkedin.com>
Author: Boris S <boryas@apache.org>
Author: Boris Shkolnik <bshkolni@linkedin.com>

Reviewers: bharathkk <codin.martial@gmail.com>

Closes #738 from sborya/KafkaConsumerImprovements

3 weeks agoSAMZA-1951: Make JMX enabled/disabled using configs for Samza containers
Sanil15 [Wed, 17 Oct 2018 19:02:40 +0000 (12:02 -0700)] 
SAMZA-1951: Make JMX enabled/disabled using configs for Samza containers

Author: Sanil15 <sanil.jain15@gmail.com>
Author: svenkata <svenkata@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #717 from Sanil15/SAMZA-1951

4 weeks agoAdded Samza Configurations to website
Daniel Chen [Wed, 17 Oct 2018 16:40:39 +0000 (09:40 -0700)] 
Added Samza Configurations to website

vjagadish
Added `CONFIGURATIONS` under `DOCUMENTATION`
Updated `configuration.md` page to work with new configs

Do we have documentation about `SystemDescriptors` anywhere on the website?
I was thinking to add it in the `configuration.md` page otherwise.

Author: Daniel Chen <dchen1@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #723 from dxichen/add-configs-to-website

4 weeks agoSAMZA-1956: Update value only descriptor serde
Daniel Chen [Wed, 17 Oct 2018 01:56:44 +0000 (18:56 -0700)] 
SAMZA-1956: Update value only descriptor serde

Changed the KVSerde to only value Serde  for the Eventhubs input and output descriptors.
Since the key is always a `String`, the key serde should always be `NoOpSerde` and will lead to an error otherwise since the Samza `serializers.SerdeManager.scala` expectes a `byte[]`

Author: Daniel Chen <dchen1@linkedin.com>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #733 from dxichen/eventhubs-example-cleanup

4 weeks agoSAMZA-1959: Modify EmbeddedTaggedRateLimiter to use total number of tasks for effecti...
Wei Song [Wed, 17 Oct 2018 01:00:51 +0000 (18:00 -0700)] 
SAMZA-1959: Modify EmbeddedTaggedRateLimiter to use total number of tasks for effective rate calculation

The EmbeddedTaggedRateLimiter currently uses number of tasks assigned to the currently container to calculate effective rate, which is inaccurate. With latest API refactoring, now it can take the total number of tasks.

Author: Wei Song <wsong@linkedin.com>

Reviewers: Peng Du <pdu@linkedin.com>

Closes #735 from weisong44/SAMZA-1959 and squashes the following commits:

ae0a0dd9 [Wei Song] SAMZA-1959: Modify EmbeddedTaggedRateLimiter to use total number of tasks for effective rate calculation
a06e8ec2 [Wei Song] Merge remote-tracking branch 'upstream/master'
2c679c39 [Wei Song] Merge remote-tracking branch 'upstream/master'
a56c28dc [Wei Song] Merge remote-tracking branch 'upstream/master'
097958c8 [Wei Song] Merge remote-tracking branch 'upstream/master'
05822f0a [Wei Song] Merge remote-tracking branch 'upstream/master'
f7480505 [Wei Song] Merge remote-tracking branch 'upstream/master'
7706ab1f [Wei Song] Merge remote-tracking branch 'upstream/master'
f5731b10 [Wei Song] Merge remote-tracking branch 'upstream/master'
1e5de45a [Wei Song] Merge remote-tracking branch 'upstream/master'
c85604e0 [Wei Song] Merge remote-tracking branch 'upstream/master'
242d8442 [Wei Song] Merge remote-tracking branch 'upstream/master'
ec7d8409 [Wei Song] Merge remote-tracking branch 'upstream/master'
e19b4dc9 [Wei Song] Merge remote-tracking branch 'upstream/master'
8ee78441 [Wei Song] Merge remote-tracking branch 'upstream/master'
1c6a2eae [Wei Song] Merge remote-tracking branch 'upstream/master'
a6c94add [Wei Song] Merge remote-tracking branch 'upstream/master'
41299b5b [Wei Song] Merge remote-tracking branch 'upstream/master'
239a0950 [Wei Song] Merge remote-tracking branch 'upstream/master'
eca00204 [Wei Song] Merge remote-tracking branch 'upstream/master'
51562391 [Wei Song] Merge remote-tracking branch 'upstream/master'
de708f5e [Wei Song] Merge remote-tracking branch 'upstream/master'
df2f8d7b [Wei Song] Merge remote-tracking branch 'upstream/master'
f28b491d [Wei Song] Merge remote-tracking branch 'upstream/master'
4782c61d [Wei Song] Merge remote-tracking branch 'upstream/master'
0440f75f [Wei Song] Merge remote-tracking branch 'upstream/master'
aae0f380 [Wei Song] Merge remote-tracking branch 'upstream/master'
a15a7c9a [Wei Song] Merge remote-tracking branch 'upstream/master'
5cbf9af9 [Wei Song] Merge remote-tracking branch 'upstream/master'
3f7ed71f [Wei Song] Added self to committer list

4 weeks agoSAMZA-1938: Support use cases to run multiple sql statements in one Samza job
Weiqing Yang [Tue, 16 Oct 2018 23:56:16 +0000 (16:56 -0700)] 
SAMZA-1938: Support use cases to run multiple sql statements in one Samza job

## What changes were proposed in this pull request?

This PR is to support the following user scenarios:
1. support users to run multiple Sql statements in the same job.
- sql1 = "Insert into testavro.simpleOutputTopic select * from testavro.SIMPLE1";
-  sql2 = "Insert into testavro.SIMPLE3 select * from testavro.SIMPLE2";

2. Support fan-out use case. For example,
- sql1 = "Insert into testavro.SIMPLE2 select * from testavro.SIMPLE1";
- sql2 = "Insert into testavro.SIMPLE3 select * from testavro.SIMPLE1";

3. Support fan-in use case. For example,
- sql1 = "Insert into testavro.simpleOutputTopic select * from testavro.SIMPLE2";
- sql2 = "Insert into testavro.simpleOutputTopic select * from testavro.SIMPLE1";

This PR makes the information about `SystemDescriptors`,  `MessageStream of input sources` and `OutputStream` stored in `QueryTranslator`. The reason is that those information belongs to the job and each job has only one `QueryTranslator` object.

## How was this patch tested?
1. Add unit tests
2. Testing in Samza SQL shell.

Author: Weiqing Yang <yangweiqing001@gmail.com>

Reviewers: Srinivasulu Punuru <spunuru@linkedin.com>

Closes #732 from weiqingy/SAMZA-1938

4 weeks agoSAMZA-1954: User provided configuration should have higher precedence than Samza...
Prateek Maheshwari [Tue, 16 Oct 2018 01:55:28 +0000 (18:55 -0700)] 
SAMZA-1954: User provided configuration should have higher precedence than Samza generated configuration.

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Reviewers: Cameron Lee <calee@linkedin.com>

Closes #728 from prateekm/config-precedence

4 weeks agoRemoving regex-input static-input exclusivity check
rmatharu@linkedin.com [Tue, 16 Oct 2018 01:36:00 +0000 (18:36 -0700)] 
Removing regex-input static-input exclusivity check

This PR converts a hard-check that fails in case of regex-defined input if there is an overlap
between regex-input and task-input, to a warning.

Why?
Because with Samza 1.0, rewrite is called multiple times to expand system descriptors, input descriptors, etc. This hard-check fails in this case causing deployment to stall. Therefore we convert it to a warning.

NOTE: This now allows users to define stream config for their inputs which may overlap with regex-input. This was earlier explicitly disallowed by the hard-check.

Author: rmatharu@linkedin.com <rmatharu@linkedin.com>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #731 from rmatharu/regexcheck

4 weeks agoSAMZA-1950: Fixing Style for Meetups
Sanil15 [Mon, 15 Oct 2018 23:54:36 +0000 (16:54 -0700)] 
SAMZA-1950: Fixing Style for Meetups

Author: Sanil15 <sanil.jain15@gmail.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #730 from Sanil15/SAMZA-1950

4 weeks agoSAMZA-1930: Blog for SAMZA 1.0 (Release Notes/Meetup Summary)
Sanil15 [Mon, 15 Oct 2018 23:48:44 +0000 (16:48 -0700)] 
SAMZA-1930: Blog for SAMZA 1.0 (Release Notes/Meetup Summary)

Author: Sanil15 <sanil.jain15@gmail.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #714 from Sanil15/SAMZA-1930

4 weeks agoFix case-studies for LinkedIn, Optimizely, Tripadvisor, Slack. Re-word some of them.
Jagadish [Mon, 15 Oct 2018 23:46:33 +0000 (16:46 -0700)] 
Fix case-studies for LinkedIn, Optimizely, Tripadvisor, Slack. Re-word some of them.

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #729 from vjagadish1989/website-reorg18

4 weeks agoMinor fixes for issues found during testing.
Prateek Maheshwari [Sat, 13 Oct 2018 19:38:39 +0000 (12:38 -0700)] 
Minor fixes for issues found during testing.

1. Made public methods in ApplicationDescriptorImpl non-final for mocking.
2. Updated CachedTableDescriptor public method return types to include type parameters.
3. Changed next release version to 1.0.0
4. Made KafkaSystemAdmin#toKafkaSpec public.

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Reviewers: Shanthoosh Venkatraman <svenkatr@linkedin.com>

Closes #725 from prateekm/minor-fixes

4 weeks agoApplication Descriptor implementation cleanup.
Prateek Maheshwari [Sat, 13 Oct 2018 02:22:54 +0000 (19:22 -0700)] 
Application Descriptor implementation cleanup.

Major changes:
1. Move withDefaultSystem to the base ApplicationDescriptor interface so that it can be used for TaskApplication as well.

2. Move some of the common state (input/output/system/table descriptors) and corresponding helper methods from the StreamApplicationDescriptorImpl/TaskApplicationDescriptorImpl to the ApplicationDescriptor class.

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Reviewers: Cameron Lee <calee@linkedin.com>, Yi Pan <nickpan47@gmail.com>

Closes #702 from prateekm/app-descriptor-cleanup

4 weeks agoConsolidating package names for System, Stream, Application and Table descriptors.
Prateek Maheshwari [Sat, 13 Oct 2018 01:34:34 +0000 (18:34 -0700)] 
Consolidating package names for System, Stream, Application and Table descriptors.

Everything in this PR is either:
1. A package name change and a corresponding file move.
2. Javadoc changes to use FQN in link tags to fix checkstyle complaints about unused imports, and corresponding fixes to make them fit within line width. No change in contents.
3. In a couple of places, changing method visibility to public with VisibleForTesting annotations for accessing them from Tests (RemoteReadWriteTable.java, RemoteReadableTable.java)

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Reviewers: Bharath Kumarasubramanian <bkumaras@linkedin.com>, Jagadish Venkatraman <vjagadish1989@gmail.com>, Yi Pan <nickpan47@gmail.com>

Closes #720 from prateekm/descriptor-package-cleanup

4 weeks agoUse different paths for logged and non-logged stores in test runner
bharathkk [Sat, 13 Oct 2018 01:18:56 +0000 (18:18 -0700)] 
Use different paths for logged and non-logged stores in test runner

We have scattered logic to determine the store paths between SamzaContainer and StorageManager.
Ideally, it would nice to consolidate them into StorageManager. For now, using different store directories for non-logged and logged stores so that regressions in determining store paths can be caught.

Author: bharathkk <codin.martial@gmail.com>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #707 from bharathkk/fix-table-test

4 weeks agoClean-up the case-studies page for Ebay, add a diagram
Jagadish [Sat, 13 Oct 2018 00:58:39 +0000 (17:58 -0700)] 
Clean-up the case-studies page for Ebay, add a diagram

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #724 from vjagadish1989/website-reorg17

4 weeks agoSAMZA-1937: Make DSL converter work when sql is a query
Weiqing Yang [Fri, 12 Oct 2018 20:13:12 +0000 (13:13 -0700)] 
SAMZA-1937: Make DSL converter work when sql is a query

## What changes were proposed in this pull request?
To support the use case that users use Samza Sql Console or Samza Sql shell to perform SELECT query.

## How was this patch tested?
1. Build passed and current unit tests passed
2. Testing in Samza Sql shell

Author: Weiqing Yang <yangweiqing001@gmail.com>

Reviewers: Aditya Toomula <atoomula@linkedin.com>

Closes #721 from weiqingy/SAMZA-1937 and squashes the following commits:

a16f717e [Weiqing Yang] Fix review comments
6bbf0b8a [Weiqing Yang] fix review comments
4658a42f [Weiqing Yang] make DSL converter work when sql is a query

4 weeks agoFix broken tests on master.
Prateek Maheshwari [Fri, 12 Oct 2018 19:17:48 +0000 (12:17 -0700)] 
Fix broken tests on master.

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #722 from prateekm/test-fixes

4 weeks agoAdd Slack to our Powered By page
Jagadish [Fri, 12 Oct 2018 07:25:40 +0000 (00:25 -0700)] 
Add Slack to our Powered By page

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish <jagadish@apache.org>

Closes #719 from vjagadish1989/website-reorg16

4 weeks agoPolish contents of the case-studies page
Jagadish [Fri, 12 Oct 2018 07:11:10 +0000 (00:11 -0700)] 
Polish contents of the case-studies page

- Only display case-studies that have high-impact
- Also, fix titles

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #718 from vjagadish1989/website-reorg14

4 weeks agoPowered By page: Final pass for consistency in language, tone
Jagadish [Fri, 12 Oct 2018 06:24:27 +0000 (23:24 -0700)] 
Powered By page: Final pass for consistency in language, tone

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #716 from vjagadish1989/website-reorg13

4 weeks agoOrder companies in the 'Powered By' page by their significance -
Jagadish [Fri, 12 Oct 2018 05:38:46 +0000 (22:38 -0700)] 
Order companies in the 'Powered By' page by their significance -

eg: Large-scale adopters of Samza in the beginning followed by smaller installations

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish <jagadish@apache.org>

Closes #715 from vjagadish1989/website-reorg12

4 weeks agoMinor: Fix typo in Core Concepts section
Jagadish [Fri, 12 Oct 2018 02:47:22 +0000 (19:47 -0700)] 
Minor: Fix typo in Core Concepts section

4 weeks agoMerge branch 'master' of https://github.com/apache/samza
Jagadish [Fri, 12 Oct 2018 02:22:30 +0000 (19:22 -0700)] 
Merge branch 'master' of https://github.com/apache/samza

4 weeks agoAdd "Powered By" pages for companies which use Samza
Jagadish [Fri, 12 Oct 2018 02:22:17 +0000 (19:22 -0700)] 
Add "Powered By" pages for companies which use Samza

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #713 from vjagadish1989/website-reorg11

4 weeks agoAdd Powered By pages for Samza users in the community
Jagadish [Fri, 12 Oct 2018 02:20:16 +0000 (19:20 -0700)] 
Add Powered By pages for Samza users in the community

4 weeks agoAdd Powered By pages for Samza users in the community
Jagadish [Fri, 12 Oct 2018 02:18:57 +0000 (19:18 -0700)] 
Add Powered By pages for Samza users in the community

4 weeks agoAdd Powered By pages for Samza users in the community
Jagadish [Fri, 12 Oct 2018 02:16:18 +0000 (19:16 -0700)] 
Add Powered By pages for Samza users in the community

4 weeks agoMerge branch 'master' of https://github.com/apache/samza
Jagadish [Fri, 12 Oct 2018 02:12:41 +0000 (19:12 -0700)] 
Merge branch 'master' of https://github.com/apache/samza

4 weeks agoAdd Powered By pages for Samza users in the community
Jagadish [Fri, 12 Oct 2018 02:10:03 +0000 (19:10 -0700)] 
Add Powered By pages for Samza users in the community

4 weeks agoUpdate active PMC members and committers
Jagadish [Thu, 11 Oct 2018 23:50:47 +0000 (16:50 -0700)] 
Update active PMC members and committers

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #712 from vjagadish1989/website-reorg9

4 weeks agoUpdate PMCs and committers
Jagadish [Thu, 11 Oct 2018 23:49:40 +0000 (16:49 -0700)] 
Update PMCs and committers

4 weeks agoSystem and Stream Descriptor API cleanup.
Prateek Maheshwari [Thu, 11 Oct 2018 23:35:08 +0000 (16:35 -0700)] 
System and Stream Descriptor API cleanup.

Major changes:
1. Made withPhysicalName(String) a protected method in StreamDescriptor.
The primary reason to set the physical name is to use stream names with special characters in them, which streamId doesn't support. This change is to make it so that the physical name setter should only be exposed by systems where it means something useful - like HDFS.
2. Renamed some methods in StreamDescriptor for clarity.

Author: Prateek Maheshwari <pmaheshwari@apache.org>

Reviewers: Cameron Lee <calee@linkedin.com>

Closes #708 from prateekm/stream-descriptor-cleanup

4 weeks agoMerge branch 'master' of https://github.com/apache/samza
Jagadish [Thu, 11 Oct 2018 23:33:44 +0000 (16:33 -0700)] 
Merge branch 'master' of https://github.com/apache/samza

4 weeks agoSamza SQL Documentation
Srinivasulu Punuru [Thu, 11 Oct 2018 23:26:24 +0000 (16:26 -0700)] 
Samza SQL Documentation

This is initial Samza SQL documentation. vjagadish I still am tracking your comment to add more details on various queries, Samza SQL components. I think i need more time for those. If i can i will try to add them before Samza 1.0. If not we can target that for the next release.

![snappy](https://user-images.githubusercontent.com/1783806/46564244-faae4f80-c8ba-11e8-8e1c-cc4d929d79f1.png)

Author: Srinivasulu Punuru <spunuru@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #695 from srinipunuru/samza-sql-docs.1

4 weeks agoAdd images for Samza's core concepts
Jagadish [Thu, 11 Oct 2018 23:23:07 +0000 (16:23 -0700)] 
Add images for Samza's core concepts

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish<jagadish@apache.org>

Closes #711 from vjagadish1989/website-reorg8

4 weeks agoAdd images for Samza's core concepts
Jagadish [Thu, 11 Oct 2018 23:21:24 +0000 (16:21 -0700)] 
Add images for Samza's core concepts

4 weeks agoMerge branch 'master' of https://github.com/apache/samza
Jagadish [Thu, 11 Oct 2018 23:20:21 +0000 (16:20 -0700)] 
Merge branch 'master' of https://github.com/apache/samza

4 weeks agoAdd top-level page for Samza's architecture
Jagadish [Thu, 11 Oct 2018 23:19:57 +0000 (16:19 -0700)] 
Add top-level page for Samza's architecture

Author: Jagadish <jvenkatraman@linkedin.com>

Reviewers: Jagadish <jagadish@apache.org>

Closes #710 from vjagadish1989/website-reorg7

4 weeks agoAdd an architecture page
Jagadish [Thu, 11 Oct 2018 23:18:52 +0000 (16:18 -0700)] 
Add an architecture page

4 weeks agoSAMZA-1771: Test Framework support for stateful testing and assertions over state
Sanil15 [Thu, 11 Oct 2018 23:13:45 +0000 (16:13 -0700)] 
SAMZA-1771: Test Framework support for stateful testing and assertions over state

- Gives StateAssert for stateful assertions
- Gives examples of Testing stateful jobs for TaskApplication and StreamApplication

Author: Sanil15 <sanil.jain15@gmail.com>
Author: Sanil Jain <snjain@linkedin.com>
Author: Sanil Jain <sanil.jain15@gmail.com>

Reviewers: Shanthoosh Venkataraman <svenkata@linkedin.com>, Prateek Maheshwari <pmaheshwari@apache.org>

Closes #683 from Sanil15/SAMZA-1771

4 weeks agoSAMZA-1891: Updated javadoc to include config behavior
Wei Song [Thu, 11 Oct 2018 22:28:30 +0000 (15:28 -0700)] 
SAMZA-1891: Updated javadoc to include config behavior

Instead of referring users to the Samza Configuration Guide, TableDescriptors should copy the documentation instead. If the goal is to let users configure their job primarily in code, it's simpler for them to see javadocs in their IDE than referring to a separate web page.

Author: Wei Song <wsong@linkedin.com>

Reviewers: Peng Du <pdu@linkedin.com>

Closes #709 from weisong44/SAMZA-1891 and squashes the following commits:

9f98e1aa [Wei Song] Merge branch 'master' into SAMZA-1891
2c679c39 [Wei Song] Merge remote-tracking branch 'upstream/master'
85081fb1 [Wei Song] SAMZA-1891: Updated javadoc to include config behavior
a56c28dc [Wei Song] Merge remote-tracking branch 'upstream/master'
097958c8 [Wei Song] Merge remote-tracking branch 'upstream/master'
05822f0a [Wei Song] Merge remote-tracking branch 'upstream/master'
f7480505 [Wei Song] Merge remote-tracking branch 'upstream/master'
7706ab1f [Wei Song] Merge remote-tracking branch 'upstream/master'
f5731b10 [Wei Song] Merge remote-tracking branch 'upstream/master'
1e5de45a [Wei Song] Merge remote-tracking branch 'upstream/master'
c85604e0 [Wei Song] Merge remote-tracking branch 'upstream/master'
242d8442 [Wei Song] Merge remote-tracking branch 'upstream/master'
ec7d8409 [Wei Song] Merge remote-tracking branch 'upstream/master'
e19b4dc9 [Wei Song] Merge remote-tracking branch 'upstream/master'
8ee78441 [Wei Song] Merge remote-tracking branch 'upstream/master'
1c6a2eae [Wei Song] Merge remote-tracking branch 'upstream/master'
a6c94add [Wei Song] Merge remote-tracking branch 'upstream/master'
41299b5b [Wei Song] Merge remote-tracking branch 'upstream/master'
239a0950 [Wei Song] Merge remote-tracking branch 'upstream/master'
eca00204 [Wei Song] Merge remote-tracking branch 'upstream/master'
51562391 [Wei Song] Merge remote-tracking branch 'upstream/master'
de708f5e [Wei Song] Merge remote-tracking branch 'upstream/master'
df2f8d7b [Wei Song] Merge remote-tracking branch 'upstream/master'
f28b491d [Wei Song] Merge remote-tracking branch 'upstream/master'
4782c61d [Wei Song] Merge remote-tracking branch 'upstream/master'
0440f75f [Wei Song] Merge remote-tracking branch 'upstream/master'
aae0f380 [Wei Song] Merge remote-tracking branch 'upstream/master'
a15a7c9a [Wei Song] Merge remote-tracking branch 'upstream/master'
5cbf9af9 [Wei Song] Merge remote-tracking branch 'upstream/master'
3f7ed71f [Wei Song] Added self to committer list

4 weeks agoMerge branch 'master' of https://github.com/apache/samza
Jagadish [Thu, 11 Oct 2018 22:04:22 +0000 (15:04 -0700)] 
Merge branch 'master' of https://github.com/apache/samza

4 weeks agoSAMZA-1948: Updated hybrid table descriptors to take underlying table descriptors
Wei Song [Thu, 11 Oct 2018 21:53:54 +0000 (14:53 -0700)] 
SAMZA-1948: Updated hybrid table descriptors to take underlying table descriptors

Hybrid Table Descriptors, e.g. CachingTableDescriptor, should take both tables as a constructor param instead of using withXYZ methods. This is better because they are required for the hybrid descriptor to work, and it helps with type inference for the descriptor.

Author: Wei Song <wsong@linkedin.com>

Reviewers: Peng Du <pdu@linkedin.com>

Closes #706 from weisong44/SAMZA-1948 and squashes the following commits:

53444419 [Wei Song] Updated based on review comments
39d9ab00 [Wei Song] SAMZA-1948 Updated hybrid table descriptors to take underlying table descriptors
a56c28dc [Wei Song] Merge remote-tracking branch 'upstream/master'
097958c8 [Wei Song] Merge remote-tracking branch 'upstream/master'
05822f0a [Wei Song] Merge remote-tracking branch 'upstream/master'
f7480505 [Wei Song] Merge remote-tracking branch 'upstream/master'
7706ab1f [Wei Song] Merge remote-tracking branch 'upstream/master'
f5731b10 [Wei Song] Merge remote-tracking branch 'upstream/master'
1e5de45a [Wei Song] Merge remote-tracking branch 'upstream/master'
c85604e0 [Wei Song] Merge remote-tracking branch 'upstream/master'
242d8442 [Wei Song] Merge remote-tracking branch 'upstream/master'
ec7d8409 [Wei Song] Merge remote-tracking branch 'upstream/master'
e19b4dc9 [Wei Song] Merge remote-tracking branch 'upstream/master'
8ee78441 [Wei Song] Merge remote-tracking branch 'upstream/master'
1c6a2eae [Wei Song] Merge remote-tracking branch 'upstream/master'
a6c94add [Wei Song] Merge remote-tracking branch 'upstream/master'
41299b5b [Wei Song] Merge remote-tracking branch 'upstream/master'
239a0950 [Wei Song] Merge remote-tracking branch 'upstream/master'
eca00204 [Wei Song] Merge remote-tracking branch 'upstream/master'
51562391 [Wei Song] Merge remote-tracking branch 'upstream/master'
de708f5e [Wei Song] Merge remote-tracking branch 'upstream/master'
df2f8d7b [Wei Song] Merge remote-tracking branch 'upstream/master'
f28b491d [Wei Song] Merge remote-tracking branch 'upstream/master'
4782c61d [Wei Song] Merge remote-tracking branch 'upstream/master'
0440f75f [Wei Song] Merge remote-tracking branch 'upstream/master'
aae0f380 [Wei Song] Merge remote-tracking branch 'upstream/master'
a15a7c9a [Wei Song] Merge remote-tracking branch 'upstream/master'
5cbf9af9 [Wei Song] Merge remote-tracking branch 'upstream/master'
3f7ed71f [Wei Song] Added self to committer list

4 weeks agoSAMZA-1868: Create new SamzaAmdmin for Kafka
Boris S [Thu, 11 Oct 2018 21:26:51 +0000 (14:26 -0700)] 
SAMZA-1868: Create new SamzaAmdmin for Kafka

This Request is a copy of #647(got garbled). This PR  already addresses all the comments brought up in the other request.

Author: Boris S <bshkolnik@linkedin.com>
Author: Boris S <boryas@apache.org>
Author: Boris Shkolnik <bshkolni@linkedin.com>

Reviewers: Shanthoosh Venkatraman <svenkatr@linkedin.com>, Prateek Maheshwari <pmaheshwari@apache.org>

Closes #662 from sborya/NewConsumerAdmin2

4 weeks agoMerge branch 'master' of https://github.com/apache/samza
Jagadish [Thu, 11 Oct 2018 19:32:27 +0000 (12:32 -0700)] 
Merge branch 'master' of https://github.com/apache/samza

4 weeks agoSAMZA-1900: Add restore logging info
Daniel Chen [Thu, 11 Oct 2018 18:56:05 +0000 (11:56 -0700)] 
SAMZA-1900: Add restore logging info

prateekm let me know if there are other places that logging should also be improved in this patch.

Author: Daniel Chen <dchen1@linkedin.com>
Author: Daniel Chen <xrchen@uwaterloo.ca>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #653 from dxichen/add-restore-logging-info

4 weeks agoSAMZA-1856: eventhub descriptors
Daniel Chen [Thu, 11 Oct 2018 18:49:00 +0000 (11:49 -0700)] 
SAMZA-1856: eventhub descriptors

prateekm lhaiesp
Add system descriptors and input/output descriptors for Eventhubs.
We should make an effort to deprecate the `systems.%s.stream.list`.

Author: Daniel Chen <dchen1@linkedin.com>

Reviewers: Prateek Maheshwari <pmaheshwari@apache.org>

Closes #696 from dxichen/eventhub-system-descriptors