aurora.git
17 months agoUpdating .auroraversion to release version 0.11.0. rel/0.11.0
Bill Farner [Wed, 23 Dec 2015 16:22:52 +0000 (08:22 -0800)] 
Updating .auroraversion to release version 0.11.0.

17 months agoUpdating .auroraversion to 0.11.0-rc1.
Bill Farner [Thu, 17 Dec 2015 23:18:44 +0000 (15:18 -0800)] 
Updating .auroraversion to 0.11.0-rc1.

17 months agoUpdating CHANGELOG for 0.11.0 release.
Bill Farner [Thu, 17 Dec 2015 23:18:44 +0000 (15:18 -0800)] 
Updating CHANGELOG for 0.11.0 release.

17 months agoRevert "Updating CHANGELOG for 0.11.0 release."
Bill Farner [Thu, 17 Dec 2015 23:10:57 +0000 (15:10 -0800)] 
Revert "Updating CHANGELOG for 0.11.0 release."

This reverts commit 5cd2767f816525ed65f830a5413de56e735d688c.

17 months agoRevert "Incrementing snapshot version to 0.12.0-SNAPSHOT."
Bill Farner [Thu, 17 Dec 2015 23:10:48 +0000 (15:10 -0800)] 
Revert "Incrementing snapshot version to 0.12.0-SNAPSHOT."

This reverts commit 20a57f1a0f80576e2311b11f63423dc8880fad4b.

17 months agoIncrementing snapshot version to 0.12.0-SNAPSHOT.
Bill Farner [Thu, 17 Dec 2015 23:03:50 +0000 (15:03 -0800)] 
Incrementing snapshot version to 0.12.0-SNAPSHOT.

17 months agoUpdating CHANGELOG for 0.11.0 release.
Bill Farner [Thu, 17 Dec 2015 23:03:50 +0000 (15:03 -0800)] 
Updating CHANGELOG for 0.11.0 release.

17 months agoAdd flag to set FrameworkInfo.principal
R.B. Boyer [Thu, 17 Dec 2015 22:36:11 +0000 (14:36 -0800)] 
Add flag to set FrameworkInfo.principal

Bugs closed: AURORA-687

Reviewed at https://reviews.apache.org/r/41525/

17 months agoImplements log rotation in the Thermos runner.
George Sirois [Thu, 17 Dec 2015 22:02:59 +0000 (14:02 -0800)] 
Implements log rotation in the Thermos runner.

Bugs closed: AURORA-95

Reviewed at https://reviews.apache.org/r/30695/

17 months agoAccept a command line argument for an executor configuration via json
Renan DelValle [Thu, 17 Dec 2015 21:48:53 +0000 (15:48 -0600)] 
Accept a command line argument for an executor configuration via json

Patch to allow Aurora to accept an executor config via commandline which overrides the default thermos one.

Reviewed at https://reviews.apache.org/r/41473/

17 months agoAdd a NEWS entry about shell health checks and HealthCheckConfig deprecations.
Joshua Cohen [Thu, 17 Dec 2015 21:15:17 +0000 (15:15 -0600)] 
Add a NEWS entry about shell health checks and HealthCheckConfig deprecations.

Reviewed at https://reviews.apache.org/r/41531/

17 months agoRefactoring HealthCheckConfig into separate structs
Dmitriy Shirchenko [Thu, 17 Dec 2015 20:33:55 +0000 (12:33 -0800)] 
Refactoring HealthCheckConfig into separate structs

Bugs closed: AURORA-1562

Reviewed at https://reviews.apache.org/r/41428/

17 months agoRemove ServerInfo.thriftAPIVersion
Bill Farner [Thu, 17 Dec 2015 18:59:20 +0000 (10:59 -0800)] 
Remove ServerInfo.thriftAPIVersion

Bugs closed: AURORA-1553

Reviewed at https://reviews.apache.org/r/41522/

17 months agoForce Windows to always use Unix line endings.
Joshua Cohen [Thu, 17 Dec 2015 18:59:15 +0000 (12:59 -0600)] 
Force Windows to always use Unix line endings.

Since everything is run inside an Ubuntu VM, we want to ensure the line endings don't get auto-translated.

Bugs closed: AURORA-1354

Reviewed at https://reviews.apache.org/r/41521/

17 months agoHTTP server cleanup - shut down the server after unit tests, remove jetty bug workaround.
Bill Farner [Thu, 17 Dec 2015 18:58:55 +0000 (10:58 -0800)] 
HTTP server cleanup - shut down the server after unit tests, remove jetty bug workaround.

Reviewed at https://reviews.apache.org/r/41453/

17 months agoRemove duplicate ExecutorConfig class.
Bill Farner [Thu, 17 Dec 2015 18:58:03 +0000 (10:58 -0800)] 
Remove duplicate ExecutorConfig class.

Reviewed at https://reviews.apache.org/r/41523/

17 months agoDeprecate JobUpdateSettings.maxWaitToInstanceRunning and UpdateConfig.restart_threshold
Bill Farner [Thu, 17 Dec 2015 18:56:51 +0000 (10:56 -0800)] 
Deprecate JobUpdateSettings.maxWaitToInstanceRunning and UpdateConfig.restart_threshold

Bugs closed: AURORA-1252, AURORA-1253

Reviewed at https://reviews.apache.org/r/41486/

17 months agoHandling task event race in updater.
Maxim Khutornenko [Thu, 17 Dec 2015 18:05:04 +0000 (10:05 -0800)] 
Handling task event race in updater.

Bugs closed: AURORA-1506, AURORA-1507

Reviewed at https://reviews.apache.org/r/41226/

17 months agoRemove the client-side updater.
Bill Farner [Wed, 16 Dec 2015 20:38:44 +0000 (12:38 -0800)] 
Remove the client-side updater.

Bugs closed: AURORA-785

Reviewed at https://reviews.apache.org/r/41368/

17 months agoAdd PMD coverage for test sources.
Bill Farner [Wed, 16 Dec 2015 00:04:44 +0000 (16:04 -0800)] 
Add PMD coverage for test sources.

Reviewed at https://reviews.apache.org/r/41413/

17 months agoUpgrade to jetty 9.3.6.v20151106.
Bill Farner [Tue, 15 Dec 2015 06:40:18 +0000 (22:40 -0800)] 
Upgrade to jetty 9.3.6.v20151106.

Reviewed at https://reviews.apache.org/r/41392/

17 months agoDisable task caching for the generateBuildProperties task.
Bill Farner [Tue, 15 Dec 2015 05:57:46 +0000 (21:57 -0800)] 
Disable task caching for the generateBuildProperties task.

Reviewed at https://reviews.apache.org/r/41391/

17 months agoAdd support for performing health checks with a shell command.
Dmitriy Shirchenko [Tue, 15 Dec 2015 05:37:44 +0000 (21:37 -0800)] 
Add support for performing health checks with a shell command.

Bugs closed: AURORA-1551

Reviewed at https://reviews.apache.org/r/41154/

17 months agoUpgrade mesos dependency to 0.24.1.
Bill Farner [Mon, 14 Dec 2015 22:02:07 +0000 (14:02 -0800)] 
Upgrade mesos dependency to 0.24.1.

Bugs closed: AURORA-1543

Reviewed at https://reviews.apache.org/r/41315/

17 months agoDocument mesos.hostname in reference
Chris Bannister [Mon, 14 Dec 2015 21:59:14 +0000 (13:59 -0800)] 
Document mesos.hostname in reference

Bugs closed: AURORA-1557

Reviewed at https://reviews.apache.org/r/41367/

17 months agoUpgrade to pants 0.0.64 and pex 1.1.1.
John Sirois [Mon, 14 Dec 2015 18:43:44 +0000 (10:43 -0800)] 
Upgrade to pants 0.0.64 and pex 1.1.1.

CHANGELOGs are here:
+ http://pantsbuild.github.io/changelog.html
+ https://pypi.python.org/pypi/pex/1.1.1

The pants upgrade ends the deprecation cycle for the `python_test_suite`
aggregator target; so these are all switched to the reccomended `target`
aggregator target.

The pex upgrade brings in support for a global `/etc/pexrc` which could
be desirable for control over the executor components.

 3rdparty/python/requirements.txt                    | 2 +-
 pants.ini                                           | 2 +-
 src/test/python/BUILD                               | 2 +-
 src/test/python/apache/aurora/BUILD                 | 2 +-
 src/test/python/apache/aurora/admin/BUILD           | 2 +-
 src/test/python/apache/aurora/client/BUILD          | 2 +-
 src/test/python/apache/aurora/client/api/BUILD      | 2 +-
 src/test/python/apache/aurora/client/cli/BUILD      | 2 +-
 src/test/python/apache/aurora/client/hooks/BUILD    | 2 +-
 src/test/python/apache/aurora/common/BUILD          | 2 +-
 src/test/python/apache/aurora/config/BUILD          | 2 +-
 src/test/python/apache/aurora/executor/BUILD        | 6 +++---
 src/test/python/apache/aurora/executor/bin/BUILD    | 2 +-
 src/test/python/apache/aurora/executor/common/BUILD | 2 +-
 src/test/python/apache/aurora/tools/BUILD           | 2 +-
 src/test/python/apache/thermos/BUILD                | 2 +-
 src/test/python/apache/thermos/common/BUILD         | 2 +-
 src/test/python/apache/thermos/config/BUILD         | 2 +-
 src/test/python/apache/thermos/core/BUILD           | 8 ++++----
 src/test/python/apache/thermos/monitoring/BUILD     | 2 +-
 src/test/python/apache/thermos/observer/BUILD       | 2 +-
 src/test/python/apache/thermos/observer/http/BUILD  | 2 +-
 22 files changed, 27 insertions(+), 27 deletions(-)

Testing Done:
Locally green: `./build-support/jenkins/build.sh`

Bugs closed: AURORA-1554

Reviewed at https://reviews.apache.org/r/41331/

17 months agoRemove outdated constraint documentation.
Bill Farner [Fri, 11 Dec 2015 19:08:41 +0000 (11:08 -0800)] 
Remove outdated constraint documentation.

Reviewed at https://reviews.apache.org/r/41155/

17 months agoRevert "Revert "Replace manual Forwarding* with `@Forward`.""
John Sirois [Thu, 10 Dec 2015 18:22:18 +0000 (10:22 -0800)] 
Revert "Revert "Replace manual Forwarding* with `@Forward`.""

This reverts commit 0e62780c48ecc89571482f5b6663914147f96234.

Bugs closed: AURORA-1550

Reviewed at https://reviews.apache.org/r/41190/

17 months agoConvert test sources to use lambdas throughout.
Bill Farner [Thu, 10 Dec 2015 18:11:35 +0000 (10:11 -0800)] 
Convert test sources to use lambdas throughout.

Reviewed at https://reviews.apache.org/r/41181/

17 months agoFixing scoped update issue.
Maxim Khutornenko [Wed, 9 Dec 2015 23:42:20 +0000 (15:42 -0800)] 
Fixing scoped update issue.

Bugs closed: AURORA-1549

Reviewed at https://reviews.apache.org/r/41145/

17 months agoRevert "Replace manual Forwarding* with `@Forward`."
John Sirois [Wed, 9 Dec 2015 16:58:20 +0000 (08:58 -0800)] 
Revert "Replace manual Forwarding* with `@Forward`."

This reverts commit 0c98e8a82177f39534db3a49162582aeb1739728.

Bugs closed: AURORA-1146

Reviewed at https://reviews.apache.org/r/41140/

17 months agoset TaskStatus.slave_id field in TaskReconsiler
Tengfei Mu [Tue, 8 Dec 2015 22:04:02 +0000 (14:04 -0800)] 
set TaskStatus.slave_id field in TaskReconsiler

Bugs closed: AURORA-1547

Reviewed at https://reviews.apache.org/r/41048/

17 months agoUse lambdas throughout the project.
Bill Farner [Tue, 8 Dec 2015 18:27:26 +0000 (10:27 -0800)] 
Use lambdas throughout the project.

Reviewed at https://reviews.apache.org/r/41074/

17 months agoUpgrade checkstyle and pmd versions.
Bill Farner [Tue, 8 Dec 2015 03:47:58 +0000 (19:47 -0800)] 
Upgrade checkstyle and pmd versions.

Reviewed at https://reviews.apache.org/r/41056/

17 months agoReplace manual Forwarding* with `@Forward`.
John Sirois [Mon, 7 Dec 2015 22:33:33 +0000 (14:33 -0800)] 
Replace manual Forwarding* with `@Forward`.

This is a bit of a straw man.  While working on
https://issues.apache.org/jira/browse/AURORA-987 I found the need for
`javapoet` and this project spun off the side.

See the `@Forward` README here for more info: https://github.com/perkuno/forward

Although I think the end result is desirable, this change does add a
dependency on a personal project.  I fancied up the presentation with a
custom org, but its still a personal project. I'd be happy enough to
move the `@Forward` code over to aurora, but it would require a seperate
gradle module to ensure the annotation processor is compiled ahead of
the main module that would use it.

So kick the tires and let me know what you think.

 build.gradle                                                                  |   6 +-
 src/main/java/org/apache/aurora/scheduler/storage/ForwardingStore.java        | 185 ------------------------------
 src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java  |  13 ++-
 src/test/java/org/apache/aurora/scheduler/thrift/aop/ForwardingThrift.java    | 309 --------------------------------------------------
 src/test/java/org/apache/aurora/scheduler/thrift/aop/MockDecoratedThrift.java |   5 +-
 5 files changed, 20 insertions(+), 498 deletions(-)

Testing Done:
Green locally `./build-support/jenkins/build.sh`.

An example of the generated code:
```java
@Generated("uno.perk.forward.apt.Processor")
class MockDecoratedThriftForwarder implements AnnotatedAuroraAdmin {
  protected final AnnotatedAuroraAdmin annotatedAuroraAdmin;

  MockDecoratedThriftForwarder(AnnotatedAuroraAdmin annotatedAuroraAdmin) {
    this.annotatedAuroraAdmin = Objects.requireNonNull(annotatedAuroraAdmin);
  }

  @Override
  public Response getRoleSummary() throws TException {
    return this.annotatedAuroraAdmin.getRoleSummary();
  }
...
```

Reviewed at https://reviews.apache.org/r/40786/

17 months ago[docs] Fix jobs list documentation in configuration tutorial.
Paul Cavallaro [Wed, 2 Dec 2015 20:32:07 +0000 (12:32 -0800)] 
[docs] Fix jobs list documentation in configuration tutorial.

Reviewed at https://reviews.apache.org/r/40881/

17 months agoRemove SessionKey from APIs and implementations.
Bill Farner [Mon, 30 Nov 2015 17:33:37 +0000 (09:33 -0800)] 
Remove SessionKey from APIs and implementations.

Since our auth* is now transport-level, we don't need it in the app-level
messages.

Bugs closed: AURORA-814

Reviewed at https://reviews.apache.org/r/40656/

17 months agoUpgrade RBTools to 0.7.5
Zameer Manji [Wed, 25 Nov 2015 02:03:21 +0000 (18:03 -0800)] 
Upgrade RBTools to 0.7.5

Upgrade to RBTools 0.7.5 and use the wheel hosted on PyPI.
Changelog: https://www.reviewboard.org/docs/releasenotes/rbtools/0.7.5/

Testing Done:
Modified the script and ran ./rbt

Reviewed at https://reviews.apache.org/r/40689/

17 months agoKill extraneous `make clean`.
John Sirois [Tue, 24 Nov 2015 22:16:16 +0000 (14:16 -0800)] 
Kill extraneous `make clean`.

We're building from a fresh tarball, so there is no need for a clean.
Importantly, the clean breaks building under centos 7 in
aurora-packaging, so this does less work and fixes that case.

Testing Done:
Locally green for `./build-support/thrift/thriftw 0.9.1 --version`
Bill confirmed the fix works for aurora-packaging CentOS 7.

Reviewed at https://reviews.apache.org/r/40675/

17 months agoSet JDK and langauge level for entire project.
Zameer Manji [Tue, 24 Nov 2015 21:46:38 +0000 (13:46 -0800)] 
Set JDK and langauge level for entire project.

To ensure IDEA is configured correctly, set JDK and langauge level for entire
project.

Testing Done:
./gradlew idea
Verified that a freshly created project uses JDK 1.8 with JDK 8 langauge
features.

Reviewed at https://reviews.apache.org/r/40673/

17 months agoRemove jersey dependency from commons code.
Bill Farner [Tue, 24 Nov 2015 18:47:52 +0000 (10:47 -0800)] 
Remove jersey dependency from commons code.

Reviewed at https://reviews.apache.org/r/40662/

18 months agoReplace Twitter checkstyle with pants checkstyle.
John Sirois [Tue, 24 Nov 2015 00:49:48 +0000 (16:49 -0800)] 
Replace Twitter checkstyle with pants checkstyle.

This installs the ~newly split off pants python checks contrib plugin.
Release notes for that were here:
  https://pypi.python.org/pypi/pantsbuild.pants/0.0.58

The plugin provides both python checkstyle (`compile.pythonstyle`), and
a python eval task (`compile.python-eval`).  The `python-eval` is turned
off since at least one of the Aurora python targets has files that have
side-effects upon import (a repl is started).

Now style checks run before compile (and thus before tests) and they
benefit from fingerprinting; ie: if you test your changes, those tests
will run style checks and when you go to commit, those checks will not
be re-run by the commit hook (although files you did not test will still
need to be checked).

A few production files were fixed up according to style failures coming
from:
+ no space after comment opening '#'
+ unused variables
+ mis-aligned hanging closing parens.

Bugs closed: AURORA-1532

Reviewed at https://reviews.apache.org/r/40310/

18 months agoKill unused `.st` files and params.
John Sirois [Tue, 24 Nov 2015 00:48:01 +0000 (16:48 -0800)] 
Kill unused `.st` files and params.

This cleans up stringtemplate use in Aurora as directed by spikes on
https://issues.apache.org/jira/browse/AURORA-987 that included upgrading
to the stringtemplate 4.x series which errors when parameters used
anywhere in a template - including un-triggered conditionals - are not
bound.

The cleanups are valid under stringtemplate 3.x as verified by testing
the affected endpoints via the vagrant vm.

Reviewed at https://reviews.apache.org/r/40466/

18 months agoCleanup thermos_executor test pexes.
John Sirois [Mon, 23 Nov 2015 21:56:45 +0000 (13:56 -0800)] 
Cleanup thermos_executor test pexes.

Previously these were built to the standard `dist/` dir, now they are
dumped to a tmp dir that's cleaned up.

Testing Done:
Green locally:
```
$ rm -rf dist/ && \
  ./pants test.pytest --no-fast src/test/python/apache/aurora/executor:
```

And no `dist/` created.

Bugs closed: AURORA-547

Reviewed at https://reviews.apache.org/r/40201/

18 months agoRemove unused and deprecated code from commons.
Zameer Manji [Fri, 20 Nov 2015 20:19:46 +0000 (12:19 -0800)] 
Remove unused and deprecated code from commons.

Testing Done:
./gradlew build -Pq

Reviewed at https://reviews.apache.org/r/40510/

18 months agoModernize the pex venv script.
John Sirois [Thu, 19 Nov 2015 16:33:19 +0000 (10:33 -0600)] 
Modernize the pex venv script.

This converts from grabbing the old Twitter python pex to grabbing
modern pex to match the version specified in 3rdparty to help keep the
pex venv up to date with the codebase dependencies.

Testing Done:
Locally ran this 2x and observed both the proper version (1.1.0) and
proper use of the cached venv in the second run:
`git clean -fdx build-support && ./build-support/pex --version`

Reviewed at https://reviews.apache.org/r/40220/

18 months agoFix typo in the scheduler deployment documentation.
Kevin Sweeney [Thu, 19 Nov 2015 04:36:49 +0000 (20:36 -0800)] 
Fix typo in the scheduler deployment documentation.

Reviewed at https://reviews.apache.org/r/36874/

18 months agoIntroduce a utility class to read executor configurations in JSON.
Bill Farner [Thu, 19 Nov 2015 04:09:46 +0000 (20:09 -0800)] 
Introduce a utility class to read executor configurations in JSON.

Reviewed at https://reviews.apache.org/r/40391/

18 months agoImprove the isolation in the thrift build.
John Sirois [Tue, 17 Nov 2015 15:27:50 +0000 (07:27 -0800)] 
Improve the isolation in the thrift build.

This improves the isolation of the thrift build by building a local
pinned bison dep. For one, this gets thrift 0.9.1 building on Arch
Linux, a rolling release that is otherwise too modern in its libs to
successfully build thrift 0.9.1.

Additionally, this change knocks out the checksumming TODO and now
checks all downloaded tarballs meet their expected hashes.  The
hashes were generated locally after checking the sha1's where
available.

Testing Done:
I can now run `./build-support/jenkins/build.sh` green locally.

I also got a hold of an OSX 10.10.5 box and was able to run
`./build-support/jenkins/build.sh` green on it using this change.

Reviewed at https://reviews.apache.org/r/40323/

18 months agoOrganize executor-related code into a package.
Bill Farner [Mon, 16 Nov 2015 19:46:05 +0000 (11:46 -0800)] 
Organize executor-related code into a package.

Reviewed at https://reviews.apache.org/r/40320/

18 months agoUpgrade to pants `0.0.59` to avoid pytest errors.
John Sirois [Mon, 16 Nov 2015 19:45:34 +0000 (11:45 -0800)] 
Upgrade to pants `0.0.59` to avoid pytest errors.

The problem is described here:
  https://github.com/pantsbuild/pants/issues/2566

Changelogs are here:
  https://pypi.python.org/pypi/pantsbuild.pants/0.0.59

 pants.ini | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Testing Done:
Before the fix stumbled into this:
```
$ ./build-support/jenkins/build.sh
...
11:22:51 00:00   [test]
11:22:51 00:00     [run_prep_command]
11:22:51 00:00     [test]
11:22:51 00:00     [pytest]
11:22:51 00:00       [run]

11:23:04 00:13         [chroot]
11:23:11 00:20   [complete]
               FAILURE

Exception message: Could not satisfy all requirements for pytest>=2.6,<2.7:
    pytest>=2.6,<2.7, pytest>=2.8.0(from: pytest-timeout)
```

That triggered a bit of a pants release firefight leading
to this RB and one for Medium's mono repo as well as patching
of the fix in at Square. Now
`./build-support/jenkins/build.sh` runs green as well.

Reviewed at https://reviews.apache.org/r/40334/

18 months agoIsolate the `third_party/` repo to `mesos.native`.
John Sirois [Mon, 16 Nov 2015 17:57:25 +0000 (11:57 -0600)] 
Isolate the `third_party/` repo to `mesos.native`.

This lifts `mesos.interface` and `mesos.native` out of
`requirements.txt`, ties their versions together and isolates the
custom repository needed by `mesos.native` to its `python_requirement`.

 3rdparty/python/BUILD            | 26 ++++++++++++++++++++++++++
 3rdparty/python/requirements.txt |  2 --
 pants.ini                        |  5 -----
 3 files changed, 26 insertions(+), 7 deletions(-)

Testing Done:
Successfully ran end-to-end with this change using a fresh vagrant
image and clean repo via:
```
vagrant destroy && \
bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
```

Bugs closed: AURORA-1538

Reviewed at https://reviews.apache.org/r/40324/

18 months agoUpgrade to checkstyle 6.12.1
Bill Farner [Sat, 14 Nov 2015 01:26:36 +0000 (17:26 -0800)] 
Upgrade to checkstyle 6.12.1

Reviewed at https://reviews.apache.org/r/40304/

18 months agoRestore the third_party python repo, needed for mesos.native egg.
Bill Farner [Fri, 13 Nov 2015 17:24:41 +0000 (09:24 -0800)] 
Restore the third_party python repo, needed for mesos.native egg.

Bugs closed: AURORA-1538

Reviewed at https://reviews.apache.org/r/40299/

18 months agoFix `./pants test src/test/python::` to work out of the box.
John Sirois [Fri, 13 Nov 2015 15:46:27 +0000 (07:46 -0800)] 
Fix `./pants test src/test/python::` to work out of the box.

The Aurora python tests cannot currently all be run together; so the ci
script passes --no-fast and devs need to remember to do this too.
Improve the dev experience by defaulting this option.

 build-support/jenkins/build.sh | 2 +-
 pants.ini                      | 7 +++++++
 2 files changed, 8 insertions(+), 1 deletion(-)

Testing Done:
Ran into this working https://issues.apache.org/jira/browse/AURORA-547.
After the fix, locally ran `./pants test src/test/python:: -- -v` green.
Previously this would lead to test failures and CPU starvation.

Bugs closed: AURORA-547

Reviewed at https://reviews.apache.org/r/40197/

18 months ago[cli] Remove unused --local flag to job inspect command
Paul Cavallaro [Fri, 13 Nov 2015 15:24:53 +0000 (07:24 -0800)] 
[cli] Remove unused --local flag to job inspect command

Reviewed at https://reviews.apache.org/r/40249/

18 months agoEliminate OOB pip install of python deps in CI.
John Sirois [Thu, 12 Nov 2015 18:56:56 +0000 (10:56 -0800)] 
Eliminate OOB pip install of python deps in CI.

Under older pants, and thus older pex, distributions were fetched using
urllib and were not robust to flaky connections. Now that Aurora is on
modern pants and pex, which uses the `requests` library for fetching and
includes 5 retries by default, this step should no longer be needed to
ensure stable CI runs.

 build-support/jenkins/build.sh | 7 -------
 pants.ini                      | 5 -----
 2 files changed, 12 deletions(-)

Testing Done:
I still can't quite run `./build-support/jenkins/build.sh` straight up
due to https://issues.apache.org/jira/browse/AURORA-1083 but ran the
following successfully locally which forces re-download of requirements
by pants:
```
$ ./pants clean-all test.pytest --no-fast src/test/python::
```

Bugs closed: AURORA-954

Reviewed at https://reviews.apache.org/r/40208/

18 months agoIncrementing snapshot version to 0.11.0-SNAPSHOT.
Zameer Manji [Thu, 12 Nov 2015 04:10:07 +0000 (20:10 -0800)] 
Incrementing snapshot version to 0.11.0-SNAPSHOT.

18 months agoUpdating CHANGELOG for 0.10.0 release.
Zameer Manji [Thu, 12 Nov 2015 04:10:07 +0000 (20:10 -0800)] 
Updating CHANGELOG for 0.10.0 release.

18 months agoRevert .auroraversion and CHANGELOG for 0.10.0 RC2
Zameer Manji [Thu, 12 Nov 2015 04:08:33 +0000 (20:08 -0800)] 
Revert .auroraversion and CHANGELOG for 0.10.0 RC2

18 months agoIncrementing snapshot version to 0.11.0-SNAPSHOT.
Zameer Manji [Thu, 12 Nov 2015 04:05:01 +0000 (20:05 -0800)] 
Incrementing snapshot version to 0.11.0-SNAPSHOT.

18 months agoUpdating CHANGELOG for 0.10.0 release.
Zameer Manji [Thu, 12 Nov 2015 04:05:01 +0000 (20:05 -0800)] 
Updating CHANGELOG for 0.10.0 release.

18 months agoRevert .auroraversion for 0.10.0-rc1.
Zameer Manji [Thu, 12 Nov 2015 04:01:17 +0000 (20:01 -0800)] 
Revert .auroraversion for 0.10.0-rc1.

18 months agoTrim whitespace for pants script.
Joe Smith [Thu, 12 Nov 2015 00:50:28 +0000 (16:50 -0800)] 
Trim whitespace for pants script.

Contains https://github.com/pantsbuild/setup/pull/9

Reviewed at https://reviews.apache.org/r/40212/

18 months agoUpdate pants bootstrap script to be agnostic to sed version.
Joe Smith [Wed, 11 Nov 2015 23:49:53 +0000 (15:49 -0800)] 
Update pants bootstrap script to be agnostic to sed version.

Reviewed at https://reviews.apache.org/r/40204/

18 months agoCreate ExecutorSettings closer to command line arguments.
Bill Farner [Wed, 11 Nov 2015 16:57:26 +0000 (08:57 -0800)] 
Create ExecutorSettings closer to command line arguments.

This is prep work for https://issues.apache.org/jira/browse/AURORA-1376

Reviewed at https://reviews.apache.org/r/40149/

18 months agoClarify the commit process for new contributors.
John Sirois [Wed, 11 Nov 2015 16:22:40 +0000 (08:22 -0800)] 
Clarify the commit process for new contributors.

Reviewed at https://reviews.apache.org/r/40161/

18 months agoUpgrade Aurora to pants 0.0.57.
John Sirois [Tue, 10 Nov 2015 23:27:53 +0000 (15:27 -0800)] 
Upgrade Aurora to pants 0.0.57.

Bugs closed: AURORA-1499

Reviewed at https://reviews.apache.org/r/39784/

18 months agoIncrementing snapshot version to 0.11.0-SNAPSHOT.
Zameer Manji [Tue, 10 Nov 2015 19:24:50 +0000 (11:24 -0800)] 
Incrementing snapshot version to 0.11.0-SNAPSHOT.

18 months agoUpdating CHANGELOG for 0.10.0 release.
Zameer Manji [Tue, 10 Nov 2015 19:24:50 +0000 (11:24 -0800)] 
Updating CHANGELOG for 0.10.0 release.

18 months agoDeleting unused function deprecation_warning
Dmitriy Shirchenko [Tue, 10 Nov 2015 02:17:36 +0000 (18:17 -0800)] 
Deleting unused function deprecation_warning
Adding test coverage to base, job_monitor, quota_check, scheduler_client

Bugs closed: AURORA-1536

Reviewed at https://reviews.apache.org/r/39958/

18 months agoCallback should only be called on drained hosts.
David Robinson [Mon, 9 Nov 2015 23:37:53 +0000 (15:37 -0800)] 
Callback should only be called on drained hosts.

Bugs closed: AURORA-1537

Reviewed at https://reviews.apache.org/r/40104/

18 months agoAdding GPG key for zmanji@apache.org.
Zameer Manji [Mon, 9 Nov 2015 22:56:38 +0000 (14:56 -0800)] 
Adding GPG key for zmanji@apache.org.

Reviewed at https://reviews.apache.org/r/40043/

18 months agoAdding build target to build kerberos auth module.
Maxim Khutornenko [Sat, 7 Nov 2015 00:47:02 +0000 (16:47 -0800)] 
Adding build target to build kerberos auth module.

Reviewed at https://reviews.apache.org/r/40042/

18 months agoAdding help message in case kerberos auth fails.
Maxim Khutornenko [Wed, 4 Nov 2015 01:00:23 +0000 (17:00 -0800)] 
Adding help message in case kerberos auth fails.

Bugs closed: AURORA-1364

Reviewed at https://reviews.apache.org/r/39797/

18 months agoupgrade psutil to 3.2.2
Dmitriy Shirchenko [Mon, 2 Nov 2015 21:04:23 +0000 (13:04 -0800)] 
upgrade psutil to 3.2.2

Bugs closed: AURORA-1531

Reviewed at https://reviews.apache.org/r/39822/

18 months agoRetry scheduler connection after unknown failures.
Stephan Erb [Mon, 2 Nov 2015 18:54:04 +0000 (10:54 -0800)] 
Retry scheduler connection after unknown failures.

Bugs closed: AURORA-1533

Reviewed at https://reviews.apache.org/r/39854/

18 months agoEliminate duplicate docu in configuration tutorial.
Stephan Erb [Sun, 1 Nov 2015 17:02:10 +0000 (09:02 -0800)] 
Eliminate duplicate docu in configuration tutorial.

Bugs closed: AURORA-829

Reviewed at https://reviews.apache.org/r/39823/

18 months agoRemove ChainedPathDetector and root argument from the observer.
Zameer Manji [Thu, 29 Oct 2015 23:25:53 +0000 (16:25 -0700)] 
Remove ChainedPathDetector and root argument from the observer.

Testing Done:
e2e tests passed

Bugs closed: AURORA-1338

Reviewed at https://reviews.apache.org/r/39629/

18 months agoModify ClusterStateImpl to be thread safe.
Zameer Manji [Thu, 29 Oct 2015 21:04:23 +0000 (14:04 -0700)] 
Modify ClusterStateImpl to be thread safe.

ClusterStateImpl exposes a synchronized multimap which does not have a
thread-safe implementation of `keySet`. This patch revises the
`ClusterStateImpl` to return an immutable copy.

Bugs closed: AURORA-1510

Reviewed at https://reviews.apache.org/r/39670/

18 months agoAdd note to CONTRIBUTING.md about default reviewers.
Joshua Cohen [Thu, 29 Oct 2015 18:38:40 +0000 (13:38 -0500)] 
Add note to CONTRIBUTING.md about default reviewers.

Reviewed at https://reviews.apache.org/r/39764/

18 months agofix client build path in the docs
Dmitriy Shirchenko [Thu, 29 Oct 2015 15:51:28 +0000 (08:51 -0700)] 
fix client build path in the docs

Bugs closed: AURORA-1529

Reviewed at https://reviews.apache.org/r/39742/

18 months agoSuppress no-op delete log noise during scheduler startup.
Bill Farner [Wed, 28 Oct 2015 03:12:16 +0000 (20:12 -0700)] 
Suppress no-op delete log noise during scheduler startup.

Reviewed at https://reviews.apache.org/r/39691/

18 months agoAdd missing binding for ZooKeeperTestServer when running a local scheduler.
Bill Farner [Mon, 26 Oct 2015 23:02:50 +0000 (16:02 -0700)] 
Add missing binding for ZooKeeperTestServer when running a local scheduler.

Reviewed at https://reviews.apache.org/r/39669/

18 months agoAdd listener to trigger scheduler shutdown on service failure.
Zameer Manji [Mon, 26 Oct 2015 21:03:52 +0000 (14:03 -0700)] 
Add listener to trigger scheduler shutdown on service failure.

Bugs closed: AURORA-1511

Reviewed at https://reviews.apache.org/r/39631/

18 months agoRemove unused Connection interface.
Zameer Manji [Sun, 25 Oct 2015 22:00:14 +0000 (15:00 -0700)] 
Remove unused Connection interface.

Reviewed at https://reviews.apache.org/r/39633/

18 months agoExplicitly declare FindBugs version and effort level.
Zameer Manji [Sun, 25 Oct 2015 20:39:33 +0000 (13:39 -0700)] 
Explicitly declare FindBugs version and effort level.

Reviewed at https://reviews.apache.org/r/39630/

19 months agoIgnore serverInfo on the client side.
Kevin Sweeney [Fri, 23 Oct 2015 03:13:10 +0000 (20:13 -0700)] 
Ignore serverInfo on the client side.

The design of this check is flawed - the client has already sent an RPC
to the scheduler and received a response for it, meaning the request has
already been processed and this check only serves to ignore its results.

Reviewed at https://reviews.apache.org/r/39563/

19 months agoRemove callable check.
Kevin Sweeney [Thu, 22 Oct 2015 23:56:00 +0000 (16:56 -0700)] 
Remove callable check.

Bugs closed: AURORA-1524

Reviewed at https://reviews.apache.org/r/39572/

19 months agoAlways set SessionKey to empty in the client.
Kevin Sweeney [Thu, 22 Oct 2015 22:47:09 +0000 (15:47 -0700)] 
Always set SessionKey to empty in the client.

Always set SessionKey to empty on the client, as it's now ignored by
the scheduler.

Testing Done:
./pants test.pytest --no-fast src/test/python/::

./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh

Reviewed at https://reviews.apache.org/r/39532/

19 months agoIgnore all SessionKeys.
Kevin Sweeney [Wed, 21 Oct 2015 23:14:15 +0000 (16:14 -0700)] 
Ignore all SessionKeys.

This change ignores all SessionKeys. Since the scheduler was doing
"requireNonNull" on SessionKey before the client needs to continue
to set SessionKey to a non-null value for this release. In the next
release the parameter can be dropped completely.

Apologies for the large diff, thankfully it's mostly red. Coverage in
SchedulerThriftInterface is very high, most of the deleted tests deal
with branches related to authentication or authorization failures,
as these are now enforced in a different layer.

Testing Done:
./graldew -Pq clean build

Reviewed at https://reviews.apache.org/r/39525/

19 months agoAdding job update diff details into "aurora job diff" command.
Maxim Khutornenko [Wed, 21 Oct 2015 18:10:17 +0000 (11:10 -0700)] 
Adding job update diff details into "aurora job diff" command.

Bugs closed: AURORA-1516

Reviewed at https://reviews.apache.org/r/39366/

19 months agoAdding getJobUpdateDiff thrift API.
Maxim Khutornenko [Wed, 14 Oct 2015 00:36:21 +0000 (17:36 -0700)] 
Adding getJobUpdateDiff thrift API.

Bugs closed: AURORA-1515

Reviewed at https://reviews.apache.org/r/39143/

19 months agoConverting to Range in ConfigGroup thrift.
Maxim Khutornenko [Fri, 9 Oct 2015 18:05:11 +0000 (11:05 -0700)] 
Converting to Range in ConfigGroup thrift.

Reviewed at https://reviews.apache.org/r/39150/

19 months agoFix minor inconsistencies in the storage documentation.
Stephan Erb [Fri, 9 Oct 2015 13:00:03 +0000 (14:00 +0100)] 
Fix minor inconsistencies in the storage documentation.

Reviewed at https://reviews.apache.org/r/39169/

19 months agoAdd a note about updating the aurora-packagine repo when changing the gradle version.
Joshua Cohen [Thu, 8 Oct 2015 04:10:14 +0000 (21:10 -0700)] 
Add a note about updating the aurora-packagine repo when changing the gradle version.

Reviewed at https://reviews.apache.org/r/39114/

19 months agoUpgrade to gradle 2.7 and checkstyle 6.11.2.
Bill Farner [Wed, 7 Oct 2015 17:09:43 +0000 (10:09 -0700)] 
Upgrade to gradle 2.7 and checkstyle 6.11.2.

Reviewed at https://reviews.apache.org/r/39095/

19 months agoBetter handling of enums and constructor-only classes in test coverage check.
Bill Farner [Wed, 7 Oct 2015 16:36:08 +0000 (09:36 -0700)] 
Better handling of enums and constructor-only classes in test coverage check.

Reviewed at https://reviews.apache.org/r/39089/

19 months agoUse Shiro to generate audit messages when available.
Kevin Sweeney [Wed, 30 Sep 2015 23:06:53 +0000 (16:06 -0700)] 
Use Shiro to generate audit messages when available.

This ensures that Shiro is always used to generate audit messages when available.

Testing Done:
./gradlew build

Reviewed at https://reviews.apache.org/r/38906/

19 months agoClean up instance id range display. Also remove deprecated max time until running...
Joshua Cohen [Wed, 30 Sep 2015 20:07:50 +0000 (15:07 -0500)] 
Clean up instance id range display. Also remove deprecated max time until running, since it's unused by the server updater.

Reviewed at https://reviews.apache.org/r/38904/