aurora.git
2 years agoUpdating .auroraversion to release version 0.6.0-incubating. rel/0.6.0-incubating
Bill Farner [Tue, 25 Nov 2014 20:45:47 +0000 (12:45 -0800)] 
Updating .auroraversion to release version 0.6.0-incubating.

2 years agoUpdating .auroraversion to 0.6.0-rc2.
Bill Farner [Thu, 13 Nov 2014 22:23:19 +0000 (14:23 -0800)] 
Updating .auroraversion to 0.6.0-rc2.

2 years agoUpdating CHANGELOG for 0.6.0 release.
Bill Farner [Thu, 13 Nov 2014 22:23:19 +0000 (14:23 -0800)] 
Updating CHANGELOG for 0.6.0 release.

2 years agoReset CHANGELOG and .auroraversion in prep for 0.6.0-RC2.
Bill Farner [Thu, 13 Nov 2014 21:20:48 +0000 (13:20 -0800)] 
Reset CHANGELOG and .auroraversion in prep for 0.6.0-RC2.

Reviewed at https://reviews.apache.org/r/28005/

2 years agoSetting the max thread limit on AsyncEventBus.
Maxim Khutornenko [Thu, 13 Nov 2014 20:26:07 +0000 (12:26 -0800)] 
Setting the max thread limit on AsyncEventBus.

Bugs closed: AURORA-926

Reviewed at https://reviews.apache.org/r/27947/

2 years agoUpgrade to gradle 2.2
Bill Farner [Thu, 13 Nov 2014 18:39:52 +0000 (10:39 -0800)] 
Upgrade to gradle 2.2

Reviewed at https://reviews.apache.org/r/27996/

2 years agoFix bad review text in review bot.
Bill Farner [Thu, 13 Nov 2014 18:05:58 +0000 (10:05 -0800)] 
Fix bad review text in review bot.

Reviewed at https://reviews.apache.org/r/27954/

2 years agoDon't fail the build when exceeding coverage thresholds.
Joshua Cohen [Thu, 13 Nov 2014 01:21:05 +0000 (17:21 -0800)] 
Don't fail the build when exceeding coverage thresholds.

Reviewed at https://reviews.apache.org/r/27949/

2 years agoStore host attributes alongside offers to reduce number of lookups.
Bill Farner [Thu, 13 Nov 2014 00:10:03 +0000 (16:10 -0800)] 
Store host attributes alongside offers to reduce number of lookups.

Bugs closed: AURORA-913

Reviewed at https://reviews.apache.org/r/27902/

2 years agoFix review bot to use correct latest diff time.
Bill Farner [Thu, 13 Nov 2014 00:05:25 +0000 (16:05 -0800)] 
Fix review bot to use correct latest diff time.

Reviewed at https://reviews.apache.org/r/27941/

2 years agoRemove non-fixed tickets from CHANGELOG.
Bill Farner [Wed, 12 Nov 2014 23:13:34 +0000 (15:13 -0800)] 
Remove non-fixed tickets from CHANGELOG.

Bugs closed: AURORA-711

Reviewed at https://reviews.apache.org/r/27934/

2 years agoFix coverage epsilon.
Joshua Cohen [Wed, 12 Nov 2014 23:07:40 +0000 (15:07 -0800)] 
Fix coverage epsilon.

BUILD SUCCESSFUL

Reviewed at https://reviews.apache.org/r/27935/

2 years agoAdd documentation for the beta-update command.
Bill Farner [Wed, 12 Nov 2014 21:04:49 +0000 (13:04 -0800)] 
Add documentation for the beta-update command.

Bugs closed: AURORA-763

Reviewed at https://reviews.apache.org/r/27089/

2 years agoFixes for the make pycharm script:
Joshua Cohen [Wed, 12 Nov 2014 19:44:14 +0000 (11:44 -0800)] 
Fixes for the make pycharm script:

1. Escape $'s
2. Use root directory name as project name
3. Exclude some noisy directories

Testing Done:
$ cat .idea/incubator-aurora.iml

Reviewed at https://reviews.apache.org/r/27892/

2 years agoBump commons version.
Zameer Manji [Wed, 12 Nov 2014 01:29:41 +0000 (17:29 -0800)] 
Bump commons version.

Reviewed at https://reviews.apache.org/r/27894/

2 years agoSend an event for host attributes changing rather than maintenance mode changing.
Bill Farner [Wed, 12 Nov 2014 00:08:04 +0000 (16:08 -0800)] 
Send an event for host attributes changing rather than maintenance mode changing.

Bugs closed: AURORA-913

Reviewed at https://reviews.apache.org/r/27889/

2 years agoMove read-modify-write logic out of AttributeStore.Mutable for simpler layering.
Bill Farner [Tue, 11 Nov 2014 19:28:02 +0000 (11:28 -0800)] 
Move read-modify-write logic out of AttributeStore.Mutable for simpler layering.

Bugs closed: AURORA-913

Reviewed at https://reviews.apache.org/r/27828/

2 years agoFix missing paren in review bot.
Bill Farner [Tue, 11 Nov 2014 19:22:11 +0000 (11:22 -0800)] 
Fix missing paren in review bot.

Reviewed at https://reviews.apache.org/r/27880/

2 years agoMake it easier to request another ReviewBot run, and flag diffs that seem to lack...
Bill Farner [Tue, 11 Nov 2014 03:50:33 +0000 (19:50 -0800)] 
Make it easier to request another ReviewBot run, and flag diffs that seem to lack test coverage.

Reviewed at https://reviews.apache.org/r/27770/

2 years agoFix false passing test case in SchedulerDriverServiceTest.
Bill Farner [Tue, 11 Nov 2014 03:42:46 +0000 (19:42 -0800)] 
Fix false passing test case in SchedulerDriverServiceTest.

Reviewed at https://reviews.apache.org/r/27772/

2 years agoCreate the PyCharm project configuration for easier IDE setup.
Bill Farner [Tue, 11 Nov 2014 03:42:00 +0000 (19:42 -0800)] 
Create the PyCharm project configuration for easier IDE setup.

Reviewed at https://reviews.apache.org/r/27779/

2 years agoAdd test coverage for WriteAheadStorage.
Bill Farner [Tue, 11 Nov 2014 03:32:10 +0000 (19:32 -0800)] 
Add test coverage for WriteAheadStorage.

Reviewed at https://reviews.apache.org/r/27542/

2 years agoRemove stracktrace redirection.
Zameer Manji [Tue, 11 Nov 2014 01:10:41 +0000 (17:10 -0800)] 
Remove stracktrace redirection.

Bugs closed: AURORA-896

Reviewed at https://reviews.apache.org/r/27710/

2 years agoPrint out the job url after scheduling a cron job.
Zameer Manji [Mon, 10 Nov 2014 22:42:08 +0000 (14:42 -0800)] 
Print out the job url after scheduling a cron job.

Bugs closed: AURORA-857

Reviewed at https://reviews.apache.org/r/27657/

2 years agoIncrementing snapshot version to 0.6.1-SNAPSHOT.
Bill Farner [Mon, 10 Nov 2014 22:20:31 +0000 (14:20 -0800)] 
Incrementing snapshot version to 0.6.1-SNAPSHOT.

2 years agoUpdating CHANGELOG for 0.6.0 release.
Bill Farner [Mon, 10 Nov 2014 22:20:31 +0000 (14:20 -0800)] 
Updating CHANGELOG for 0.6.0 release.

2 years agoUndo 0.6.0 .auroraversion and CHANGELOG changes to prepare for 0.6.0-RC1
Bill Farner [Mon, 10 Nov 2014 21:16:00 +0000 (13:16 -0800)] 
Undo 0.6.0 .auroraversion and CHANGELOG changes to prepare for 0.6.0-RC1

Bugs closed: AURORA-711

Reviewed at https://reviews.apache.org/r/27830/

2 years agoUse async eventbus
Kevin Sweeney [Sat, 8 Nov 2014 00:54:12 +0000 (16:54 -0800)] 
Use async eventbus

This change uses AsyncEventBus in production code. Some test code
still expects eventSink.post() to block. I plan to address in a
follow-up review.

Testing Done:
./gradlew -Pq build

Bugs closed: AURORA-920

Reviewed at https://reviews.apache.org/r/27762/

2 years agoSimplify management of the driver lifecycle using AbstractidleService.
Bill Farner [Fri, 7 Nov 2014 23:08:35 +0000 (15:08 -0800)] 
Simplify management of the driver lifecycle using AbstractidleService.

Bugs closed: AURORA-920

Reviewed at https://reviews.apache.org/r/27746/

2 years agoFixing beta-update OK status messaging.
Maxim Khutornenko [Fri, 7 Nov 2014 22:09:16 +0000 (14:09 -0800)] 
Fixing beta-update OK status messaging.

Bugs closed: AURORA-784

Reviewed at https://reviews.apache.org/r/27712/

2 years agoFixing the log replay for the job update history pruner.
Maxim Khutornenko [Fri, 7 Nov 2014 22:05:11 +0000 (14:05 -0800)] 
Fixing the log replay for the job update history pruner.

Bugs closed: AURORA-912

Reviewed at https://reviews.apache.org/r/27598/

2 years agoRemove SchedulerActive, replace with explicit services.
Kevin Sweeney [Fri, 7 Nov 2014 04:06:05 +0000 (20:06 -0800)] 
Remove SchedulerActive, replace with explicit services.

This is the first part of a larger change that will allow us to
make pubsub events async as it makes the dependency on subscribers
processing a SchedulerActive event explicit.

Testing Done:
./gradlew -Pq build
test_end_to_end.sh

Bugs closed: AURORA-920

Reviewed at https://reviews.apache.org/r/27716/

2 years agoReplace twitter.commons.io.FileUtils dependency w/ guava's Files
Joshua Cohen [Thu, 6 Nov 2014 20:49:52 +0000 (12:49 -0800)] 
Replace twitter.commons.io.FileUtils dependency w/ guava's Files

Reviewed at https://reviews.apache.org/r/27545/

2 years agoMark inotify disk collector test as flaky.
Joshua Cohen [Thu, 6 Nov 2014 20:47:39 +0000 (12:47 -0800)] 
Mark inotify disk collector test as flaky.

Bugs closed: AURORA-916

Reviewed at https://reviews.apache.org/r/27630/

2 years agoRemove duplicate call to log handling code. This duplicate call added another
Zameer Manji [Thu, 6 Nov 2014 20:39:32 +0000 (12:39 -0800)] 
Remove duplicate call to log handling code. This duplicate call added another
stderr handler to the logger which caused each message to be printed twice to
stderr.

Bugs closed: AURORA-793

Reviewed at https://reviews.apache.org/r/27650/

2 years agoConvert most uses of Mock to create_autospec, remove some uses of mocking altogether.
Bill Farner [Thu, 6 Nov 2014 05:06:40 +0000 (21:06 -0800)] 
Convert most uses of Mock to create_autospec, remove some uses of mocking altogether.

Bugs closed: AURORA-889

Reviewed at https://reviews.apache.org/r/27628/

2 years agoIncrease robustness when reading PEX-INFO
Zameer Manji [Wed, 5 Nov 2014 22:29:26 +0000 (14:29 -0800)] 
Increase robustness when reading PEX-INFO

Bugs closed: AURORA-456

Reviewed at https://reviews.apache.org/r/27586/

2 years agoRemove --bind-var for cron deschedule.
Zameer Manji [Wed, 5 Nov 2014 19:25:32 +0000 (11:25 -0800)] 
Remove --bind-var for cron deschedule.

Bugs closed: AURORA-854

Reviewed at https://reviews.apache.org/r/27591/

2 years agoRetry Driver.killTask periodically until the task changes states.
Bill Farner [Tue, 4 Nov 2014 21:59:24 +0000 (13:59 -0800)] 
Retry Driver.killTask periodically until the task changes states.

Bugs closed: AURORA-910

Reviewed at https://reviews.apache.org/r/27467/

2 years agoRevert "Adding resource consumption calculation for cron jobs."
Maxim Khutornenko [Mon, 3 Nov 2014 22:05:01 +0000 (14:05 -0800)] 
Revert "Adding resource consumption calculation for cron jobs."

This reverts commit 37cb06335a834de1894c882810a822f5f2ef69bc.

Reviewed at https://reviews.apache.org/r/27541/

2 years agoOptionally ignore unknown transaction operations.
Bill Farner [Mon, 3 Nov 2014 20:37:19 +0000 (12:37 -0800)] 
Optionally ignore unknown transaction operations.

Reviewed at https://reviews.apache.org/r/27538/

2 years agoCleaning up old vagrant cluster names.
Maxim Khutornenko [Mon, 3 Nov 2014 18:54:58 +0000 (10:54 -0800)] 
Cleaning up old vagrant cluster names.

Reviewed at https://reviews.apache.org/r/27352/

2 years agoAdding resource consumption calculation for cron jobs.
Maxim Khutornenko [Mon, 3 Nov 2014 18:53:06 +0000 (10:53 -0800)] 
Adding resource consumption calculation for cron jobs.

Bugs closed: AURORA-825

Reviewed at https://reviews.apache.org/r/27317/

2 years agoRemove dependency on application-http.
Bill Farner [Fri, 31 Oct 2014 21:39:49 +0000 (14:39 -0700)] 
Remove dependency on application-http.

Reviewed at https://reviews.apache.org/r/27375/

2 years agoAdd stat gauge coverage for AsyncModule and MemStorage.
Bill Farner [Fri, 31 Oct 2014 21:00:39 +0000 (14:00 -0700)] 
Add stat gauge coverage for AsyncModule and MemStorage.

Bugs closed: AURORA-884

Reviewed at https://reviews.apache.org/r/27371/

2 years agoDocument thermos checkpoint files, include them in git archives.
Bill Farner [Fri, 31 Oct 2014 00:16:11 +0000 (17:16 -0700)] 
Document thermos checkpoint files, include them in git archives.

Bugs closed: AURORA-711

Reviewed at https://reviews.apache.org/r/27357/

2 years agoAdd a script to generate a PyCharm-compatible virtualenv.
Kevin Sweeney [Thu, 30 Oct 2014 18:10:34 +0000 (11:10 -0700)] 
Add a script to generate a PyCharm-compatible virtualenv.

Note there are a few changes to test code to allow running without pants as the test runner:

* Use relative imports in test packages.
* Added blank `__init__.py` files to test directories that need to use relative imports now.
* Catch PexInfo errors that result from not running inside a PEX.
* Add requirements.txt to .gitignore (we generate it from pants) - it helps
pycharm offer quick fixes and identify packages with updates available.

To create a PyCharm project:

Set the project interpreter to `build-support/python/pycharm.venv/bin/python`.
Mark `src/main/python` as a Sources Root.
Go to Settings > Python Integrated Tools and set `Default test runner` to `py.test`.

Testing Done:
./pants src/test/python:all

./build-support/python/make-pycharm-virtualenv
source build-support/python/pycharm.venv/bin/activate
PYTHONPATH=src/main/python py.test src/test/python

Tested PyCharm Community Edition 3.4. Basic navigation (Ctrl-click) works. Find usages
works. Py.Test works if configured manually. Not a great experience, but better than no IDE
whatsoever for some things.

Reviewed at https://reviews.apache.org/r/27364/

2 years agoAdd FakeStatsProvider and check stat values in SchedulerLifecycleTest.
Bill Farner [Thu, 30 Oct 2014 01:36:52 +0000 (18:36 -0700)] 
Add FakeStatsProvider and check stat values in SchedulerLifecycleTest.

Bugs closed: AURORA-884

Reviewed at https://reviews.apache.org/r/27365/

2 years agoAdd explicit test coverage for stat gauge in TaskSchedulerImpl.
Bill Farner [Thu, 30 Oct 2014 00:45:04 +0000 (17:45 -0700)] 
Add explicit test coverage for stat gauge in TaskSchedulerImpl.

Bugs closed: AURORA-884

Reviewed at https://reviews.apache.org/r/27363/

2 years agoMake executor overhead configurable via CLI.
Zameer Manji [Wed, 29 Oct 2014 21:56:43 +0000 (14:56 -0700)] 
Make executor overhead configurable via CLI.

Bugs closed: AURORA-830

Reviewed at https://reviews.apache.org/r/27309/

2 years agoFix minor discrepancies in RC script.
Bill Farner [Tue, 28 Oct 2014 23:26:58 +0000 (16:26 -0700)] 
Fix minor discrepancies in RC script.

Reviewed at https://reviews.apache.org/r/27316/

2 years agoIncrementing snapshot version to 0.6.1-SNAPSHOT.
Bill Farner [Tue, 28 Oct 2014 22:41:07 +0000 (15:41 -0700)] 
Incrementing snapshot version to 0.6.1-SNAPSHOT.

2 years agoUpdating CHANGELOG for 0.6.0 release.
Bill Farner [Tue, 28 Oct 2014 22:41:07 +0000 (15:41 -0700)] 
Updating CHANGELOG for 0.6.0 release.

2 years agoAdding missing JobConfiguration backfill for task config job field.
Maxim Khutornenko [Tue, 28 Oct 2014 20:55:40 +0000 (13:55 -0700)] 
Adding missing JobConfiguration backfill for task config job field.

Bugs closed: AURORA-899

Reviewed at https://reviews.apache.org/r/27262/

2 years agoFixing transient error message formatting in scheduler_client.py
Maxim Khutornenko [Tue, 28 Oct 2014 20:54:27 +0000 (13:54 -0700)] 
Fixing transient error message formatting in scheduler_client.py

Bugs closed: AURORA-900

Reviewed at https://reviews.apache.org/r/27266/

2 years agoFixing error reporting in v1 client updater.
Maxim Khutornenko [Tue, 28 Oct 2014 20:53:09 +0000 (13:53 -0700)] 
Fixing error reporting in v1 client updater.

Bugs closed: AURORA-901

Reviewed at https://reviews.apache.org/r/27289/

2 years agoMake job key backfilling more robust.
Bill Farner [Tue, 28 Oct 2014 01:05:57 +0000 (18:05 -0700)] 
Make job key backfilling more robust.

Bugs closed: AURORA-897

Reviewed at https://reviews.apache.org/r/27258/

2 years agoRaise .auroraversion to 0.6.0-SNAPSHOT
Bill Farner [Tue, 28 Oct 2014 00:48:08 +0000 (17:48 -0700)] 
Raise .auroraversion to 0.6.0-SNAPSHOT

Bugs closed: AURORA-711

Reviewed at https://reviews.apache.org/r/27253/

2 years agoUse ship-it instead of +/-1.
Bill Farner [Mon, 27 Oct 2014 18:51:52 +0000 (11:51 -0700)] 
Use ship-it instead of +/-1.

Reviewed at https://reviews.apache.org/r/27188/

2 years agoFix error when "job create" is called with "--open-browser".
Mark Chu-Carroll [Mon, 27 Oct 2014 15:08:12 +0000 (11:08 -0400)] 
Fix error when "job create" is called with "--open-browser".

Reviewed at https://reviews.apache.org/r/27084/

2 years agoAllow the invoker to ignore patterns in git-clean.
Bill Farner [Fri, 24 Oct 2014 23:26:07 +0000 (16:26 -0700)] 
Allow the invoker to ignore patterns in git-clean.

Reviewed at https://reviews.apache.org/r/27181/

2 years agoCache the host's maintenance status with offer.
Zameer Manji [Fri, 24 Oct 2014 23:11:31 +0000 (16:11 -0700)] 
Cache the host's maintenance status with offer.

Bugs closed: AURORA-878

Reviewed at https://reviews.apache.org/r/27100/

2 years agoAdd a script that publishes build results to review board.
Bill Farner [Fri, 24 Oct 2014 22:43:04 +0000 (15:43 -0700)] 
Add a script that publishes build results to review board.

Bugs closed: AURORA-883

Reviewed at https://reviews.apache.org/r/27145/

2 years agoMove from github to bintray for pants support binaries.
Joe Smith [Fri, 24 Oct 2014 22:13:21 +0000 (15:13 -0700)] 
Move from github to bintray for pants support binaries.

Reviewed at https://reviews.apache.org/r/27114/

2 years agoUpgrade psutil to 2.1.3
Joe Smith [Fri, 24 Oct 2014 20:24:04 +0000 (13:24 -0700)] 
Upgrade psutil to 2.1.3

Testing Done:
e2e tests and on OS X 10.10 and from within the vagrant image:

Bugs closed: AURORA-627

Reviewed at https://reviews.apache.org/r/27129/

2 years agoAdd Junit XML output for Python tests.
Kevin Sweeney [Fri, 24 Oct 2014 01:04:27 +0000 (18:04 -0700)] 
Add Junit XML output for Python tests.

Verified that this is the correct directory in Jenkins by peeking
over wfarner's shoulder.

Attempted to add coverage as well but it looks like pants doesn't
support anything beyond console output yet (even though pycov supports
coberatura-style output).

Testing Done:
./build-support/jenkins/build.sh

Reviewed at https://reviews.apache.org/r/27098/

2 years agoUpdating storage.md with late review comments.
Maxim Khutornenko [Fri, 24 Oct 2014 01:00:17 +0000 (18:00 -0700)] 
Updating storage.md with late review comments.

Bugs closed: AURORA-839

Reviewed at https://reviews.apache.org/r/26845/

2 years agoAdding storage configuration and recovery from backup doc.
Maxim Khutornenko [Fri, 24 Oct 2014 00:18:43 +0000 (17:18 -0700)] 
Adding storage configuration and recovery from backup doc.

Bugs closed: AURORA-839

Reviewed at https://reviews.apache.org/r/26851/

2 years agoPreparing for Identity struct deprecation (client and executor).
Maxim Khutornenko [Thu, 23 Oct 2014 23:00:38 +0000 (16:00 -0700)] 
Preparing for Identity struct deprecation (client and executor).

Bugs closed: AURORA-84

Reviewed at https://reviews.apache.org/r/26954/

2 years agoPreparing for Identity struct deprecation (scheduler).
Maxim Khutornenko [Thu, 23 Oct 2014 23:00:10 +0000 (16:00 -0700)] 
Preparing for Identity struct deprecation (scheduler).

Bugs closed: AURORA-84

Reviewed at https://reviews.apache.org/r/26762/

2 years agoAdd a script to update build-support/pants_requirements.txt.
Joshua Cohen [Thu, 23 Oct 2014 18:45:49 +0000 (11:45 -0700)] 
Add a script to update build-support/pants_requirements.txt.
Ran that script and it generated a slightly different output (no more
system dependencies).

Also fix a bug in the pants script that causes us to bootstrap pants
on every run (missing $PANTS_VERSION).

Testing Done:
Ran the update requirements script and ran pants multiple times to
confirm it doesn't bootstrap.

Reviewed at https://reviews.apache.org/r/27097/

2 years agoMake "job create" return an error if the job wasn't created successfully.
Mark Chu-Carroll [Thu, 23 Oct 2014 18:46:02 +0000 (14:46 -0400)] 
Make "job create" return an error if the job wasn't created successfully.

"job create" now calls the api check_status method, and generates an error message like the following if it doesn't find the created job:

     Error occurred while creating job west/bozo/test/hello

Bugs closed: aurora-886

Reviewed at https://reviews.apache.org/r/27082/

2 years agoFollow the pantsbuild pants_support_baseurls for OS X Yosemite support.
Joe Smith [Thu, 23 Oct 2014 18:23:29 +0000 (11:23 -0700)] 
Follow the pantsbuild pants_support_baseurls for OS X Yosemite support.

Testing Done:
22:19:24 incubator-aurora $ ./pants ./src/test/python/apache/aurora/admin:all

Bugs closed: AURORA-867

Reviewed at https://reviews.apache.org/r/27066/

2 years agoImprove status command output ordering.
Mark Chu-Carroll [Thu, 23 Oct 2014 17:23:15 +0000 (13:23 -0400)] 
Improve status command output ordering.

- Sort tasks by instance number.
- Sort events by timestamp.

Bugs closed: aurora-879

Reviewed at https://reviews.apache.org/r/27047/

2 years agoDefining schema for the heartbeat RPC.
Maxim Khutornenko [Thu, 23 Oct 2014 17:10:30 +0000 (10:10 -0700)] 
Defining schema for the heartbeat RPC.

Bugs closed: AURORA-690

Reviewed at https://reviews.apache.org/r/26531/

2 years agoFreeze pants requirements.
Kevin Sweeney [Thu, 23 Oct 2014 17:00:51 +0000 (10:00 -0700)] 
Freeze pants requirements.

This will help us avoid flaky builds in the future (some of pants's
transitive dependencies use unqualified package names).

Testing Done:
rm -fr build-support/pants.venv
./pants src/test/python:all

Reviewed at https://reviews.apache.org/r/27063/

2 years agoFix typo in pants bash script.
Kevin Sweeney [Thu, 23 Oct 2014 00:45:12 +0000 (17:45 -0700)] 
Fix typo in pants bash script.

Forgot to update the review at https://reviews.apache.org/r/27062/
to match my local branch.

Testing Done:
% rm -fr build-support/pants.venv
% ./pants
% ls -l build-support/pants.venv
total 16
drwxrwxr-x 2 ksweeney ksweeney 4096 Oct 22 17:37 bin
-rw-rw-r-- 1 ksweeney ksweeney    7 Oct 22 17:37 BOOTSTRAPPED
drwxrwxr-x 2 ksweeney ksweeney 4096 Oct 22 17:36 include
drwxrwxr-x 3 ksweeney ksweeney 4096 Oct 22 17:36 lib

Reviewed at https://reviews.apache.org/r/27062/

2 years agoUpgrade virtualenv to 1.11.6.
Kevin Sweeney [Thu, 23 Oct 2014 00:18:05 +0000 (17:18 -0700)] 
Upgrade virtualenv to 1.11.6.

See changelog: https://virtualenv.pypa.io/en/latest/news.html

Of note, this upgrades the bundled pip to one that supports https proxies.

Testing Done:
git clean -fdx
./pants src/test/python:all

Reviewed at https://reviews.apache.org/r/27057/

2 years agoAdd some wiggle room before requiring min coverage thresholds be raised.
Joshua Cohen [Wed, 22 Oct 2014 23:11:29 +0000 (16:11 -0700)] 
Add some wiggle room before requiring min coverage thresholds be raised.

Bugs closed: AURORA-880

Reviewed at https://reviews.apache.org/r/27045/

2 years agoUse virtualenv to build pants instead of pex.
Kevin Sweeney [Wed, 22 Oct 2014 21:14:29 +0000 (14:14 -0700)] 
Use virtualenv to build pants instead of pex.

The goal of this change is to reduce CI flakiness by using `pip
install` rather than `pex` to bootstrap pants. This still reaches out
to external servers (and thus has the potential to be flaky) but is
at least configurable (via `--default-timeout`). In a future review
we can consider mirroring to svn.apache.org and disabling PyPI lookups.

This requires a change to production executor code to use
`sys.executable` rather than chmod+x-ing the runner PEX. Failure to
do this results in the pants virtualenv's `site-packages` shadowing
the dependencies in the `pex` (fairly obviously as there is a test
failure related to psutil1/2 conflicts). I'm sure I'm papering over
a PEX bug here but IMO this is fine as the only reason for it is to
execute the thermos runner PEX under a different python interpreter
than the executor is using (based on the hash-bang in the pex). This
is not a configuration we need to support IMO.

Testing Done:
git clean -fdx
./pants src/test/python:all

Bugs closed: AURORA-876

Reviewed at https://reviews.apache.org/r/27009/

2 years agoUpgrade mesos to 0.20.1
Kevin Sweeney [Wed, 22 Oct 2014 18:10:31 +0000 (11:10 -0700)] 
Upgrade mesos to 0.20.1

See https://github.com/apache/mesos/blob/master/CHANGELOG

Testing Done:
./gradlew -Pq build
./pants src/test/python:all
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh

Bugs closed: AURORA-875

Reviewed at https://reviews.apache.org/r/26991/

2 years agoRe-building Aurora components before running e2e tests.
Maxim Khutornenko [Wed, 22 Oct 2014 16:05:09 +0000 (09:05 -0700)] 
Re-building Aurora components before running e2e tests.

Bugs closed: AURORA-498

Reviewed at https://reviews.apache.org/r/26998/

2 years agoAdding quota check into scheduleCronJob RPC.
Maxim Khutornenko [Wed, 22 Oct 2014 16:01:30 +0000 (09:01 -0700)] 
Adding quota check into scheduleCronJob RPC.

Bugs closed: AURORA-824

Reviewed at https://reviews.apache.org/r/26997/

2 years agoFixing python style check issues.
Maxim Khutornenko [Tue, 21 Oct 2014 22:45:07 +0000 (15:45 -0700)] 
Fixing python style check issues.

Fixing build broken by https://reviews.apache.org/r/26363 due
to python style check issues.

Reviewed at https://reviews.apache.org/r/27001/

2 years agoPrinting coverage report path on failed analysis.
Maxim Khutornenko [Tue, 21 Oct 2014 21:34:52 +0000 (14:34 -0700)] 
Printing coverage report path on failed analysis.

Reviewed at https://reviews.apache.org/r/26985/

2 years agoDownload thrift from archive.a.o.
Kevin Sweeney [Tue, 21 Oct 2014 21:30:09 +0000 (14:30 -0700)] 
Download thrift from archive.a.o.

The us.apache.org URL 404s and breaks our build.

Testing Done:
./gradlew bootstrapThrift

Bugs closed: AURORA-877

Reviewed at https://reviews.apache.org/r/26995/

2 years agoMake the large-update check in the client update command consider instance parameters.
Mark Chu-Carroll [Tue, 21 Oct 2014 19:57:33 +0000 (15:57 -0400)] 
Make the large-update check in the client update command consider instance parameters.

Bugs closed: aurora-792

Reviewed at https://reviews.apache.org/r/26363/

2 years agoUse toLocalTime angular filter rather than duplicate logic in controller.
David McLaughlin [Tue, 21 Oct 2014 18:21:20 +0000 (11:21 -0700)] 
Use toLocalTime angular filter rather than duplicate logic in controller.

Bugs closed: AURORA-873

Reviewed at https://reviews.apache.org/r/26956/

2 years agoScript to build mesos.native egg in clean vagrant environment
Kevin Sweeney [Tue, 21 Oct 2014 18:14:13 +0000 (11:14 -0700)] 
Script to build mesos.native egg in clean vagrant environment

Our integration tests have been broken recently by a lack of available
precompiled mesos.native eggs. This patch adds a script to build them
so that anybody can upgrade our mesos version without breaking our
e2e tests.

Added README to
https://svn.apache.org/repos/asf/incubator/aurora/3rdparty

Testing Done:
eggs in https://svn.apache.org/repos/asf/incubator/aurora/3rdparty
were built with this script.

Reviewed at https://reviews.apache.org/r/26962/

2 years agoMake v2 job status look more like v1.
Mark Chu-Carroll [Tue, 21 Oct 2014 18:05:43 +0000 (14:05 -0400)] 
Make v2 job status look more like v1.

Adds a header line to each task in a task status line specifying the
instance number, host, etc., the way v1 did. (We had originally
planned not to display that information in v2, but we also have tests
that depend on it being present.)

Bugs closed: AURORA-874

Reviewed at https://reviews.apache.org/r/26982/

2 years agoFix checkstyle.
Kevin Sweeney [Tue, 21 Oct 2014 00:56:36 +0000 (17:56 -0700)] 
Fix checkstyle.

Testing Done:
build-support/python/checkstyle-check src

Reviewed at https://reviews.apache.org/r/26961/

2 years agoUse mesos.native egg from svn.apache.org
Kevin Sweeney [Mon, 20 Oct 2014 22:52:57 +0000 (15:52 -0700)] 
Use mesos.native egg from svn.apache.org

Testing Done:
% ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
OK (all tests passed)

Bugs closed: AURORA-863

Reviewed at https://reviews.apache.org/r/26852/

2 years agoImprove error messages in client commands.
Mark Chu-Carroll [Mon, 20 Oct 2014 19:42:47 +0000 (15:42 -0400)] 
Improve error messages in client commands.

When a client command fails due to an error in an API call,
it used to log whatever messages came back in the reply from the server,
and then generate a message saying "... see log for details".

This change improves that behavior. Now, an error message is generated
saying something like "Command failed due to error reported by server",
followed by a group of indented lines containing the messages from the
server. For example, a new error message would look like:

Server reported error restarting job west/bozo/test/hello
    Job 'west/bozo/test/hello' not found.

Bugs closed: aurora-771

Reviewed at https://reviews.apache.org/r/26881/

2 years agoDocument common problems and solutions when creating a new cluster.
Bill Farner [Mon, 20 Oct 2014 19:35:54 +0000 (12:35 -0700)] 
Document common problems and solutions when creating a new cluster.

Bugs closed: AURORA-840

Reviewed at https://reviews.apache.org/r/26832/

2 years agoAdd a message when coverage exceeds minimum.
Joshua Cohen [Mon, 20 Oct 2014 18:37:07 +0000 (11:37 -0700)] 
Add a message when coverage exceeds minimum.

This way we'll be sure to continually raise the minimum threshold as we exceed it.

Reviewed at https://reviews.apache.org/r/26902/

2 years agoDecouple initial scheduling delay from first scheduling penalty.
Joshua Cohen [Sat, 18 Oct 2014 00:18:21 +0000 (17:18 -0700)] 
Decouple initial scheduling delay from first scheduling penalty.

Bugs closed: AURORA-868

Reviewed at https://reviews.apache.org/r/26901/

2 years agoAdd client cluster configuration docs.
Joshua Cohen [Fri, 17 Oct 2014 20:50:08 +0000 (13:50 -0700)] 
Add client cluster configuration docs.

Bugs closed: AURORA-846

Reviewed at https://reviews.apache.org/r/26834/

2 years agoMoving post_drain script execution into host_maintenance.py
Maxim Khutornenko [Fri, 17 Oct 2014 20:03:08 +0000 (13:03 -0700)] 
Moving post_drain script execution into host_maintenance.py

Bugs closed: AURORA-806

Reviewed at https://reviews.apache.org/r/26431/

2 years agoAdding wait loop into host_drain status monitoring.
Maxim Khutornenko [Fri, 17 Oct 2014 19:53:36 +0000 (12:53 -0700)] 
Adding wait loop into host_drain status monitoring.

Bugs closed: AURORA-820

Reviewed at https://reviews.apache.org/r/26458/

2 years agoAdding executor health check disabler
David Pan [Fri, 17 Oct 2014 19:32:51 +0000 (12:32 -0700)] 
Adding executor health check disabler

The health check disabler allows health checks for a job
to be snoozed temporarily by touching a snooze file in
the job's sandbox.

Bugs closed: AURORA-795

Reviewed at https://reviews.apache.org/r/26383/