aurora.git
18 months agoUpdating .auroraversion to release version 0.13.0. rel/0.13.0
jfarrell [Fri, 15 Apr 2016 03:41:35 +0000 (23:41 -0400)] 
Updating .auroraversion to release version 0.13.0.

18 months agoUpdating .auroraversion to 0.13.0-rc0.
jfarrell [Tue, 12 Apr 2016 03:23:04 +0000 (23:23 -0400)] 
Updating .auroraversion to 0.13.0-rc0.

18 months agoIncrementing snapshot version to 0.13.1-SNAPSHOT.
jfarrell [Tue, 12 Apr 2016 03:23:04 +0000 (23:23 -0400)] 
Incrementing snapshot version to 0.13.1-SNAPSHOT.

18 months agoUpdating CHANGELOG for 0.13.0 release.
jfarrell [Tue, 12 Apr 2016 03:23:04 +0000 (23:23 -0400)] 
Updating CHANGELOG for 0.13.0 release.

18 months agoRenaming ResourceType.getName()
Maxim Khutornenko [Mon, 11 Apr 2016 23:02:20 +0000 (16:02 -0700)] 
Renaming ResourceType.getName()

Reviewed at https://reviews.apache.org/r/46057/

18 months agoMoving resource-related classes into a new package
Maxim Khutornenko [Mon, 11 Apr 2016 21:45:39 +0000 (14:45 -0700)] 
Moving resource-related classes into a new package

Reviewed at https://reviews.apache.org/r/46051/

18 months agoRemoving unused quota_check.py
Maxim Khutornenko [Mon, 11 Apr 2016 21:43:47 +0000 (14:43 -0700)] 
Removing unused quota_check.py

Reviewed at https://reviews.apache.org/r/46045/

18 months agoRemove support for positional command line arguments,
Bill Farner [Fri, 8 Apr 2016 19:28:43 +0000 (12:28 -0700)] 
Remove support for positional command line arguments,

Reviewed at https://reviews.apache.org/r/45939/

18 months agoRemove support for reading command line argument values from files.
Bill Farner [Fri, 8 Apr 2016 19:20:54 +0000 (12:20 -0700)] 
Remove support for reading command line argument values from files.

Reviewed at https://reviews.apache.org/r/45936/

18 months agoRemove support for canonical command line arg names.
Bill Farner [Fri, 8 Apr 2016 19:03:59 +0000 (12:03 -0700)] 
Remove support for canonical command line arg names.

Reviewed at https://reviews.apache.org/r/45935/

18 months agoFix bug when checking to ensure that all db migrations on the classpath have been...
Joshua Cohen [Fri, 8 Apr 2016 18:48:05 +0000 (13:48 -0500)] 
Fix bug when checking to ensure that all db migrations on the classpath have been applied.

While testing applying the multiple migrations for storing images, I noticed that we were rolling
one back on startup when no migration should be performed.

It turned out I misunderstood the semantics of the Java8 stream API filter/findFirst combination.

Reviewed at https://reviews.apache.org/r/45933/

18 months agoAdd support for automated DB migrations and rollbacks when restoring from dbScript...
Joshua Cohen [Fri, 8 Apr 2016 00:33:24 +0000 (19:33 -0500)] 
Add support for automated DB migrations and rollbacks when restoring from dbScript stored in a
snapshot.

Bugs closed: AURORA-1648

Reviewed at https://reviews.apache.org/r/45816/

18 months agoGeneralize ZooKeeper Test Utils.
John Sirois [Thu, 7 Apr 2016 21:03:08 +0000 (15:03 -0600)] 
Generalize ZooKeeper Test Utils.

This makes way for Curator client based in-process integration tests.

Bugs closed: AURORA-1468

Reviewed at https://reviews.apache.org/r/45886/

18 months agoSet DiscoveryInfo in mesos tasks.
Zhitao Li [Thu, 7 Apr 2016 04:31:48 +0000 (21:31 -0700)] 
Set DiscoveryInfo in mesos tasks.

This allows alternative service discovery methodologies
to find tasks from Aurora (e.g. mesos-dns), especially
the dynamic port mapping.

Bugs closed: AURORA-1629

Reviewed at https://reviews.apache.org/r/45177/

18 months agoFix packer builder to not prompt for input.
Bill Farner [Thu, 7 Apr 2016 02:55:49 +0000 (19:55 -0700)] 
Fix packer builder to not prompt for input.

Reviewed at https://reviews.apache.org/r/45858/

18 months agoFactor out a discovery package.
John Sirois [Thu, 7 Apr 2016 02:46:04 +0000 (20:46 -0600)] 
Factor out a discovery package.

This will be the home for the Curator implementations.

In the process, lift the `ServerSetMonitor` to a top-level class and add
a test. Also tighten up the `ServiceDiscoveryModule` and make
requirements and exports clear.

Bugs closed: AURORA-1468

Reviewed at https://reviews.apache.org/r/45850/

18 months agoSkip flaky test test_health_checker_metrics.
Bill Farner [Thu, 7 Apr 2016 01:06:05 +0000 (18:06 -0700)] 
Skip flaky test test_health_checker_metrics.

Reviewed at https://reviews.apache.org/r/45372/

18 months agoRemove AddInstancesConfig parameter from addInstances RPC.
Bill Farner [Thu, 7 Apr 2016 00:20:29 +0000 (17:20 -0700)] 
Remove AddInstancesConfig parameter from addInstances RPC.

Bugs closed: AURORA-1595

Reviewed at https://reviews.apache.org/r/45725/

18 months agoImplement custom MyBatis cache to record cache statistics.
Zameer Manji [Wed, 6 Apr 2016 23:10:17 +0000 (16:10 -0700)] 
Implement custom MyBatis cache to record cache statistics.

This swaps out our use of the MyBatis default cache with a custom one backed by
Guava. The benefit of this is that we can now plumb cache statistics out which
is helpful in debugging performance problems with MyBatis.

Reviewed at https://reviews.apache.org/r/45821/

18 months agoLift the standard `ServerSet` encoding.
John Sirois [Wed, 6 Apr 2016 22:49:32 +0000 (16:49 -0600)] 
Lift the standard `ServerSet` encoding.

This exposes the standard `ServerSet` `ServiceInstance` encoding to the
`ServerSet` interface for conforming implementations to leverage.

Bugs closed: AURORA-1468

Reviewed at https://reviews.apache.org/r/45829/

18 months agoSlim the `ServerSet` interface.
John Sirois [Wed, 6 Apr 2016 19:11:25 +0000 (13:11 -0600)] 
Slim the `ServerSet` interface.

This makes the path to a Curator implementation to satisfy the contract
with `LeaderRedirect` simpler by introducing `ServiceGroupMonitor`
which hides server set change events, just exposing the current active
set via a query method.  This is all the `LeaderRedirect` needs since
it is driven by user-generated events (HTTP requests).

Bugs closed: AURORA-1468

Reviewed at https://reviews.apache.org/r/45770/

18 months agoRemove TaskQuery from killTasks RPC.
Bill Farner [Wed, 6 Apr 2016 18:55:41 +0000 (11:55 -0700)] 
Remove TaskQuery from killTasks RPC.

Bugs closed: AURORA-1591, AURORA-1592

Reviewed at https://reviews.apache.org/r/45701/

18 months agoSmall Documentation fixes and nitpicks.
Tony Dong [Wed, 6 Apr 2016 17:37:59 +0000 (10:37 -0700)] 
Small Documentation fixes and nitpicks.

Reviewed at https://reviews.apache.org/r/45779/

18 months agoAdding jq to package build.sh.
Zhitao Li [Wed, 6 Apr 2016 15:37:20 +0000 (17:37 +0200)] 
Adding jq to package build.sh.

Reviewed at https://reviews.apache.org/r/45782/

18 months agoRemove lock-related constructs from the API.
Bill Farner [Tue, 5 Apr 2016 22:27:26 +0000 (15:27 -0700)] 
Remove lock-related constructs from the API.

Bugs closed: AURORA-1581

Reviewed at https://reviews.apache.org/r/45718/

18 months agoUse message provided by Mesos if present when tasks fail due to resource limit violat...
Joshua Cohen [Tue, 5 Apr 2016 18:09:56 +0000 (13:09 -0500)] 
Use message provided by Mesos if present when tasks fail due to resource limit violations.

Reviewed at https://reviews.apache.org/r/45754/

18 months agoCreate an ELB-friendly endpoint to detect the leading scheduler.
Ashwin Murthy [Tue, 5 Apr 2016 16:52:40 +0000 (09:52 -0700)] 
Create an ELB-friendly endpoint to detect the leading scheduler.

Reviewed at https://reviews.apache.org/r/45511/

18 months agoExtract a SingletonService interface.
John Sirois [Tue, 5 Apr 2016 02:42:17 +0000 (20:42 -0600)] 
Extract a SingletonService interface.

This makes space for introducing an Apache Curator implementation.

Bugs closed: AURORA-1468

Reviewed at https://reviews.apache.org/r/45723/

18 months agoAdd pants .cache/ dir to .gitignore.
John Sirois [Tue, 5 Apr 2016 00:27:04 +0000 (18:27 -0600)] 
Add pants .cache/ dir to .gitignore.

18 months agoAdd auroraversion to build properties.
Florian Pfeiffer [Mon, 4 Apr 2016 19:09:53 +0000 (14:09 -0500)] 
Add auroraversion to build properties.

Bugs closed: Aurora-1653

Reviewed at https://reviews.apache.org/r/45569/

18 months agoRemove deprecated executor argument '-announcer-enable'
Bill Farner [Mon, 4 Apr 2016 18:53:07 +0000 (11:53 -0700)] 
Remove deprecated executor argument '-announcer-enable'

Also did a slight shuffle of release notes, 2 items shouldn't have been
in the deprecations/removals section.

Bugs closed: AURORA-1606

Reviewed at https://reviews.apache.org/r/45694/

18 months agoUpgrade build tools.
John Sirois [Mon, 4 Apr 2016 17:58:59 +0000 (11:58 -0600)] 
Upgrade build tools.

This brings Gradle to 2.12, release notes here:
  https://docs.gradle.org/2.11/release-notes
  https://docs.gradle.org/2.12/release-notes

Pants is upgraded to 0.0.80, release notes here:
  http://pantsbuild.github.io/changelog.html

Reviewed at https://reviews.apache.org/r/45661/

18 months agoFix language typo in documentation.
se choi [Mon, 4 Apr 2016 16:29:28 +0000 (09:29 -0700)] 
Fix language typo in documentation.

Reviewed at https://reviews.apache.org/r/45675/

18 months agoExecute shell-based health checks as the task user.
Bill Farner [Thu, 31 Mar 2016 21:48:15 +0000 (14:48 -0700)] 
Execute shell-based health checks as the task user.

Bugs closed: AURORA-1641

Reviewed at https://reviews.apache.org/r/45506/

18 months agoIntroduce "preemptible" flag in TierInfo with backward compatible support for "produc...
Amol Deshmukh [Wed, 30 Mar 2016 23:02:20 +0000 (16:02 -0700)] 
Introduce "preemptible" flag in TierInfo with backward compatible support for "production" flag in
TaskConfig.

Bugs closed: AURORA-1616

Reviewed at https://reviews.apache.org/r/45222/

18 months agoAdd ACL support for announcer.
Kunal Thakar [Wed, 30 Mar 2016 20:34:44 +0000 (13:34 -0700)] 
Add ACL support for announcer.

Reviewed at https://reviews.apache.org/r/45042/

18 months agoVarious minor documentation fixes. Most of them dead links.
Stephan Erb [Wed, 30 Mar 2016 19:48:50 +0000 (21:48 +0200)] 
Various minor documentation fixes. Most of them dead links.

Reviewed at https://reviews.apache.org/r/45444/

18 months agoUse correct query to serve /maintenance.
Bill Farner [Wed, 30 Mar 2016 19:15:17 +0000 (12:15 -0700)] 
Use correct query to serve /maintenance.

Fixes the 2 issues identified in AURORA-1652.

Bugs closed: AURORA-1652

Reviewed at https://reviews.apache.org/r/45456/

18 months agoFixes to RELEASE-NOTES for new documentation structure.
George Sirois [Tue, 29 Mar 2016 17:34:34 +0000 (19:34 +0200)] 
Fixes to RELEASE-NOTES for new documentation structure.

Reviewed at https://reviews.apache.org/r/45436/

18 months agoAdds the ability to set HOME to the sandbox before running the executor.
George Sirois [Tue, 29 Mar 2016 16:23:54 +0000 (09:23 -0700)] 
Adds the ability to set HOME to the sandbox before running the executor.

In instances where the root filesystem is read-only, it is desirable to
have the executor/runner extract themselves into the sandbox.

Reviewed at https://reviews.apache.org/r/45396/

18 months agoReorganize Documentation
Stephan Erb [Mon, 28 Mar 2016 20:53:55 +0000 (22:53 +0200)] 
Reorganize Documentation

This started as a spike to structure the documentation in a way that makes it more approachable.
In addition, I believe the new structure will allow us to extend and improve the documentation
more easily, as the different sections have more room to grow into something useful
(eg. service discovery).

The new structure was inspired by the documentation of Hubspot's Singularity scheduler. What I
have done was mostly to cut & paste documentation and code examples and embedded those into the
following:

* getting-started: the most basic information for all users
* features: proper explanation of our most important features. This should make it much easier
  for people to discover Aurora's unique selling points.
* operators: stuff only the operatos will care about
* developers: stuff only contributors and committers acare about.
* references: the details.

Reviewed at https://reviews.apache.org/r/45392/

18 months agoTreat empty and null collections equivalently in task queries.
John Sirois [Mon, 28 Mar 2016 17:11:36 +0000 (11:11 -0600)] 
Treat empty and null collections equivalently in task queries.

Previously, `null` was handled differently from an empty collection in
task queries. For the Go thrift bindings, this was problematic since
zero values in Go are useful in almost all cases and in particular in the
case of maps (used to represent sets).  In these cases unset `TaskQuery`
collection parameters are serialized as empty collections (empty
maps) instead of `nil` (`null`), leading to the inability to use the
query API in any natural way.

Reviewed at https://reviews.apache.org/r/45193/

18 months agoEliminate I* isSet methods for collection fields.
John Sirois [Mon, 28 Mar 2016 15:28:25 +0000 (09:28 -0600)] 
Eliminate I* isSet methods for collection fields.

These methods always returned `true`, which was misleading.  Eliminate
these query methods and force users to consult the collection for
emptiness to make intentions more clear.

Bugs closed: AURORA-1650

Reviewed at https://reviews.apache.org/r/45366/

18 months agoImprove mname and structdump documentation
Stephan Erb [Fri, 25 Mar 2016 14:39:14 +0000 (15:39 +0100)] 
Improve mname and structdump documentation

Reviewed at https://reviews.apache.org/r/45182/

19 months agoFixup install docs.
John Sirois [Wed, 23 Mar 2016 14:45:00 +0000 (08:45 -0600)] 
Fixup install docs.

This set of fixes eliminates the experimental rpm warnings and updates
the rpm instructions to use the officially released packages.  The deb
instructions are updated as well, in particular to take advantage of the
mesosphere deb repository and with movement of special dep installations
to the appropriate sections requiring them.

This fix RB does not address the new Debian Jessie debs, instead
focusing on getting the existing instructions corrected.

Bugs closed: AURORA-1647

Reviewed at https://reviews.apache.org/r/45167/

19 months agoHTTPLifecycleConfig to HttpLifecycleConfig typo
se choi [Wed, 23 Mar 2016 13:03:29 +0000 (07:03 -0600)] 
HTTPLifecycleConfig to HttpLifecycleConfig typo

Reviewed at https://reviews.apache.org/r/45206/

19 months agoEnsure final processes are executed when ephemeral daemon processes exist.
Amol Deshmukh [Wed, 23 Mar 2016 00:22:40 +0000 (17:22 -0700)] 
Ensure final processes are executed when ephemeral daemon processes exist.

Bugs closed: AURORA-1642

Reviewed at https://reviews.apache.org/r/45115/

19 months agoDescheduling a cron should not fail if the job is not scheduled.
Stephan Erb [Tue, 22 Mar 2016 23:23:46 +0000 (00:23 +0100)] 
Descheduling a cron should not fail if the job is not scheduled.

This allows clients to be more declarative rather than imperative
when expressing that the given job should not be scheduled.

Bugs closed: AURORA-1645

Reviewed at https://reviews.apache.org/r/45135/

19 months agoRevert "Add support for storing and fetching images as properties of task configs."
Zameer Manji [Tue, 22 Mar 2016 20:41:07 +0000 (13:41 -0700)] 
Revert "Add support for storing and fetching images as properties of task configs."

This reverts commit d5d7ec0eb5703d6bda8c43cd0586684a550a575a. As outlined in
`AURORA-1648` the schema changes here are not necessarily reflected in H2 after
a restore from backup.

Bugs closed: AURORA-1648

Reviewed at https://reviews.apache.org/r/45178/

19 months agoAdd support for storing and fetching images as properties of task configs.
Joshua Cohen [Tue, 22 Mar 2016 17:25:05 +0000 (12:25 -0500)] 
Add support for storing and fetching images as properties of task configs.

Bugs closed: AURORA-1635

Reviewed at https://reviews.apache.org/r/45112/

19 months agoFix flickering of the Aurora logo on refresh.
Stephan Erb [Tue, 22 Mar 2016 09:44:09 +0000 (10:44 +0100)] 
Fix flickering of the Aurora logo on refresh.

Refreshing a subpage of /scheduler in Chrome temporarily resize the Aurora
logo. This seems to be due to us loading the styles in the <body>
after first rending the logo. Moving the styles to the header fixes this.

Reviewed at https://reviews.apache.org/r/45055/

19 months agoImprove presentation and organization of the scheduler landing page
Stephan Erb [Mon, 21 Mar 2016 22:44:27 +0000 (23:44 +0100)] 
Improve presentation and organization of the scheduler landing page

I have opted for a very simple, incremental improvement here.

Reviewed at https://reviews.apache.org/r/45076/

19 months agoCreate scheduler-configuration.md
Stephan Erb [Fri, 18 Mar 2016 21:20:28 +0000 (22:20 +0100)] 
Create scheduler-configuration.md

Having a simple dump of the all scheduler configuration options makes it much easier to discover or find them via a public search engine.
Ideally we would have a way to update this automatically, but anything is better than not having them listed at all.

Reviewed at https://reviews.apache.org/r/44770/

19 months agoAllow overriding hostname before announcing (https://issues.apache.org/jira/browse...
Kunal Thakar [Fri, 18 Mar 2016 19:14:11 +0000 (20:14 +0100)] 
Allow overriding hostname before announcing (https://issues.apache.org/jira/browse/AURORA-1611)

Reviewed at https://reviews.apache.org/r/44532/

19 months agoSpeed up Vagrant provisioning by using a custom base box.
Bill Farner [Fri, 18 Mar 2016 01:07:00 +0000 (18:07 -0700)] 
Speed up Vagrant provisioning by using a custom base box.

Reviewed at https://reviews.apache.org/r/44871/

19 months agoDo not split the shell command string passed into shell health check script
Dmitriy Shirchenko [Tue, 15 Mar 2016 20:55:38 +0000 (14:55 -0600)] 
Do not split the shell command string passed into shell health check script

Fixing bug where you could not pass in shell command into health checker with environment variables. When environment variable assignement was passed in, only that part would get executed and 0 would always get returned.

Bugs closed: AURORA-1633

Reviewed at https://reviews.apache.org/r/44827/

19 months agoRename NEWS to RELEASE-NOTES.md
Bill Farner [Tue, 15 Mar 2016 02:48:21 +0000 (19:48 -0700)] 
Rename NEWS to RELEASE-NOTES.md

Reviewed at https://reviews.apache.org/r/44806/

19 months agoAdd scheduler support for running tasks using the mesos Docker containerizer.
Bill Farner [Mon, 14 Mar 2016 20:49:13 +0000 (13:49 -0700)] 
Add scheduler support for running tasks using the mesos Docker containerizer.

This is currently labeled as experimental.

Only the most basic wiring is added here, and assumes that the provided image
includes an ENTRYPOINT.  Unlike Docker support via the thermos executor, this
approach allows containers with an entrypoint, and does not impose environment
requirements on the image (e.g. python interpreter, libmesos dependencies).

Note that when using this, other familiar Aurora facilities that relate to the
thermos executor will not work.  For example, browsing task logs is not
supported.

Support for exercising this from the client will come shortly.

Reviewed at https://reviews.apache.org/r/44685/

19 months agoLimit the scope of hot-reload resources.
John Sirois [Fri, 11 Mar 2016 22:30:53 +0000 (15:30 -0700)] 
Limit the scope of hot-reload resources.

This limits the hot-reload resources to those found under
dist/resources/main/scheduler (all the UI resources), in order to
prevent confusion when rebuilding the scheduler for use in the vagrant
vm with non-ui resource changes (say to the db schema.sql).

Reviewed at https://reviews.apache.org/r/44726/

19 months agoExposing DSL defined variables to shell health checkers
Dmitriy Shirchenko [Fri, 11 Mar 2016 21:48:59 +0000 (14:48 -0700)] 
Exposing DSL defined variables to shell health checkers

Bugs closed: AURORA-1622

Reviewed at https://reviews.apache.org/r/44486/

19 months agoLog exceptions raised when thermos running attempts to execute a process.
Joshua Cohen [Fri, 11 Mar 2016 01:41:10 +0000 (19:41 -0600)] 
Log exceptions raised when thermos running attempts to execute a process.

Reviewed at https://reviews.apache.org/r/44680/

19 months agoLog and terminate scheduler on updater thread failure.
Maxim Khutornenko [Thu, 10 Mar 2016 20:32:38 +0000 (12:32 -0800)] 
Log and terminate scheduler on updater thread failure.

Bugs closed: AURORA-1630

Reviewed at https://reviews.apache.org/r/44493/

19 months agoAdding non-role-exclusive dedicated constraint support
Maxim Khutornenko [Thu, 10 Mar 2016 20:19:58 +0000 (12:19 -0800)] 
Adding non-role-exclusive dedicated constraint support

Reviewed at https://reviews.apache.org/r/44602/

19 months agoAdd installation instructions for aurora-cli on Mac OS X
se choi [Thu, 10 Mar 2016 16:37:57 +0000 (08:37 -0800)] 
Add installation instructions for aurora-cli on Mac OS X

Reviewed at https://reviews.apache.org/r/44607/

19 months agoAdd latest design documents
Stephan Erb [Wed, 9 Mar 2016 21:08:13 +0000 (22:08 +0100)] 
Add latest design documents

Reviewed at https://reviews.apache.org/r/44589/

19 months agoUpgrade pants to 0.0.75.
John Sirois [Wed, 9 Mar 2016 20:10:30 +0000 (13:10 -0700)] 
Upgrade pants to 0.0.75.

The only Aurora-impacting change is faster pythonstyle checks.
See the CHANGELOG here: http://pantsbuild.github.io/changelog.html.

Reviewed at https://reviews.apache.org/r/44517/

19 months agoDon't destroy session between requests with TRequestsTransport
Kunal Thakar [Wed, 9 Mar 2016 19:58:13 +0000 (20:58 +0100)] 
Don't destroy session between requests with TRequestsTransport

As an API consumer, I want the API client to reuse open connections and not create a new one for every query.

Reviewed at https://reviews.apache.org/r/42656/

19 months agoUpdate configuration-reference.md documentation on supporting private docker registry.
krish i [Wed, 9 Mar 2016 19:52:52 +0000 (20:52 +0100)] 
Update configuration-reference.md documentation on supporting private docker registry.

There is no parameter in aurora v0.11 named enable_docker_parameters. However, there is a parameter called allow_docker_parameters

Reviewed at https://reviews.apache.org/r/44430/

19 months agoUpgrade ZooKeeper to 3.4.8.
John Sirois [Tue, 8 Mar 2016 22:30:52 +0000 (15:30 -0700)] 
Upgrade ZooKeeper to 3.4.8.

Changes are mainly correcting server bugs, including a fix for server
shutdown deadlocks (ZOOKEEPER-2347). That fix in particular allows
Aurora to upgrade past 3.4.6.

See the release notes here:
  https://zookeeper.apache.org/doc/r3.4.8/releasenotes.html

Reviewed at https://reviews.apache.org/r/44519/

19 months agoFixed bug with LocalSchedulerMain not starting up due to missing Modules.
Amol Deshmukh [Tue, 8 Mar 2016 21:55:12 +0000 (13:55 -0800)] 
Fixed bug with LocalSchedulerMain not starting up due to missing Modules.

Reviewed at https://reviews.apache.org/r/44526/

19 months agoImplementing db snapshotting
Maxim Khutornenko [Tue, 8 Mar 2016 18:10:45 +0000 (10:10 -0800)] 
Implementing db snapshotting

Bugs closed: AURORA-1627

Reviewed at https://reviews.apache.org/r/44471/

19 months agoAURORA-1616: [part 2] make tier_config mandatory argument when starting up the scheduler.
Amol Deshmukh [Fri, 4 Mar 2016 22:42:44 +0000 (14:42 -0800)] 
AURORA-1616: [part 2] make tier_config mandatory argument when starting up the scheduler.

Reviewed at https://reviews.apache.org/r/43925/

19 months agoShow principal in auth failure messaage.
Zameer Manji [Fri, 4 Mar 2016 18:36:10 +0000 (13:36 -0500)] 
Show principal in auth failure messaage.

Bugs closed: AURORA-1628

Reviewed at https://reviews.apache.org/r/44354/

19 months agoInstall thrift package from the offical debs.
Zameer Manji [Fri, 4 Mar 2016 17:46:13 +0000 (12:46 -0500)] 
Install thrift package from the offical debs.

`twopercent` pointed out on IRC that due to the people.a.o change our vagrant
provisioning was broken. This ports the change from aurora-packaging to our
vagrant script.

Reviewed at https://reviews.apache.org/r/44347/

19 months agoUpdating scheduler backup restore instructions.
Maxim Khutornenko [Fri, 4 Mar 2016 17:16:52 +0000 (09:16 -0800)] 
Updating scheduler backup restore instructions.

Bugs closed: AURORA-1605

Reviewed at https://reviews.apache.org/r/43622/

19 months agoDropping bulkLoad() from Storage
Maxim Khutornenko [Fri, 4 Mar 2016 02:36:18 +0000 (18:36 -0800)] 
Dropping bulkLoad() from Storage

Bugs closed: AURORA-1324

Reviewed at https://reviews.apache.org/r/44368/

19 months agoComplete TODO in Tasks.java.
Zameer Manji [Thu, 3 Mar 2016 18:58:43 +0000 (13:58 -0500)] 
Complete TODO in Tasks.java.

This completes the TODO in Tasks.java by removing the unused code.

Reviewed at https://reviews.apache.org/r/44219/

19 months agoUpdate configuration-reference.md documentation on supporting private docker registry.
krish i [Thu, 3 Mar 2016 16:45:08 +0000 (08:45 -0800)] 
Update configuration-reference.md documentation on supporting private docker registry.

Reviewed at https://reviews.apache.org/r/44341/

19 months agoDo not modify list populated by MyBatis.
Zameer Manji [Fri, 26 Feb 2016 23:58:01 +0000 (15:58 -0800)] 
Do not modify list populated by MyBatis.

A ConcurrentModificationException can occur when multiple threads are sorting
the same task event list. This is possible because we have enabled MyBatis
caching with the readOnly flag which can use the same instance to populate
different `DbScheduledTask` instances.

Bugs closed: AURORA-1625

Reviewed at https://reviews.apache.org/r/44106/

19 months agoAURORA-1616: [part 1] refactor tier management initialization to
Amol Deshmukh [Wed, 24 Feb 2016 01:02:55 +0000 (17:02 -0800)] 
AURORA-1616: [part 1] refactor tier management initialization to
make it pluggable in tests. Also, renamed tiers-example.json to tiers.json.

Reviewed at https://reviews.apache.org/r/43669/

20 months agoBump mybatis dependency to 3.3.1. The release notes[1] indicate a few changes
Zameer Manji [Tue, 23 Feb 2016 18:47:17 +0000 (10:47 -0800)] 
Bump mybatis dependency to 3.3.1. The release notes[1] indicate a few changes
including one performance improvement[2].

[1]: http://blog.mybatis.org/2016/02/mybatis-331-and-mybatis-spring-124.html
[2]: https://github.com/mybatis/mybatis-3/issues/574

Reviewed at https://reviews.apache.org/r/43866/

20 months agoGive the task status expand icon a pointer cursor to make it clear that it's clickable.
Joshua Cohen [Mon, 22 Feb 2016 19:40:14 +0000 (13:40 -0600)] 
Give the task status expand icon a pointer cursor to make it clear that it's clickable.

Reviewed at https://reviews.apache.org/r/43784/

20 months agoRemove timestamp from task IDs.
Bill Farner [Fri, 19 Feb 2016 23:12:44 +0000 (15:12 -0800)] 
Remove timestamp from task IDs.

Reviewed at https://reviews.apache.org/r/42896/

20 months agoDo not create a set inside DbTaskStore#fetchTasks.
Zameer Manji [Fri, 19 Feb 2016 20:49:02 +0000 (12:49 -0800)] 
Do not create a set inside DbTaskStore#fetchTasks.

The TaskStore interface allows for an Iterable to be returned. Taking advantage
of this and returning the list MyBatis creates results in a performance
improvement.

Reviewed at https://reviews.apache.org/r/43643/

20 months agoBump virtualenv version to 14.0.6
Stephan Erb [Mon, 15 Feb 2016 18:25:28 +0000 (19:25 +0100)] 
Bump virtualenv version to 14.0.6

Changelog of the new virtualev version is here:
  https://virtualenv.pypa.io/en/latest/changes.html

Important change for us: "Fix an error where virtualenv didn’t pass in a working ssl certificate for pip, causing “weird” errors related to ssl." We have seen this happen at least once.

Reviewed at https://reviews.apache.org/r/43568/

20 months agoAlways close Deflater/Inflater streams
Stephan Erb [Sun, 14 Feb 2016 22:34:10 +0000 (23:34 +0100)] 
Always close Deflater/Inflater streams

Closing Deflater/Inflater streams explicitly frees their native memory instantly. Otherwise, the memory will only be freed once the object finalizer runs.

I got the idea from this article http://www.evanjones.ca/java-native-leak-bug.html. Unfortunately, we cannot use the Java close-with-resource feature, as `TTransport` in our current Thrift version does not implement `Closable`.

Reviewed at https://reviews.apache.org/r/43567/

20 months agoUpdate pants wrapper to match upstream
Benjamin Staffin [Sat, 13 Feb 2016 02:01:30 +0000 (19:01 -0700)] 
Update pants wrapper to match upstream

Picking up this fix:
https://github.com/pantsbuild/setup/commit/990b364cd196cdeee3ac9ec9bc4607df444bbb9f

Makes pants work when a user's homedir is NFS mounted to multiple OSes.

Reviewed at https://reviews.apache.org/r/43554/

20 months agoIncrease throughput of DbTaskStore
Zameer Manji [Fri, 12 Feb 2016 21:32:54 +0000 (13:32 -0800)] 
Increase throughput of DbTaskStore

Profiling master indicated that the bottleneck was MyBatis populating ResultSets
and populating the resulting objects. This patch removes subselects, which
reduces the number of ResultSets and removes the population of an object via a
constructor which is slower than populating an object via setters.

Reviewed at https://reviews.apache.org/r/43457/

20 months agoExplain observer `--mesos-root` configuration.
John Sirois [Fri, 12 Feb 2016 19:42:08 +0000 (12:42 -0700)] 
Explain observer `--mesos-root` configuration.

This adds a note about configuration of the observer's `--mesos-root` to
match the Mesos master `--work_dir` for proper observer functioning.

Bugs closed: AURORA-1617

Reviewed at https://reviews.apache.org/r/43534/

20 months agoMaking OfferManager.getOffers() return a new set rather than a view
Maxim Khutornenko [Fri, 12 Feb 2016 01:31:12 +0000 (17:31 -0800)] 
Making OfferManager.getOffers() return a new set rather than a view

Bugs closed: AURORA-1615

Reviewed at https://reviews.apache.org/r/43503/

20 months agoFix the executor to fail tasks quickly in the event of unknown exceptions when creati...
Joshua Cohen [Thu, 11 Feb 2016 19:23:49 +0000 (13:23 -0600)] 
Fix the executor to fail tasks quickly in the event of unknown exceptions when creating sandboxes.

Bugs closed: AURORA-1614

Reviewed at https://reviews.apache.org/r/43486/

20 months agoDocumenting aurora job add command.
Maxim Khutornenko [Thu, 11 Feb 2016 00:18:41 +0000 (16:18 -0800)] 
Documenting aurora job add command.

Reviewed at https://reviews.apache.org/r/43458/

20 months agoImplementing 'aurora job add' command.
Maxim Khutornenko [Wed, 10 Feb 2016 23:12:36 +0000 (15:12 -0800)] 
Implementing 'aurora job add' command.

Bugs closed: AURORA-1258

Reviewed at https://reviews.apache.org/r/43373/

20 months agoReject NULL for enum values in H2.
Zameer Manji [Wed, 10 Feb 2016 21:22:44 +0000 (13:22 -0800)] 
Reject NULL for enum values in H2.

While I was working on something else, I introduced an error where MyBatis was
populating a thrift enum value and was reading in NULL. I realized that the code
would currently treat this as enum value 0 and mask the error. This patch
modifies the schema and TEnum type handler to reject populating a thrift enum
value from NULL.

Reviewed at https://reviews.apache.org/r/43397/

20 months agoImproving job update query performance.
Maxim Khutornenko [Wed, 10 Feb 2016 03:12:22 +0000 (19:12 -0800)] 
Improving job update query performance.

Reapplying a patch reverted in https://reviews.apache.org/r/42922/

Original RB: https://reviews.apache.org/r/42882/

Bugs closed: AURORA-1600

Reviewed at https://reviews.apache.org/r/43396/

20 months agoReapply deprecated thrift removal patches.
Maxim Khutornenko [Wed, 10 Feb 2016 00:14:40 +0000 (16:14 -0800)] 
Reapply deprecated thrift removal patches.

Bugs closed: AURORA-1603, AURORA-1604

Reviewed at https://reviews.apache.org/r/43375/

20 months agoUpdate to Mesos 0.26.0
Joshua Cohen [Tue, 9 Feb 2016 17:34:18 +0000 (11:34 -0600)] 
Update to Mesos 0.26.0

Release notes: https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.26.0

Reviewed at https://reviews.apache.org/r/43253/

20 months agoFixup release email links.
John Sirois [Tue, 9 Feb 2016 02:25:51 +0000 (19:25 -0700)] 
Fixup release email links.

Previously the tag link did not fail, but pointed to master
and the signature link did fail.

20 months agoFixup release script to tag HEAD of the release branch.
John Sirois [Mon, 8 Feb 2016 22:55:23 +0000 (15:55 -0700)] 
Fixup release script to tag HEAD of the release branch.

Previously it tried to tag the non-existant rel/[version].

Bugs closed: AURORA-1558

Reviewed at https://reviews.apache.org/r/43342/

20 months agoUpgrade to pants 0.0.71.
John Sirois [Mon, 8 Feb 2016 20:19:37 +0000 (13:19 -0700)] 
Upgrade to pants 0.0.71.

This bumps us to yesterday's regular weekly release.
The changelog is here:
  http://pantsbuild.github.io/changelog.html

There is an improvement handling badly behaved python tests; besides
that there are no changes of note directly impacting Aurora.

Reviewed at https://reviews.apache.org/r/43290/