swarmbucket: allow overriding config

swarmbucket: allow overriding config

swarming.override_builder_cfg can override most of builder configuration
defined on the server. This allows users to specify dimensions
dynamically.

The only config value that cannot be changed is builder name. Builder
names will remain static.

Also change the way we undefine properties: instead of 
properties_j: "foo:"
do
properties_j: "foo:null"

This prevents from having null property values, but it is fine.


BUG=

Committed: https://chromium.googlesource.com/infra/infra/+/16eca9957490bed920371d6fcad67a8aeb2d17ac

[nodir, 2016-08-04 01:00:44]

Description was changed from

==========
swarmbucket: allow overriding config

swarming.override_builder_cfg can override most of builder configuration
defined on the server. This allows users to specify dimensions
dynamically.

The only config value that cannot be changed is builder name. Builder
names will remain static.

BUG=
==========

to

==========
swarmbucket: allow overriding config

swarming.override_builder_cfg can override most of builder configuration
defined on the server. This allows users to specify dimensions
dynamically.

The only config value that cannot be changed is builder name. Builder
names will remain static.

Also change the way we undefine properties: instead of 
properties_j: "foo:"
do
properties_j: "foo:null"

This prevents from having null property values, but it is fine.


BUG=
==========

[nodir, 2016-08-04 01:01:58]

nodir@chromium.org changed reviewers:
+ borenet@google.com, vadimsh@chromium.org

[nodir, 2016-08-04 01:01:59]

PTAL

Skia needs to set dimensions dynamically, I think it is reasonable

Example of request:
https://apis-explorer.appspot.com/apis-explorer/?base=https://5238-8c8fd5a-ta...

Scheduled with 64 cores:
https://chromium-swarm-dev.appspot.com/user/task/306be2fa8dcb7e10

[nodir, 2016-08-04 21:29:28]

ping

[borenet, 2016-08-05 14:53:54]

On 2016/08/04 21:29:28, nodir wrote:
> ping

(expressing my support for this CL without any real review authority)

[Vadim Sh., 2016-08-05 18:24:17]

Can this be made configurable? At least with a simple boolean switch in bucket
config: "overiddable: true" (which is false by default).

Currently buckets are sort of access control restriction. By defining a bucket
we say that only such-and-such jobs can be issued by whoever can write to the
bucket. I'm concerned that with overrides, users may start abusing them.

[nodir, 2016-08-05 18:49:44]

On 2016/08/05 18:24:17, Vadim Sh. wrote:
> Can this be made configurable? At least with a simple boolean switch in bucket
> config: "overiddable: true" (which is false by default).
> 
> Currently buckets are sort of access control restriction. By defining a bucket
> we say that only such-and-such jobs can be issued by whoever can write to the
> bucket. 

this invariant still holds:
1) cfg overriding code won't be executed unless the requester has permissions to
schedule a build to that bucket
2) you cannot override ACLs anyway. In fact the only thing you can override that
you could not override before this CLs is dimensions: users were always able to
specify properties (that control the behavior) and tags

i find it important the default to be that users can specify which bot they want
to run a build on and that's "id" dimension. I could make separate
"swarming.bot_id" parameter to restrict access only to that dimension, but it is
somewhat ugly

There is a possibility that we may add sensitive settings in builder in future.
I've added a SECURITY WARNING. Perhaps we will do something like

int sensetive_field = 1;
bool sensetive_field_overridable = 2; // false by default

wdyt?

[Vadim Sh., 2016-08-05 19:00:07]

lgtm *shrug*

[nodir, 2016-08-05 19:19:09]

The CQ bit was checked by nodir@chromium.org

[commit-bot: I haz the power, 2016-08-05 19:19:14]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-08-05 19:33:19]

Description was changed from

==========
swarmbucket: allow overriding config

swarming.override_builder_cfg can override most of builder configuration
defined on the server. This allows users to specify dimensions
dynamically.

The only config value that cannot be changed is builder name. Builder
names will remain static.

Also change the way we undefine properties: instead of 
properties_j: "foo:"
do
properties_j: "foo:null"

This prevents from having null property values, but it is fine.


BUG=
==========

to

==========
swarmbucket: allow overriding config

swarming.override_builder_cfg can override most of builder configuration
defined on the server. This allows users to specify dimensions
dynamically.

The only config value that cannot be changed is builder name. Builder
names will remain static.

Also change the way we undefine properties: instead of 
properties_j: "foo:"
do
properties_j: "foo:null"

This prevents from having null property values, but it is fine.


BUG=

Committed:
https://chromium.googlesource.com/infra/infra/+/16eca9957490bed920371d6fcad67...
==========

[commit-bot: I haz the power, 2016-08-05 19:33:20]

Committed patchset #3 (id:40001) as
https://chromium.googlesource.com/infra/infra/+/16eca9957490bed920371d6fcad67...

[nodir, 2016-08-06 01:00:26]

On 2016/08/05 19:33:20, commit-bot: I haz the power wrote:
> Committed patchset #3 (id:40001) as
>
https://chromium.googlesource.com/infra/infra/+/16eca9957490bed920371d6fcad67...

borenet, this is deployed

[borenet, 2016-08-08 12:22:33]

On 2016/08/06 01:00:26, nodir wrote:
> On 2016/08/05 19:33:20, commit-bot: I haz the power wrote:
> > Committed patchset #3 (id:40001) as
> >
>
https://chromium.googlesource.com/infra/infra/+/16eca9957490bed920371d6fcad67...
> 
> borenet, this is deployed

Confirmed working, thanks!