Milo: Split pubsub and the prpc api out to its own modules

Milo: Split pubsub and the prpc api out to its own modules

"make up" still updates everything

BUG=643465
Review-Url: https://codereview.chromium.org/2755273003
Committed: https://github.com/luci/luci-go/commit/f3bc112bcbeb968cf79bcc8766d759bb65538a53

[Ryan Tseng, 2017-03-17 23:44:04]

The CQ bit was checked by hinoka@google.com to run a CQ dry run

[commit-bot: I haz the power, 2017-03-17 23:44:07]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[hinoka, 2017-03-17 23:45:10]

Description was changed from

==========
Milo: Split pubsub out to its own module

"make up" still updates everythign

BUG=643465
==========

to

==========
Milo: Split pubsub out to its own module

"make up" still updates everything

BUG=643465
==========

[hinoka, 2017-03-17 23:45:10]

hinoka@chromium.org changed reviewers:
+ estaab@chromium.org, nodir@chromium.org

[hinoka, 2017-03-17 23:45:33]

hinoka@chromium.org changed reviewers:
+ hinoka@chromium.org

[hinoka, 2017-03-17 23:45:33]

ptal.  This is the simplest way to split it out - Build two of the same thing
and use dispatch.yaml to call them separately

https://codereview.chromium.org/2755273003/diff/1/milo/Makefile
File milo/Makefile (right):

https://codereview.chromium.org/2755273003/diff/1/milo/Makefile#newcode25
milo/Makefile:25: appcfg.py -A luci-milo update_dispatch
$(projdir)/appengine/frontend
Doesn't seem like gae.py does this for you...

https://codereview.chromium.org/2755273003/diff/1/milo/appengine/frontend/cro...
File milo/appengine/frontend/cron.yaml (right):

https://codereview.chromium.org/2755273003/diff/1/milo/appengine/frontend/cro...
milo/appengine/frontend/cron.yaml:8: - description: LUCI Config datastore cache
periodic refresh
This was inadvertently broken earlier...

[commit-bot: I haz the power, 2017-03-17 23:52:17]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-17 23:52:18]

Dry run: This issue passed the CQ dry run.

[Ryan Tseng, 2017-03-18 00:05:22]

The CQ bit was checked by hinoka@google.com to run a CQ dry run

[commit-bot: I haz the power, 2017-03-18 00:05:33]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-03-18 00:14:42]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-18 00:14:43]

Dry run: Try jobs failed on following builders:
  Luci-go Linux Trusty 32-on-64 Tester on luci.infra.try (JOB_FAILED,
https://luci-milo.appspot.com/swarming/task/34f78d81d0499a10)
  Luci-go Linux Trusty 64 Tester on luci.infra.try (JOB_FAILED,
https://luci-milo.appspot.com/swarming/task/34f78d842b624b10)

[Ryan Tseng, 2017-03-18 00:15:21]

The CQ bit was checked by hinoka@google.com to run a CQ dry run

[commit-bot: I haz the power, 2017-03-18 00:15:23]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[hinoka, 2017-03-18 00:15:37]

Description was changed from

==========
Milo: Split pubsub out to its own module

"make up" still updates everything

BUG=643465
==========

to

==========
Milo: Split pubsub and the prpc api out to its own modules

"make up" still updates everything

BUG=643465
==========

[commit-bot: I haz the power, 2017-03-18 00:23:12]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-18 00:23:13]

Dry run: Try jobs failed on following builders:
  Luci-go Linux Trusty 32-on-64 Tester on luci.infra.try (JOB_FAILED,
https://luci-milo.appspot.com/swarming/task/34f796777003dc10)

[nodir, 2017-03-20 22:22:57]

https://codereview.chromium.org/2755273003/diff/1/milo/Makefile
File milo/Makefile (right):

https://codereview.chromium.org/2755273003/diff/1/milo/Makefile#newcode25
milo/Makefile:25: appcfg.py -A luci-milo update_dispatch
$(projdir)/appengine/frontend
On 2017/03/17 23:45:33, hinoka wrote:
> Doesn't seem like gae.py does this for you...

true

https://codereview.chromium.org/2755273003/diff/40001/milo/Makefile
File milo/Makefile (right):

https://codereview.chromium.org/2755273003/diff/40001/milo/Makefile#newcode16
milo/Makefile:16:
$(projdir)/../../../../../../luci/appengine/components/tools/gae.py upload -A
luci-milo --app-dir $(projdir)/appengine default pubsub api
gae.py upload requires valid Go env. infra'git's go env puts gae.py in the PATH,
so I think you can just use `gae.py` instead of this long path

https://codereview.chromium.org/2755273003/diff/40001/milo/appengine/frontend...
File milo/appengine/frontend/module-api.yaml (right):

https://codereview.chromium.org/2755273003/diff/40001/milo/appengine/frontend...
milo/appengine/frontend/module-api.yaml:9: - url: /.*
still require admin for internal URLs

consider adding auth checks in the code too, a la security in depth

https://codereview.chromium.org/2755273003/diff/40001/milo/appengine/frontend...
File milo/appengine/frontend/module-pubsub.yaml (right):

https://codereview.chromium.org/2755273003/diff/40001/milo/appengine/frontend...
milo/appengine/frontend/module-pubsub.yaml:8: # Everything goes here.
here too

[Ryan Tseng, 2017-03-20 23:46:09]

The CQ bit was checked by hinoka@google.com to run a CQ dry run

[commit-bot: I haz the power, 2017-03-20 23:46:19]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Ryan Tseng, 2017-03-20 23:49:19]

The CQ bit was checked by hinoka@google.com to run a CQ dry run

[commit-bot: I haz the power, 2017-03-20 23:49:26]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-03-20 23:55:55]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-20 23:55:55]

Dry run: This issue passed the CQ dry run.

[hinoka, 2017-03-21 00:56:15]

https://codereview.chromium.org/2755273003/diff/40001/milo/Makefile
File milo/Makefile (right):

https://codereview.chromium.org/2755273003/diff/40001/milo/Makefile#newcode16
milo/Makefile:16:
$(projdir)/../../../../../../luci/appengine/components/tools/gae.py upload -A
luci-milo --app-dir $(projdir)/appengine default pubsub api
On 2017/03/20 22:22:57, nodir wrote:
> gae.py upload requires valid Go env. infra'git's go env puts gae.py in the
PATH,
> so I think you can just use `gae.py` instead of this long path

Done.

https://codereview.chromium.org/2755273003/diff/40001/milo/appengine/frontend...
File milo/appengine/frontend/module-api.yaml (right):

https://codereview.chromium.org/2755273003/diff/40001/milo/appengine/frontend...
milo/appengine/frontend/module-api.yaml:9: - url: /.*
On 2017/03/20 22:22:57, nodir wrote:
> still require admin for internal URLs
> 
> consider adding auth checks in the code too, a la security in depth

Done.

Will add admin in a separate CL.

https://codereview.chromium.org/2755273003/diff/40001/milo/appengine/frontend...
File milo/appengine/frontend/module-pubsub.yaml (right):

https://codereview.chromium.org/2755273003/diff/40001/milo/appengine/frontend...
milo/appengine/frontend/module-pubsub.yaml:8: # Everything goes here.
On 2017/03/20 22:22:57, nodir wrote:
> here too

Done.

[nodir, 2017-03-21 01:08:06]

lgtm

[hinoka, 2017-03-21 04:32:17]

The CQ bit was checked by hinoka@chromium.org

[commit-bot: I haz the power, 2017-03-21 04:32:28]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-03-21 04:38:14]

CQ is committing da patch.

Bot data: {"patchset_id": 80001, "attempt_start_ts": 1490070737423840,
"parent_rev": "800889164a7b4f564f76199a387ebc119a6ab088", "commit_rev":
"f3bc112bcbeb968cf79bcc8766d759bb65538a53"}

[commit-bot: I haz the power, 2017-03-21 04:38:18]

Description was changed from

==========
Milo: Split pubsub and the prpc api out to its own modules

"make up" still updates everything

BUG=643465
==========

to

==========
Milo: Split pubsub and the prpc api out to its own modules

"make up" still updates everything

BUG=643465

Review-Url: https://codereview.chromium.org/2755273003
Committed:
https://github.com/luci/luci-go/commit/f3bc112bcbeb968cf79bcc8766d759bb65538a53
==========

[commit-bot: I haz the power, 2017-03-21 04:38:19]

Committed patchset #5 (id:80001) as
https://github.com/luci/luci-go/commit/f3bc112bcbeb968cf79bcc8766d759bb65538a53