Implement config service cache on top of datastore

Implement config service cache on top of datastore

Implement "server/config" backend on top of appengine/datastorecache.
Integrate it into "gaeconfig" via setting.

BUG=chromium:674378
TEST=None

Review-Url: https://codereview.chromium.org/2576923003
Committed: https://github.com/luci/luci-go/commit/a6a07ceefd2ad7f50c32250df5be42d3972f3ab8

[dnj, 2016-12-15 05:11:54]

dnj@chromium.org changed reviewers:
+ iannucci@chromium.org

[dnj, 2016-12-15 05:11:54]

PTAL. Full stack is:
https://codereview.chromium.org/2580713002 (base)
https://codereview.chromium.org/2578893002 (textproto)
https://codereview.chromium.org/2576993002 (access)
https://codereview.chromium.org/2576343002 (erroring)
https://codereview.chromium.org/2582433002 (test)
https://codereview.chromium.org/2573403002 (caching)
https://codereview.chromium.org/2575383002 (gaeconfig)
https://codereview.chromium.org/2576923003 <-- this

And then win!

[iannucci, 2017-01-07 21:18:37]

lgtm

https://codereview.chromium.org/2576923003/diff/1/appengine/gaeconfig/ds.go
File appengine/gaeconfig/ds.go (right):

https://codereview.chromium.org/2576923003/diff/1/appengine/gaeconfig/ds.go#n...
appengine/gaeconfig/ds.go:80: // layer. For cron refresh, his will be the
generic Loader provided by
this

https://codereview.chromium.org/2576923003/diff/1/appengine/gaeconfig/ds.go#n...
appengine/gaeconfig/ds.go:106: // denied without needing to actually on the
input parameters alone.
I think something went missing in this sentance. or extra?

https://codereview.chromium.org/2576923003/diff/1/appengine/gaeconfig/ds.go#n...
appengine/gaeconfig/ds.go:129: v, err := cache.Get(dc.withHandler(c, l, 0),
encKey)
the datastorecache handler will implicitly do a probabalistic datastore write
into the entry to keep it alive with the cron backend, right?

[dnj, 2017-01-10 03:32:40]

PTAL, updated!

https://codereview.chromium.org/2576923003/diff/1/appengine/gaeconfig/ds.go
File appengine/gaeconfig/ds.go (right):

https://codereview.chromium.org/2576923003/diff/1/appengine/gaeconfig/ds.go#n...
appengine/gaeconfig/ds.go:80: // layer. For cron refresh, his will be the
generic Loader provided by
On 2017/01/07 21:18:36, iannucci wrote:
> this

his will be Done.

https://codereview.chromium.org/2576923003/diff/1/appengine/gaeconfig/ds.go#n...
appengine/gaeconfig/ds.go:106: // denied without needing to actually on the
input parameters alone.
On 2017/01/07 21:18:36, iannucci wrote:
> I think something went missing in this sentance. or extra?

I cleaned this up, breaking into two stages: one to do pre-access check, one to
tweak request.

https://codereview.chromium.org/2576923003/diff/1/appengine/gaeconfig/ds.go#n...
appengine/gaeconfig/ds.go:129: v, err := cache.Get(dc.withHandler(c, l, 0),
encKey)
On 2017/01/07 21:18:36, iannucci wrote:
> the datastorecache handler will implicitly do a probabalistic datastore write
> into the entry to keep it alive with the cron backend, right?

Yes. If the entry is missing, it will be added, and if it's not missing, it will
probabilistically update its "last accessed" time to reflect that it shouldn't
be pruned.

[iannucci, 2017-01-11 03:02:39]

lgtm

[dnj, 2017-01-11 23:22:28]

The CQ bit was checked by dnj@chromium.org

[commit-bot: I haz the power, 2017-01-11 23:22:35]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-01-11 23:30:28]

CQ is committing da patch.

Bot data: {"patchset_id": 20001, "attempt_start_ts": 1484176948791250,
"parent_rev": "45e9a86d32dabca5671d0c1773e124dbc2ec8b5c", "commit_rev":
"a6a07ceefd2ad7f50c32250df5be42d3972f3ab8"}

[commit-bot: I haz the power, 2017-01-11 23:30:31]

Description was changed from

==========
Implement config service cache on top of datastore

Implement "server/config" backend on top of appengine/datastorecache.
Integrate it into "gaeconfig" via setting.

BUG=chromium:674378
TEST=None
==========

to

==========
Implement config service cache on top of datastore

Implement "server/config" backend on top of appengine/datastorecache.
Integrate it into "gaeconfig" via setting.

BUG=chromium:674378
TEST=None

Review-Url: https://codereview.chromium.org/2576923003
Committed:
https://github.com/luci/luci-go/commit/a6a07ceefd2ad7f50c32250df5be42d3972f3ab8
==========

[commit-bot: I haz the power, 2017-01-11 23:30:32]

Committed patchset #2 (id:20001) as
https://github.com/luci/luci-go/commit/a6a07ceefd2ad7f50c32250df5be42d3972f3ab8