Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(84)

Issue 1240573002: Reland: Refactor current GAE abstraction library to be free of the SDK* (Closed)

Created:
5 years, 5 months ago by iannucci
Modified:
5 years, 5 months ago
Reviewers:
dnj, Vadim Sh., M-A Ruel
CC:
chromium-reviews
Base URL:
https://chromium.googlesource.com/infra/infra.git@master
Target Ref:
refs/heads/master
Project:
infra
Visibility:
Public.

Description

Reland: Refactor current GAE abstraction library to be free of the SDK* Also improves the struct serialization implementation over the SDK's: * 'non-struct' representation is now map[string][]values, instead of []PropertyList. This makes working with PropertyLoadSaver much easier. * Fields can implement a DSPropertyConverter interface to transparently be serialized to/from some other datastore-compatible type. This makes many uses of PropertyLoadSaver unnecessary. * Lots of reflection code moved out of structPLS.{Load,Save} into the creation of the struct codec (e.g. it happens once instead of on every load/save now). * Struct codecs are directly accessible by users, allowing easier implementation of PropertyLoadSaver interface. * Struct codecs support 'special' tagged fields. This will be used for implementing an embedded-key Datastore interface without adding any extra reflection code. * DSProperty's now have strictly private fields. This ensures that every DSProperty instance contains only valid types for Value (since it's not possible to construct one with an incompatible Value now), which simplifies error checking code. * DSProperty does automatic upconversion of close-but-not-exact types (like MyString -> string, and int -> int64). This upconversion was previously only available for struct serialization. * DSProperty's now retain a sortable 'Type' field to avoid unnecessary additional reflection when serializing them (such as to memcache). * There is now a consistent, sortable binary serialization format for DSPropertyMap's, which will be used for the follow-on transparent memcache implementation. It's also used internally by the in-memory testing implementation of the datastore. *I alias some error types from the new-SDK, but users of the abstraction no longer need to import the SDK at all. Original broke because endpoints needed to be moved to a version which relies on the new SDK. R=maruel@chromium.org, vadimsh@chromium.org BUG= Committed: https://chromium.googlesource.com/infra/infra/+/9d34bb03c2adec2dec0369712b2011b6659e4e54

Patch Set 1 : Original Patch #

Patch Set 2 : fix build #

Patch Set 3 : expand coverage range to fit 32bit test expectations #

Unified diffs Side-by-side diffs Delta from patch set Stats (+10577 lines, -8488 lines) Patch
M go/.gitignore View 1 chunk +4 lines, -4 lines 0 comments Download
M go/Goopfile View 1 1 chunk +1 line, -1 line 0 comments Download
M go/Goopfile.lock View 1 1 chunk +1 line, -1 line 0 comments Download
M go/src/infra/gae/epservice/example/model.go View 1 1 chunk +0 lines, -2 lines 0 comments Download
M go/src/infra/gae/epservice/example/service_add.go View 1 2 chunks +10 lines, -9 lines 0 comments Download
M go/src/infra/gae/epservice/example/service_cas.go View 1 2 chunks +10 lines, -8 lines 0 comments Download
M go/src/infra/gae/epservice/example/service_currentvalue.go View 1 2 chunks +8 lines, -6 lines 0 comments Download
M go/src/infra/gae/epservice/example/service_list.go View 1 2 chunks +5 lines, -5 lines 0 comments Download
A + go/src/infra/gae/libs/gae/brokenfeatures.go View 6 chunks +26 lines, -22 lines 0 comments Download
A + go/src/infra/gae/libs/gae/brokenfeatures_test.go View 4 chunks +21 lines, -19 lines 0 comments Download
A + go/src/infra/gae/libs/gae/context.go View 1 chunk +6 lines, -5 lines 0 comments Download
A go/src/infra/gae/libs/gae/doc.go View 1 chunk +62 lines, -0 lines 0 comments Download
A + go/src/infra/gae/libs/gae/dummy.go View 8 chunks +53 lines, -63 lines 0 comments Download
A + go/src/infra/gae/libs/gae/dummy_test.go View 4 chunks +28 lines, -19 lines 0 comments Download
A go/src/infra/gae/libs/gae/gae.infra_testing View 1 chunk +4 lines, -0 lines 0 comments Download
A + go/src/infra/gae/libs/gae/globalinfo.go View 1 chunk +10 lines, -30 lines 0 comments Download
A go/src/infra/gae/libs/gae/helper/datastore.go View 1 chunk +77 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/helper/datastore_impl.go View 1 chunk +509 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/helper/datastore_key.go View 1 chunk +236 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/helper/datastore_key_test.go View 1 chunk +241 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/helper/datastore_test.go View 1 chunk +1688 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/helper/generic_key.go View 1 chunk +84 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/helper/helper.infra_testing View 1 chunk +4 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/helper/internal/protos/datastore/README.md View 1 chunk +5 lines, -0 lines 0 comments Download
A + go/src/infra/gae/libs/gae/helper/internal/protos/datastore/datastore.infra_testing View 1 chunk +1 line, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/helper/internal/protos/datastore/datastore_v3.proto View 1 chunk +541 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/helper/internal/protos/datastore/datastore_v3.pb.go View 1 chunk +2776 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/helper/serialize.go View 1 chunk +335 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/helper/serialize_test.go View 1 chunk +329 lines, -0 lines 0 comments Download
A + go/src/infra/gae/libs/gae/mathrand.go View 1 chunk +3 lines, -3 lines 0 comments Download
A go/src/infra/gae/libs/gae/memcache.go View 1 chunk +79 lines, -0 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/README.md View 0 chunks +-1 lines, --1 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/context.go View 5 chunks +5 lines, -5 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/datastore_query.go View 21 chunks +50 lines, -47 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/doc.go View 0 chunks +-1 lines, --1 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/gkvlite_utils.go View 0 chunks +-1 lines, --1 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/gkvlite_utils_test.go View 0 chunks +-1 lines, --1 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/globalinfo.go View 2 chunks +18 lines, -8 lines 0 comments Download
A go/src/infra/gae/libs/gae/memory/memcache.go View 1 chunk +228 lines, -0 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/memcache_test.go View 6 chunks +63 lines, -65 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/memory.infra_testing View 1 2 1 chunk +1 line, -1 line 0 comments Download
A go/src/infra/gae/libs/gae/memory/plist.go View 1 chunk +298 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/memory/plist_test.go View 1 chunk +392 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/memory/raw_datstore.go View 1 chunk +155 lines, -0 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/raw_datstore_data.go View 17 chunks +135 lines, -137 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/raw_datstore_test.go View 13 chunks +161 lines, -252 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/taskqueue.go View 14 chunks +80 lines, -97 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/taskqueue_data.go View 14 chunks +46 lines, -41 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/taskqueue_test.go View 20 chunks +66 lines, -67 lines 0 comments Download
A + go/src/infra/gae/libs/gae/memory/testing_utils_test.go View 3 chunks +34 lines, -41 lines 0 comments Download
A go/src/infra/gae/libs/gae/prod/context.go View 1 chunk +23 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/prod/datastore_key.go View 1 chunk +82 lines, -0 lines 0 comments Download
A + go/src/infra/gae/libs/gae/prod/doc.go View 1 chunk +2 lines, -2 lines 0 comments Download
A + go/src/infra/gae/libs/gae/prod/globalinfo.go View 4 chunks +19 lines, -21 lines 0 comments Download
A go/src/infra/gae/libs/gae/prod/memcache.go View 1 chunk +156 lines, -0 lines 0 comments Download
A + go/src/infra/gae/libs/gae/prod/prod.infra_testing View 0 chunks +-1 lines, --1 lines 0 comments Download
A go/src/infra/gae/libs/gae/prod/raw_datastore.go View 1 chunk +156 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/prod/raw_datastore_type_converter.go View 1 chunk +85 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/prod/taskqueue.go View 1 chunk +150 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/properties.go View 1 chunk +356 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/properties_test.go View 1 chunk +165 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/raw_datastore.go View 1 chunk +116 lines, -0 lines 0 comments Download
A + go/src/infra/gae/libs/gae/taskqueue.go View 4 chunks +10 lines, -11 lines 0 comments Download
A + go/src/infra/gae/libs/gae/taskqueue_testable.go View 1 chunk +5 lines, -9 lines 0 comments Download
A + go/src/infra/gae/libs/gae/testable.go View 1 chunk +1 line, -1 line 0 comments Download
A go/src/infra/gae/libs/gae/upstream_errors.go View 1 chunk +109 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/upstream_errors_test.go View 1 chunk +60 lines, -0 lines 0 comments Download
A go/src/infra/gae/libs/gae/upstream_types.go View 1 chunk +161 lines, -0 lines 0 comments Download
M go/src/infra/gae/libs/memlock/memlock.go View 4 chunks +13 lines, -16 lines 0 comments Download
M go/src/infra/gae/libs/memlock/memlock_test.go View 3 chunks +9 lines, -10 lines 0 comments Download
M go/src/infra/gae/libs/meta/eg.go View 2 chunks +7 lines, -13 lines 0 comments Download
M go/src/infra/gae/libs/meta/eg_test.go View 3 chunks +8 lines, -8 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/brokenfeatures.go View 1 chunk +0 lines, -157 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/brokenfeatures_test.go View 1 chunk +0 lines, -126 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/context.go View 1 chunk +0 lines, -18 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/datastore.go View 1 chunk +0 lines, -139 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/doc.go View 1 chunk +0 lines, -40 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/dummy.go View 1 chunk +0 lines, -185 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/dummy_test.go View 1 chunk +0 lines, -76 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/gae/commonErrors/commonErrors.infra_testing View 1 chunk +0 lines, -3 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/gae/commonErrors/errors.go View 1 chunk +0 lines, -42 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/gae/context.go View 1 chunk +0 lines, -43 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/gae/datastore.go View 1 chunk +0 lines, -118 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/gae/doc.go View 1 chunk +0 lines, -7 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/gae/gae.infra_testing View 1 chunk +0 lines, -3 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/gae/globalinfo.go View 1 chunk +0 lines, -87 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/gae/memcache.go View 1 chunk +0 lines, -111 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/gae/taskqueue.go View 1 chunk +0 lines, -68 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/globalinfo.go View 1 chunk +0 lines, -80 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/mathrand.go View 1 chunk +0 lines, -42 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memcache.go View 1 chunk +0 lines, -106 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/README.md View 1 chunk +0 lines, -114 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/binutils.go View 1 chunk +0 lines, -98 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/binutils_test.go View 1 chunk +0 lines, -54 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/context.go View 1 chunk +0 lines, -168 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/datastore.go View 1 chunk +0 lines, -206 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/datastore_data.go View 1 chunk +0 lines, -477 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/datastore_query.go View 1 chunk +0 lines, -567 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/datastore_test.go View 1 chunk +0 lines, -682 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/doc.go View 1 chunk +0 lines, -7 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/gkvlite_utils.go View 1 chunk +0 lines, -156 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/gkvlite_utils_test.go View 1 chunk +0 lines, -128 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/globalinfo.go View 1 chunk +0 lines, -41 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/internal/goon/goon.go View 1 chunk +0 lines, -104 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/internal/goon/goon.infra_testing View 1 chunk +0 lines, -3 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/key.go View 1 chunk +0 lines, -236 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/key_test.go View 1 chunk +0 lines, -51 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/memcache.go View 1 chunk +0 lines, -196 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/memcache_test.go View 1 chunk +0 lines, -215 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/memory.infra_testing View 1 chunk +0 lines, -4 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/plist.go View 1 chunk +0 lines, -637 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/plist_test.go View 1 chunk +0 lines, -481 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/taskqueue.go View 1 chunk +0 lines, -301 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/taskqueue_data.go View 1 chunk +0 lines, -265 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/taskqueue_test.go View 1 chunk +0 lines, -467 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/memory/testing_utils_test.go View 1 chunk +0 lines, -122 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/taskqueue.go View 1 chunk +0 lines, -79 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/taskqueue_testable.go View 1 chunk +0 lines, -26 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/testable.go View 1 chunk +0 lines, -10 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/unsafe/doc.go View 1 chunk +0 lines, -8 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/unsafe/memcache.go View 1 chunk +0 lines, -58 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/unsafe/memcache_test.go View 1 chunk +0 lines, -24 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/unsafe/unsafe.infra_testing View 1 chunk +0 lines, -4 lines 0 comments Download
D go/src/infra/gae/libs/wrapper/wrapper.infra_testing View 1 chunk +0 lines, -4 lines 0 comments Download

Messages

Total messages: 13 (5 generated)
iannucci
5 years, 5 months ago (2015-07-14 04:54:01 UTC) #1
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1240573002/20001
5 years, 5 months ago (2015-07-14 16:33:39 UTC) #3
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1240573002/40001
5 years, 5 months ago (2015-07-14 16:38:55 UTC) #5
iannucci
ptal, ps#1 is the original which g'splod'd the other day. I think it should be ...
5 years, 5 months ago (2015-07-14 16:39:26 UTC) #7
commit-bot: I haz the power
Dry run: This issue passed the CQ dry run.
5 years, 5 months ago (2015-07-14 16:41:58 UTC) #9
dnj
rs-lgtm hoping for less g-splosion.
5 years, 5 months ago (2015-07-14 16:47:41 UTC) #10
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1240573002/40001
5 years, 5 months ago (2015-07-14 16:48:55 UTC) #12
commit-bot: I haz the power
5 years, 5 months ago (2015-07-14 16:52:22 UTC) #13
Message was sent while issue was closed.
Committed patchset #3 (id:40001) as
https://chromium.googlesource.com/infra/infra/+/9d34bb03c2adec2dec0369712b201...

Powered by Google App Engine
This is Rietveld 408576698