Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(113)

Issue 813443002: Overhaul database package (Closed)

Created:
6 years ago by borenet
Modified:
5 years, 11 months ago
CC:
reviews_skia.org, tfarina
Base URL:
https://skia.googlesource.com/buildbot@master
Target Ref:
refs/heads/master
Visibility:
Public.

Description

Overhaul database package - Remove SQLite - Centralize/unify database usage, add lots of config helper functions - Add setup script - Add buildbot_migratedb BUG=skia: Committed: https://skia.googlesource.com/buildbot/+/bee4e6a84182c691f7ea2c984d54d2ece0230741

Patch Set 1 #

Total comments: 2

Patch Set 2 : Add script for setting up test db #

Total comments: 3

Patch Set 3 : Add README for database package, re-add *.db to .gitignore #

Total comments: 2

Patch Set 4 : Move README #

Total comments: 3

Patch Set 5 : Add MySQLTestDatabase for shared test usage #

Total comments: 2

Patch Set 6 : Add buildbot_migratedb #

Total comments: 2

Patch Set 7 : Allow caller to specify the database to use; only use sk_testing in tests #

Patch Set 8 : Actually use the requested database, fix buildbot db tests #

Patch Set 9 : More reorganization #

Patch Set 10 : Comments/readme cleanup #

Total comments: 4

Patch Set 11 : Unskip the CT GS tests #

Total comments: 2

Patch Set 12 : Unify migratedb, add --local flag #

Patch Set 13 : Add db flags #

Patch Set 14 : Override DB host when --local is set #

Patch Set 15 : Fix localhost override #

Patch Set 16 : rebase #

Patch Set 17 : Fix newline in password #

Total comments: 12

Patch Set 18 : Address comments #

Patch Set 19 : Rebase #

Total comments: 2

Patch Set 20 : Assume the MySQL password is "" when running locally #

Unified diffs Side-by-side diffs Delta from patch set Stats (+433 lines, -536 lines) Patch
A README.md View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 chunk +24 lines, -0 lines 0 comments Download
M datahopper/Makefile View 1 2 3 4 5 1 chunk +3 lines, -0 lines 0 comments Download
A datahopper/go/buildbot_migratedb/main.go View 1 2 3 4 5 6 7 8 9 10 11 12 1 chunk +52 lines, -0 lines 0 comments Download
M datahopper/go/datahopper/main.go View 1 2 3 4 5 6 7 8 9 10 11 12 2 chunks +7 lines, -9 lines 0 comments Download
M go/buildbot/buildbot_test.go View 1 2 3 4 5 6 7 8 9 10 11 12 11 chunks +44 lines, -68 lines 0 comments Download
M go/buildbot/db.go View 1 2 3 4 5 6 3 chunks +16 lines, -3 lines 0 comments Download
M go/buildbot/db_setup.go View 1 2 3 4 5 6 7 8 9 10 11 12 3 chunks +13 lines, -79 lines 0 comments Download
M go/database/database.go View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 5 chunks +123 lines, -47 lines 0 comments Download
A go/database/setup_test_db View 1 2 3 4 5 6 7 8 1 chunk +4 lines, -0 lines 0 comments Download
A go/database/setup_test_db.sql View 1 2 3 4 5 6 7 8 9 10 11 12 1 chunk +16 lines, -0 lines 0 comments Download
M go/database/testutil/testutil.go View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 6 chunks +65 lines, -47 lines 0 comments Download
M golden/go/correctness_migratedb/main.go View 1 2 3 4 5 6 7 8 9 10 11 12 2 chunks +9 lines, -27 lines 0 comments Download
M golden/go/db/db.go View 1 2 3 4 5 6 7 8 9 10 11 12 4 chunks +5 lines, -84 lines 0 comments Download
M golden/go/db/db_test.go View 1 chunk +0 lines, -4 lines 0 comments Download
M golden/go/expstorage/expstorage_test.go View 1 2 3 4 5 6 7 8 9 10 11 12 1 chunk +5 lines, -34 lines 0 comments Download
M golden/go/skiacorrectness/main.go View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 4 chunks +8 lines, -6 lines 0 comments Download
M perf/DESIGN.md View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 chunks +5 lines, -10 lines 0 comments Download
M perf/go/db/db.go View 1 2 3 4 5 6 7 8 9 10 11 12 5 chunks +5 lines, -84 lines 0 comments Download
M perf/go/db/db_test.go View 1 chunk +0 lines, -4 lines 0 comments Download
M perf/go/ingest/main.go View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 2 chunks +9 lines, -2 lines 0 comments Download
M perf/go/perf_migratedb/main.go View 1 2 3 4 5 6 7 8 9 10 11 12 2 chunks +11 lines, -27 lines 0 comments Download
M perf/go/skiaperf/main.go View 1 2 3 4 5 6 7 8 9 10 11 12 2 chunks +9 lines, -1 line 0 comments Download

Messages

Total messages: 34 (3 generated)
borenet
https://codereview.chromium.org/813443002/diff/1/golden/go/db/db.go File golden/go/db/db.go (right): https://codereview.chromium.org/813443002/diff/1/golden/go/db/db.go#newcode23 golden/go/db/db.go:23: DB_CONN_TMPL = "%s:%s@tcp(%s:%s)/%s?parseTime=true" I'd like to further unify this ...
6 years ago (2014-12-16 18:42:05 UTC) #2
jcgregorio
On 2014/12/16 18:42:05, borenet wrote: > https://codereview.chromium.org/813443002/diff/1/golden/go/db/db.go > File golden/go/db/db.go (right): > > https://codereview.chromium.org/813443002/diff/1/golden/go/db/db.go#newcode23 > ...
6 years ago (2014-12-16 20:08:39 UTC) #3
jcgregorio
https://codereview.chromium.org/813443002/diff/1/golden/go/db/db.go File golden/go/db/db.go (right): https://codereview.chromium.org/813443002/diff/1/golden/go/db/db.go#newcode23 golden/go/db/db.go:23: DB_CONN_TMPL = "%s:%s@tcp(%s:%s)/%s?parseTime=true" On 2014/12/16 18:42:05, borenet wrote: > ...
6 years ago (2014-12-16 20:09:24 UTC) #4
borenet
Added script for db setup. https://codereview.chromium.org/813443002/diff/20001/go/database/setup_test_db File go/database/setup_test_db (right): https://codereview.chromium.org/813443002/diff/20001/go/database/setup_test_db#newcode1 go/database/setup_test_db:1: #!/bin/bash Should this run ...
6 years ago (2014-12-16 20:24:18 UTC) #5
stephana
https://codereview.chromium.org/813443002/diff/20001/go/database/setup_test_db File go/database/setup_test_db (right): https://codereview.chromium.org/813443002/diff/20001/go/database/setup_test_db#newcode1 go/database/setup_test_db:1: #!/bin/bash On 2014/12/16 20:24:18, borenet wrote: > Should this ...
6 years ago (2014-12-17 13:13:38 UTC) #6
borenet
https://codereview.chromium.org/813443002/diff/20001/go/database/setup_test_db File go/database/setup_test_db (right): https://codereview.chromium.org/813443002/diff/20001/go/database/setup_test_db#newcode1 go/database/setup_test_db:1: #!/bin/bash On 2014/12/17 13:13:38, stephana wrote: > On 2014/12/16 ...
6 years ago (2014-12-17 14:55:14 UTC) #7
stephana
Agreed including a reference into an error message is not work the effort. https://codereview.chromium.org/813443002/diff/40001/go/database/README File ...
6 years ago (2014-12-17 15:28:49 UTC) #8
borenet
https://codereview.chromium.org/813443002/diff/40001/go/database/README File go/database/README (right): https://codereview.chromium.org/813443002/diff/40001/go/database/README#newcode6 go/database/README:6: convenience in setting up this test database and user. ...
6 years ago (2014-12-17 15:39:17 UTC) #9
tfarina
https://codereview.chromium.org/813443002/diff/60001/go/database/setup_test_db.sql File go/database/setup_test_db.sql (right): https://codereview.chromium.org/813443002/diff/60001/go/database/setup_test_db.sql#newcode3 go/database/setup_test_db.sql:3: GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP ON sk_testing.* TO 'test_user'@'localhost'; where is the ...
6 years ago (2014-12-17 18:49:31 UTC) #10
borenet
Okay, I think I have the tests fixed at patch set 5. https://codereview.chromium.org/813443002/diff/60001/go/database/setup_test_db.sql File go/database/setup_test_db.sql ...
6 years ago (2014-12-17 21:02:21 UTC) #11
borenet
https://codereview.chromium.org/813443002/diff/80001/go/buildbot/db_setup.go File go/buildbot/db_setup.go (right): https://codereview.chromium.org/813443002/diff/80001/go/buildbot/db_setup.go#newcode37 go/buildbot/db_setup.go:37: db, err := sqlx.Open("mysql", conf.MySQLString) On 2014/12/17 21:02:21, borenet ...
6 years ago (2014-12-18 14:03:44 UTC) #12
jcgregorio
https://codereview.chromium.org/813443002/diff/100001/go/database/setup_test_db File go/database/setup_test_db (right): https://codereview.chromium.org/813443002/diff/100001/go/database/setup_test_db#newcode1 go/database/setup_test_db:1: #!/bin/bash File wasn't executable when I patched this CL ...
6 years ago (2014-12-18 15:21:13 UTC) #13
borenet
Okay, ready for another round of review at patch set 10. https://codereview.chromium.org/813443002/diff/180001/ct/go/util/gs_test.go File ct/go/util/gs_test.go (right): ...
6 years ago (2014-12-18 19:12:43 UTC) #14
rmistry
https://codereview.chromium.org/813443002/diff/180001/ct/go/util/gs_test.go File ct/go/util/gs_test.go (right): https://codereview.chromium.org/813443002/diff/180001/ct/go/util/gs_test.go#newcode16 ct/go/util/gs_test.go:16: t.Skip("WARNING: Skipping this test due to https://code.google.com/p/skia/issues/detail?id=3261") On 2014/12/18 ...
6 years ago (2014-12-18 19:18:13 UTC) #16
borenet
https://codereview.chromium.org/813443002/diff/180001/ct/go/util/gs_test.go File ct/go/util/gs_test.go (right): https://codereview.chromium.org/813443002/diff/180001/ct/go/util/gs_test.go#newcode16 ct/go/util/gs_test.go:16: t.Skip("WARNING: Skipping this test due to https://code.google.com/p/skia/issues/detail?id=3261") On 2014/12/18 ...
6 years ago (2014-12-18 19:19:23 UTC) #17
rmistry
On 2014/12/18 19:19:23, borenet wrote: > https://codereview.chromium.org/813443002/diff/180001/ct/go/util/gs_test.go > File ct/go/util/gs_test.go (right): > > https://codereview.chromium.org/813443002/diff/180001/ct/go/util/gs_test.go#newcode16 > ...
6 years ago (2014-12-18 19:54:38 UTC) #18
borenet
On 2014/12/18 19:54:38, rmistry wrote: > On 2014/12/18 19:19:23, borenet wrote: > > https://codereview.chromium.org/813443002/diff/180001/ct/go/util/gs_test.go > ...
6 years ago (2014-12-18 19:56:56 UTC) #19
rmistry
On 2014/12/18 19:56:56, borenet wrote: > On 2014/12/18 19:54:38, rmistry wrote: > > On 2014/12/18 ...
6 years ago (2014-12-18 19:58:19 UTC) #20
borenet
On 2014/12/18 19:58:19, rmistry wrote: > On 2014/12/18 19:56:56, borenet wrote: > > On 2014/12/18 ...
6 years ago (2014-12-18 20:01:08 UTC) #21
rmistry
On 2014/12/18 20:01:08, borenet wrote: > On 2014/12/18 19:58:19, rmistry wrote: > > On 2014/12/18 ...
6 years ago (2014-12-18 20:04:46 UTC) #22
borenet
Enhancements to migratedb. PTAL
6 years ago (2014-12-18 22:10:46 UTC) #23
borenet
Added host/port/user/database name flags to database package. Prompt for password or get it from metadata ...
6 years ago (2014-12-19 17:17:34 UTC) #24
jcgregorio
https://codereview.chromium.org/813443002/diff/200001/README.md File README.md (right): https://codereview.chromium.org/813443002/diff/200001/README.md#newcode12 README.md:12: $ ./setup_test_db Requires a local installation of MySQL. For ...
6 years ago (2014-12-19 20:27:06 UTC) #25
borenet
Uploaded patch set 18 https://codereview.chromium.org/813443002/diff/200001/README.md File README.md (right): https://codereview.chromium.org/813443002/diff/200001/README.md#newcode12 README.md:12: $ ./setup_test_db On 2014/12/19 20:27:05, ...
6 years ago (2014-12-19 20:39:15 UTC) #26
borenet
Friendly post-vacation ping.
5 years, 11 months ago (2015-01-05 16:32:03 UTC) #27
jcgregorio
https://codereview.chromium.org/813443002/diff/360001/go/database/database.go File go/database/database.go (right): https://codereview.chromium.org/813443002/diff/360001/go/database/database.go#newcode77 go/database/database.go:77: fmt.Printf("Enter password for MySQL user %s at %s:%d: ", ...
5 years, 11 months ago (2015-01-05 18:33:18 UTC) #28
borenet
https://codereview.chromium.org/813443002/diff/360001/go/database/database.go File go/database/database.go (right): https://codereview.chromium.org/813443002/diff/360001/go/database/database.go#newcode77 go/database/database.go:77: fmt.Printf("Enter password for MySQL user %s at %s:%d: ", ...
5 years, 11 months ago (2015-01-05 19:03:20 UTC) #29
borenet
On 2015/01/05 19:03:20, borenet wrote: > https://codereview.chromium.org/813443002/diff/360001/go/database/database.go > File go/database/database.go (right): > > https://codereview.chromium.org/813443002/diff/360001/go/database/database.go#newcode77 > ...
5 years, 11 months ago (2015-01-05 19:04:17 UTC) #30
jcgregorio
On 2015/01/05 19:04:17, borenet wrote: > On 2015/01/05 19:03:20, borenet wrote: > > https://codereview.chromium.org/813443002/diff/360001/go/database/database.go > ...
5 years, 11 months ago (2015-01-05 20:48:06 UTC) #31
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/813443002/380001
5 years, 11 months ago (2015-01-05 20:59:03 UTC) #33
commit-bot: I haz the power
5 years, 11 months ago (2015-01-05 20:59:16 UTC) #34
Message was sent while issue was closed.
Committed patchset #20 (id:380001) as
https://skia.googlesource.com/buildbot/+/bee4e6a84182c691f7ea2c984d54d2ece023...

Powered by Google App Engine
This is Rietveld 408576698