Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(200)

Side by Side Diff: dashboard/dashboard/delete_test_data.py

Issue 2162963002: [polymer] Merge of master into polymer10-migration (Closed) Base URL: git@github.com:catapult-project/catapult.git@polymer10-migration
Patch Set: Merge polymer10-migration int polymer10-merge Created 4 years, 5 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
1 # Copyright 2016 The Chromium Authors. All rights reserved. 1 # Copyright 2016 The Chromium Authors. All rights reserved.
2 # Use of this source code is governed by a BSD-style license that can be 2 # Use of this source code is governed by a BSD-style license that can be
3 # found in the LICENSE file. 3 # found in the LICENSE file.
4 4
5 """Task queue task which deletes a TestMetadata, its subtests, and all their 5 """Task queue task which deletes a TestMetadata, its subtests, and all their
6 Rows. 6 Rows.
7 7
8 A delete consists of listing all TestMetadata entities which match the name 8 A delete consists of listing all TestMetadata entities which match the name
9 pattern, and then, for each, recursively deleting all child Row entities and 9 pattern, and then, for each, recursively deleting all child Row entities and
10 then TestMetadatas. 10 then TestMetadatas.
11 11
12 For any delete, there could be hundreds of TestMetadatas and many thousands of 12 For any delete, there could be hundreds of TestMetadatas and many thousands of
13 Rows. Datastore operations often time out after a few hundred deletes(), so this 13 Rows. Datastore operations often time out after a few hundred deletes(), so this
14 task is split up using the task queue. 14 task is split up using the task queue.
15 """ 15 """
16 16
17 import logging
18
17 from google.appengine.api import mail 19 from google.appengine.api import mail
18 from google.appengine.api import taskqueue 20 from google.appengine.api import taskqueue
19 from google.appengine.ext import ndb 21 from google.appengine.ext import ndb
20 22
21 from dashboard import datastore_hooks 23 from dashboard import datastore_hooks
22 from dashboard import list_tests 24 from dashboard import list_tests
23 from dashboard import request_handler 25 from dashboard import request_handler
24 from dashboard import utils 26 from dashboard import utils
25 from dashboard.models import graph_data 27 from dashboard.models import graph_data
26 28
(...skipping 99 matching lines...) Expand 10 before | Expand all | Expand 10 after
126 task_params = { 128 task_params = {
127 'test_key': test_key_urlsafe, 129 'test_key': test_key_urlsafe,
128 } 130 }
129 taskqueue.add( 131 taskqueue.add(
130 url='/delete_test_data', 132 url='/delete_test_data',
131 params=task_params, 133 params=task_params,
132 queue_name=_TASK_QUEUE_NAME) 134 queue_name=_TASK_QUEUE_NAME)
133 135
134 136
135 def _DeleteTestData(test_key): 137 def _DeleteTestData(test_key):
138 logging.info('DELETING TEST DATA FOR %s', utils.TestPath(test_key))
136 futures = [] 139 futures = []
137 num_tests_processed = 0 140 num_tests_processed = 0
138 finished = True 141 finished = True
139 descendants = list_tests.GetTestDescendants(test_key) 142 descendants = list_tests.GetTestDescendants(test_key)
140 for descendant in descendants: 143 for descendant in descendants:
141 rows = graph_data.GetLatestRowsForTest( 144 rows = graph_data.GetLatestRowsForTest(
142 descendant, _ROWS_TO_DELETE_AT_ONCE, keys_only=True) 145 descendant, _ROWS_TO_DELETE_AT_ONCE, keys_only=True)
143 if rows: 146 if rows:
144 futures.extend(ndb.delete_multi_async(rows)) 147 futures.extend(ndb.delete_multi_async(rows))
145 finished = False 148 finished = False
(...skipping 22 matching lines...) Expand all
168 if not test or not test.sheriff: 171 if not test or not test.sheriff:
169 return 172 return
170 body = _SHERIFF_ALERT_EMAIL_BODY % { 173 body = _SHERIFF_ALERT_EMAIL_BODY % {
171 'test_path': utils.TestPath(test.key), 174 'test_path': utils.TestPath(test.key),
172 'sheriff': test.sheriff.string_id(), 175 'sheriff': test.sheriff.string_id(),
173 } 176 }
174 mail.send_mail(sender='gasper-alerts@google.com', 177 mail.send_mail(sender='gasper-alerts@google.com',
175 to='chrome-performance-monitoring-alerts@google.com', 178 to='chrome-performance-monitoring-alerts@google.com',
176 subject='Sheriffed Test Deleted', 179 subject='Sheriffed Test Deleted',
177 body=body) 180 body=body)
OLDNEW
« no previous file with comments | « dashboard/dashboard/delete_old_tests.py ('k') | dashboard/dashboard/elements/primary-button.html » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698