Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(571)

Side by Side Diff: slave/skia_slave_scripts/utils/sync_bucket_subdir.py

Issue 311433002: Revert of Adds uploading picture benchmark JSON data (Closed) Base URL: https://skia.googlesource.com/buildbot.git@master
Patch Set: Created 6 years, 6 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « slave/skia_slave_scripts/upload_bench_results.py ('k') | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 #!/usr/bin/env python 1 #!/usr/bin/env python
2 # Copyright (c) 2012 The Chromium Authors. All rights reserved. 2 # Copyright (c) 2012 The Chromium Authors. All rights reserved.
3 # Use of this source code is governed by a BSD-style license that can be 3 # Use of this source code is governed by a BSD-style license that can be
4 # found in the LICENSE file. 4 # found in the LICENSE file.
5 5
6 """Synchronize multiple files with a Google Storage Bucket and subdir. 6 """Synchronize multiple files with a Google Storage Bucket and subdir.
7 For a lack of a better word 'synchronization' was used, warnings: 7 For a lack of a better word 'synchronization' was used, warnings:
8 1) Downloads files that are not found locally 8 1) Downloads files that are not found locally
9 2) Uploads files that are not found in the bucket 9 2) Uploads files that are not found in the bucket
10 3) Does NOT check version, date or content of the file. If the file is reported 10 3) Does NOT check version, date or content of the file. If the file is reported
(...skipping 12 matching lines...) Expand all
23 import re 23 import re
24 24
25 25
26 DEFAULT_PERFDATA_GS_BASE = 'gs://chromium-skia-gm' 26 DEFAULT_PERFDATA_GS_BASE = 'gs://chromium-skia-gm'
27 KNOWN_FILENAMES = r'^bench_([0-9a-fr]*)_data.*' 27 KNOWN_FILENAMES = r'^bench_([0-9a-fr]*)_data.*'
28 IGNORE_UPLOAD_FILENAMES = ('.DS_Store') 28 IGNORE_UPLOAD_FILENAMES = ('.DS_Store')
29 29
30 30
31 def SyncBucketSubdir(directory, dest_gsbase=DEFAULT_PERFDATA_GS_BASE, subdir='', 31 def SyncBucketSubdir(directory, dest_gsbase=DEFAULT_PERFDATA_GS_BASE, subdir='',
32 do_upload=True, do_download=True, filenames_filter=KNOWN_FILENAMES, 32 do_upload=True, do_download=True, filenames_filter=KNOWN_FILENAMES,
33 exclude_json=False,
34 min_download_revision=0): 33 min_download_revision=0):
35 """ synchronizes a local directory with a cloud one 34 """ synchronizes a local directory with a cloud one
36 35
37 dir: directory to synchronize 36 dir: directory to synchronize
38 dest_gsbase: gs:// bucket to synchronize 37 dest_gsbase: gs:// bucket to synchronize
39 subdir: optional subdirectory within the bucket, multiple directory levels 38 subdir: optional subdirectory within the bucket, multiple directory levels
40 are supported, using Unix relative path syntax ("outer/innner") 39 are supported, using Unix relative path syntax ("outer/innner")
41 do_upload: True to perform upload, False otherwise 40 do_upload: True to perform upload, False otherwise
42 do_download: True to perform download, False otherwise 41 do_download: True to perform download, False otherwise
43 filenames_filter: is a regular expression used to match known file names, 42 filenames_filter: is a regular expression used to match known file names,
(...skipping 35 matching lines...) Expand 10 before | Expand all | Expand 10 after
79 78
80 # Uploads only files not present on the cloud storage 79 # Uploads only files not present on the cloud storage
81 if do_upload: 80 if do_upload:
82 to_upload = local_files.difference(cloud_files) 81 to_upload = local_files.difference(cloud_files)
83 for file_name in to_upload: 82 for file_name in to_upload:
84 if file_name not in IGNORE_UPLOAD_FILENAMES: 83 if file_name not in IGNORE_UPLOAD_FILENAMES:
85 match = re.search(filenames_filter, file_name) 84 match = re.search(filenames_filter, file_name)
86 if not match: 85 if not match:
87 # Ignore other files, rather than raising an exception 86 # Ignore other files, rather than raising an exception
88 continue 87 continue
89 # Ignore JSON files if the flag is set.
90 if exclude_json and file_name.endswith('.json'):
91 continue
92 # Ignore force builds without a revision number. 88 # Ignore force builds without a revision number.
93 if match.group(1) != '': 89 if match.group(1) != '':
94 upload_to_bucket.upload_to_bucket(os.path.join(directory, file_name), 90 upload_to_bucket.upload_to_bucket(os.path.join(directory, file_name),
95 dest_gsbase, 91 dest_gsbase,
96 subdir) 92 subdir)
97 return 0 93 return 0
OLDNEW
« no previous file with comments | « slave/skia_slave_scripts/upload_bench_results.py ('k') | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698