Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(148)

Side by Side Diff: appengine/findit/util_scripts/iterator.py

Issue 2435863003: [Findit] Add local git parsers. (Closed)
Patch Set: Rebase. Created 4 years, 1 month ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
1 # Copyright 2016 The Chromium Authors. All rights reserved. 1 # Copyright 2016 The Chromium Authors. All rights reserved.
2 # Use of this source code is governed by a BSD-style license that can be 2 # Use of this source code is governed by a BSD-style license that can be
3 # found in the LICENSE file. 3 # found in the LICENSE file.
4 4
5 """Fetches entities and iterate over and process them.""" 5 """Fetches entities and iterate over and process them."""
6 6
7 import os 7 import os
8 8
9 import remote_api 9 import remote_api # pylint: disable=W
10 10
11 _DEFAULT_BATCH_SIZE = 1000 11 _DEFAULT_BATCH_SIZE = 1000
12 12
13 13
14 def ProjectEntity(entity, fields): 14 # TODO(crbug.com/662540): Add unittests.
15 def ProjectEntity(entity, fields): # pragma: no cover.
15 """Projects fields from entity. Returns dict.""" 16 """Projects fields from entity. Returns dict."""
16 entity_info = {} 17 entity_info = {}
17 for field in fields: 18 for field in fields:
18 if hasattr(entity, field): 19 if hasattr(entity, field):
19 entity_info[field] = getattr(entity, field) 20 entity_info[field] = getattr(entity, field)
20 else: 21 else:
21 entity_info[field] = None 22 entity_info[field] = None
22 entity_info['id'] = entity.key.id() 23 entity_info['id'] = entity.key.id()
23 return entity_info 24 return entity_info
24 25
25 26
27 # TODO(crbug.com/662540): Add unittests.
26 def Iterate(query, 28 def Iterate(query,
27 fields, 29 fields,
28 app_id, 30 app_id,
29 filter_func=None, 31 filter_func=None,
30 batch_size=_DEFAULT_BATCH_SIZE, 32 batch_size=_DEFAULT_BATCH_SIZE,
31 batch_run=False): 33 batch_run=False): # pragma: no cover.
32 """Iterates entities queried by query. 34 """Iterates entities queried by query.
33 35
34 Args: 36 Args:
35 query (ndb.Query): The query to fetch entities. 37 query (ndb.Query): The query to fetch entities.
36 fields (list): Field names of an entity to be projected to a dict. 38 fields (list): Field names of an entity to be projected to a dict.
37 If a given field name is not available, it is set to None. 39 If a given field name is not available, it is set to None.
38 'id' is always added by default as an integer. 40 'id' is always added by default as an integer.
39 app_id (str): App engine app id. 41 app_id (str): App engine app id.
40 filter_func (function): A function that does in memory filtering. 42 filter_func (function): A function that does in memory filtering.
41 batch_size (int): The number of entities to query at one time. 43 batch_size (int): The number of entities to query at one time.
(...skipping 18 matching lines...) Expand all
60 if batch_run: 62 if batch_run:
61 yield entities 63 yield entities
62 else: 64 else:
63 for entity in entities: 65 for entity in entities:
64 yield entity 66 yield entity
65 67
66 if not more: 68 if not more:
67 break 69 break
68 70
69 cursor = next_cursor 71 cursor = next_cursor
OLDNEW
« no previous file with comments | « appengine/findit/util_scripts/git_checkout/test/local_git_parsers_test.py ('k') | appengine/findit/util_scripts/remote_api.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698