Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(512)

Side by Side Diff: appengine/gce-backend/README.md

Issue 2243923002: Refactors GCE backend config updates. (Closed) Base URL: https://chromium.googlesource.com/external/github.com/luci/luci-py@master
Patch Set: . Created 4 years, 4 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « appengine/components/components/config/common_test.py ('k') | appengine/gce-backend/config.py » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # GCE Backend 1 # GCE Backend
2 2
3 A GCE backend for Machine Provider. 3 A GCE backend for Machine Provider.
4 4
5 GCE Backend is composed of a series of cron jobs. Each cron runs independently 5 GCE Backend is composed of a series of cron jobs. Each cron runs independently
6 of the others and performs idempotent operations. Many of the jobs are only 6 of the others and performs idempotent operations. Many of the jobs are only
7 eventually consistent, converging on the desired state over multiple calls. 7 eventually consistent, converging on the desired state over multiple calls.
8 8
9 9
10 ## Setting up 10 ## Setting up
11 11
12 * Visit http://console.cloud.google.com and create a project. Replace 12 * Visit http://console.cloud.google.com and create a project. Replace
13 `<appid>` below with your project id. 13 `<appid>` below with your project id.
14 * Visit Google Cloud Console, IAM & Admin, click Add Member and add someone 14 * Visit Google Cloud Console, IAM & Admin, click Add Member and add someone
15 else so you can safely be hit by a bus. 15 else so you can safely be hit by a bus.
16 * Upload the code with: `./tools/gae upl -x -A <appid>` 16 * Upload the code with: `./tools/gae upl -x -A <appid>`
17 * Enable Pub/Sub and Compute Engine. 17 * Enable Pub/Sub and Compute Engine.
18 * In Pub/Sub, create a topic with the same name as 18 * In Pub/Sub, create a topic with the same name as
19 pubsub.get\_machine\_provider\_topic(), and a pull subscription with the 19 pubsub.get\_machine\_provider\_topic(), and a pull subscription with the
20 same name as pubsub.get\_machine\_provider\_subscription(). On the topic, 20 same name as pubsub.get\_machine\_provider\_subscription(). On the topic,
21 authorize the Machine Provider's default service account as a publisher, 21 authorize the Machine Provider's default service account as a publisher,
22 e.g. machine-provider@appspot.gserviceaccount.com. 22 e.g. machine-provider@appspot.gserviceaccount.com.
23 * Store the following three entities in the datastore: 23 * Use the ConfigApi to set the ConfigSettings datastore entity to the
24 * config.Configuration: Most of this entity will be updated by the app, 24 address of the config service, e.g. luci-config.appspot.com.
25 but the config\_set field needs to be set to the name of the config set 25 * Store the following entity in the datastore:
26 to fetch from the config service.
27 * components.config.common.ConfigSettings: Set service\_hostname to the
28 address of the config service, e.g. luci-config.appspot.com. All others
29 fields are unused.
30 * components.machine\_provider.utils.MachineProviderConfiguration: Set 26 * components.machine\_provider.utils.MachineProviderConfiguration: Set
31 instance\_url to the URL for the Machine Provider service, e.g. 27 instance\_url to the URL for the Machine Provider service, e.g.
32 https://machine-provider.appspot.com. 28 https://machine-provider.appspot.com.
33 * TODO(smut): Simplify the above. 29 * TODO(smut): Simplify the above.
34 30
35 31
36 TODO(smut): Move the following into doc/ as applicable. 32 TODO(smut): Move the following into doc/ as applicable.
37 33
38 34
39 ## Config 35 ## Config
40 36
41 The configuration defines the desired instance templates, and the desired 37 The configuration defines the desired instance templates, and the desired
42 instance group managers created from those templates. Configured in the config 38 instance group managers created from those templates. Configured in the config
43 service as templates.cfg and managers.cfg respectively. 39 service as templates.cfg and managers.cfg respectively.
44 40
45 41
46 ## import-config 42 ## import-config
47 43
48 Responsible for importing the current GCE configuration from the config service. 44 Responsible for importing the current GCE configuration from the config service.
49 When a new config is detected, the local copy of the config in the datastore is 45 When a new config is detected, the local copy of the config in the datastore is
50 updated. 46 updated. This task ensures that the valid manager and template configurations
47 are updated synchronously.
51 48
52 49
53 ## process-config 50 ## process-config
54 51
55 Responsible for creating an entity structure corresponding to the config. 52 Responsible for creating an entity structure corresponding to the config.
56 53
57 A models.InstanceTemplate is created for each template defined in templates.cfg 54 A models.InstanceTemplate is created for each template defined in templates.cfg
58 which has a globally unique base\_name. Any reuse of base\_name, even when the 55 which has a globally unique base\_name. Any reuse of base\_name, even when the
59 configured project is different, is considered to be another revision of the 56 configured project is different, is considered to be another revision of the
60 same instance template. A revision of an instance template is one set of fields 57 same instance template. A revision of an instance template is one set of fields
(...skipping 89 matching lines...) Expand 10 before | Expand all | Expand 10 after
150 147
151 ## delete-instance-templates 148 ## delete-instance-templates
152 149
153 Deletes GCE instance templates that aren't found in the config and have no 150 Deletes GCE instance templates that aren't found in the config and have no
154 instance group managers configured to use them. 151 instance group managers configured to use them.
155 152
156 153
157 ## cleanup-entities 154 ## cleanup-entities
158 155
159 Deletes entities that are no longer needed. 156 Deletes entities that are no longer needed.
OLDNEW
« no previous file with comments | « appengine/components/components/config/common_test.py ('k') | appengine/gce-backend/config.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698