tools/android/loading Switch the GCE worker to pull queues

tools/android/loading/cloud/backend/README.md

Index: tools/android/loading/cloud/backend/README.md
diff --git a/tools/android/loading/gce/README.md b/tools/android/loading/cloud/backend/README.md
similarity index 64%
rename from tools/android/loading/gce/README.md
rename to tools/android/loading/cloud/backend/README.md
index 2bbd06b7c624175152e8fdb4886606376c959063..5f534f2268e7a9c24ce8089afec7518d0e4f1990 100644
--- a/tools/android/loading/gce/README.md
+++ b/tools/android/loading/cloud/backend/README.md
@@ -20,7 +20,7 @@ ninja -C $BUILD_DIR -j1000 -l60 chrome chrome_sandbox
 # CLOUD_STORAGE_PATH is the path in Google Cloud Storage under which the
 # Clovis deployment will be uploaded.
 
-./tools/android/loading/gce/deploy.sh $BUILD_DIR $CLOUD_STORAGE_PATH
+./tools/android/loading/cloud/backend/deploy.sh $BUILD_DIR $CLOUD_STORAGE_PATH
 ```
 
 ## Start the app in the cloud
@@ -32,15 +32,15 @@ gcloud compute instances create clovis-tracer-1 \
  --machine-type n1-standard-1 \
  --image ubuntu-14-04 \
  --zone europe-west1-c \
- --scopes cloud-platform \
- --metadata cloud-storage-path=$CLOUD_STORAGE_PATH,auto-start=true \
+ --scopes cloud-platform,https://www.googleapis.com/auth/cloud-taskqueue \
+ --metadata cloud-storage-path=$CLOUD_STORAGE_PATH,taskqueue_tag=some_tag \
  --metadata-from-file \
-     startup-script=$CHROMIUM_SRC/tools/android/loading/gce/startup-script.sh
+     startup-script=$CHROMIUM_SRC/tools/android/loading/cloud/backend/startup-script.sh
 ```
 
 **Note:** To start an instance without automatically starting the app on it,
-remove the `--metadata auto-start=true` argument. This can be useful when doing
-iterative development on the instance, to be able to restart the app manually.
+add a `auto-start=false` metadata. This can be useful when doing iterative
+development on the instance, to be able to restart the app manually.
 
 This should output the IP address of the instance.
 Otherwise the IP address can be retrieved by doing:
@@ -60,24 +60,9 @@ gcloud compute instances get-serial-port-output clovis-tracer-1
 
 ## Use the app
 
-Check that `http://<instance-ip>:8080/test` prints `hello` when opened in a
-browser.
-
-To send a list of URLs to process:
-
-```shell
-curl -X POST -d @urls.json http://<instance-ip>:8080/set_tasks
-```
-
-where `urls.json` is a JSON dictionary with the keys:
-
-*   `urls`: array of URLs
-*   `repeat_count`: Number of times each URL will be loaded. Each load of a URL
-    generates a separate trace file. Optional.
-*   `emulate_device`: Name of the device to emulate. Optional.
-*   `emulate_network`: Type of network emulation. Optional.
-
-You can follow the progress at `http://<instance-ip>:8080/status`.
+Create tasks from the associated AppEngine application, see [documentation][3].
+Make sure the `taskqueue_tag` of the AppEngine request matches the one of the
+ComputeEngine instances.
 
 ## Stop the app in the cloud
 
@@ -98,7 +83,16 @@ From a new directory, set up a local environment:
 ```shell
 virtualenv env
 source env/bin/activate
-pip install -r $CHROMIUM_SRC/tools/android/loading/gce/pip_requirements.txt
+pip install -r \
+    $CHROMIUM_SRC/tools/android/loading/cloud/backend/pip_requirements.txt
+```
+
+The first time, you may need to get more access tokens:
+
+```shell
+gcloud beta auth application-default login --scopes \
+    https://www.googleapis.com/auth/cloud-taskqueue \
+    https://www.googleapis.com/auth/cloud-platform
 ```
 
 Create a JSON file describing the deployment configuration:
@@ -115,7 +109,8 @@ cat >$CONFIG_FILE << EOF
   "project_name" : "$PROJECT_NAME",
   "cloud_storage_path" : "$CLOUD_STORAGE_PATH",
   "chrome_path" : "$CHROME_PATH",
-  "src_path" : "$CHROMIUM_SRC"
+  "src_path" : "$CHROMIUM_SRC",
+  "taskqueue_tag" : "some_tag"
 }
 EOF
 ```
@@ -123,12 +118,11 @@ EOF
 Launch the app, passing the path to the deployment configuration file:
 
 ```shell
-gunicorn --workers=1 --bind 127.0.0.1:8080 \
-    --pythonpath $CHROMIUM_SRC/tools/android/loading/gce \
-    'main:StartApp('\"$CONFIG_FILE\"')'
+python $CHROMIUM_SRC/tools/android/loading/cloud/backend/worker.py \
+    --config $CONFIG_FILE
 ```
 
-You can now [use the app][2], which is located at http://localhost:8080.
+You can now [use the app][2].
 
 Tear down the local environment:
 
@@ -138,3 +132,4 @@ deactivate
 
 [1]: https://cloud.google.com/sdk
 [2]: #Use-the-app
+[3]: ../frontend/README.md