Add setup scripts for a Graphite server.

Add setup scripts for a Graphite server.

BUG=skia:

Committed: https://skia.googlesource.com/buildbot/+/bdfa827ef0bcb15f2d11333072b29274190eb377

[jcgregorio, 2014-06-05 18:32:38]

On 2014/06/05 13:59:54, jcgregorio wrote:

Up and running at http://skiamonitor.com/dashboard/#Graphite

[jcgregorio, 2014-06-09 22:27:04]

On 2014/06/05 18:32:38, jcgregorio wrote:
> On 2014/06/05 13:59:54, jcgregorio wrote:
> 
> Up and running at http://skiamonitor.com/dashboard/#Graphite

ping

[rmistry, 2014-06-10 11:57:59]

https://codereview.chromium.org/310253003/diff/120001/compute_engine_scripts/...
File compute_engine_scripts/monitoring/README.md (right):

https://codereview.chromium.org/310253003/diff/120001/compute_engine_scripts/...
compute_engine_scripts/monitoring/README.md:1: Monitoring (Graphite)
Can you also please add a small section to
compute_engine_scripts/buildbots/README talking about what is in the monitoring
directory.

https://codereview.chromium.org/310253003/diff/120001/compute_engine_scripts/...
compute_engine_scripts/monitoring/README.md:13: =================
The different constants below could change based on the values in vm_config.sh
and based on the zone we are in.
Maybe it makes sense to create vm_create_server.sh, vm_delete_server.sh and
vm_setup_server.sh scripts in this directory that use the constants in
vm_config.sh

You can then refer to those scripts here instead of hardcoding values.

https://codereview.chromium.org/310253003/diff/120001/compute_engine_scripts/...
File compute_engine_scripts/monitoring/graphite_setup.sh (right):

https://codereview.chromium.org/310253003/diff/120001/compute_engine_scripts/...
compute_engine_scripts/monitoring/graphite_setup.sh:50: sudo /etc/init.d/apache2
restart
Should we create a crontab that runs on reboot in case of VM crashes, this will
ensure that the server automatically comes up, we do this for all slaves and
masters.

[jcgregorio, 2014-06-16 12:36:24]

https://codereview.chromium.org/310253003/diff/120001/compute_engine_scripts/...
File compute_engine_scripts/monitoring/README.md (right):

https://codereview.chromium.org/310253003/diff/120001/compute_engine_scripts/...
compute_engine_scripts/monitoring/README.md:1: Monitoring (Graphite)
On 2014/06/10 11:57:58, rmistry wrote:
> Can you also please add a small section to
> compute_engine_scripts/buildbots/README talking about what is in the
monitoring
> directory.

Done.

https://codereview.chromium.org/310253003/diff/120001/compute_engine_scripts/...
compute_engine_scripts/monitoring/README.md:13: =================
On 2014/06/10 11:57:58, rmistry wrote:
> The different constants below could change based on the values in vm_config.sh
> and based on the zone we are in.
> Maybe it makes sense to create vm_create_server.sh, vm_delete_server.sh and
> vm_setup_server.sh scripts in this directory that use the constants in
> vm_config.sh
> 
> You can then refer to those scripts here instead of hardcoding values.

Done.

https://codereview.chromium.org/310253003/diff/120001/compute_engine_scripts/...
File compute_engine_scripts/monitoring/graphite_setup.sh (right):

https://codereview.chromium.org/310253003/diff/120001/compute_engine_scripts/...
compute_engine_scripts/monitoring/graphite_setup.sh:50: sudo /etc/init.d/apache2
restart
Added monit config to keep Apache running.

On 2014/06/10 11:57:58, rmistry wrote:
> Should we create a crontab that runs on reboot in case of VM crashes, this
will
> ensure that the server automatically comes up, we do this for all slaves and
> masters.

[rmistry, 2014-06-16 13:03:38]

A few minor comments

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
File compute_engine_scripts/buildbots/README (right):

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
compute_engine_scripts/buildbots/README:46: Setup scripts for skiamonitor.org, a
Graphite server.
Ah I was mistaken, I thought monitoring was a subdirectory under buildbots.
I guess we can either:
* Make monitoring a subdirectory under buildbots.
or
* Leave monitoring where it is but specify in monitoring/README.md that if its
constants need to be modified then they are specified in
compute_engine_scripts/buildbots/vm_config.sh

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
compute_engine_scripts/buildbots/README:46: Setup scripts for skiamonitor.org, a
Graphite server.
"Setup scripts for" -> "Scripts to create/setup/delete"

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
File compute_engine_scripts/monitoring/vm_create_instance.sh (right):

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
compute_engine_scripts/monitoring/vm_create_instance.sh:8: # Basically SSH in,
update the copy of this repo and jump to a shell script in
Please update or delete this comment.

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
compute_engine_scripts/monitoring/vm_create_instance.sh:14:
--service_account_scopes="https://www.googleapis.com/auth/devstorage.full_control"
\
This constant is specified in vm_config.sh

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
compute_engine_scripts/monitoring/vm_create_instance.sh:16:
--machine_type=n1-standard-1 \
Could you create a constant in vm_config.sh for the machine_type, maybe
something like MONITORING_MACHINE_TYPE.

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
compute_engine_scripts/monitoring/vm_create_instance.sh:17:
--image=backports-debian-7-wheezy-v20140331 \
Could you create a constant in vm_config.sh for the image, maybe something like
MONITORING_IMAGE

[jcgregorio, 2014-06-16 13:22:43]

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
File compute_engine_scripts/buildbots/README (right):

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
compute_engine_scripts/buildbots/README:46: Setup scripts for skiamonitor.org, a
Graphite server.
On 2014/06/16 13:03:37, rmistry wrote:
> Ah I was mistaken, I thought monitoring was a subdirectory under buildbots.
> I guess we can either:
> * Make monitoring a subdirectory under buildbots.
> or
> * Leave monitoring where it is but specify in monitoring/README.md that if its
> constants need to be modified then they are specified in
> compute_engine_scripts/buildbots/vm_config.sh

I chose the second route.

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
compute_engine_scripts/buildbots/README:46: Setup scripts for skiamonitor.org, a
Graphite server.
On 2014/06/16 13:03:37, rmistry wrote:
> "Setup scripts for" -> "Scripts to create/setup/delete"

Done.

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
File compute_engine_scripts/monitoring/vm_create_instance.sh (right):

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
compute_engine_scripts/monitoring/vm_create_instance.sh:8: # Basically SSH in,
update the copy of this repo and jump to a shell script in
On 2014/06/16 13:03:37, rmistry wrote:
> Please update or delete this comment.

Done.

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
compute_engine_scripts/monitoring/vm_create_instance.sh:14:
--service_account_scopes="https://www.googleapis.com/auth/devstorage.full_control"
\
On 2014/06/16 13:03:37, rmistry wrote:
> This constant is specified in vm_config.sh

Done.

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
compute_engine_scripts/monitoring/vm_create_instance.sh:16:
--machine_type=n1-standard-1 \
On 2014/06/16 13:03:37, rmistry wrote:
> Could you create a constant in vm_config.sh for the machine_type, maybe
> something like MONITORING_MACHINE_TYPE.

Done.

https://codereview.chromium.org/310253003/diff/160001/compute_engine_scripts/...
compute_engine_scripts/monitoring/vm_create_instance.sh:17:
--image=backports-debian-7-wheezy-v20140331 \
On 2014/06/16 13:03:38, rmistry wrote:
> Could you create a constant in vm_config.sh for the image, maybe something
like
> MONITORING_IMAGE

Done.

[rmistry, 2014-06-16 13:26:00]

LGTM

[jcgregorio, 2014-06-16 13:28:53]

The CQ bit was checked by jcgregorio@google.com

[commit-bot: I haz the power, 2014-06-16 13:29:09]

CQ is trying da patch. Follow status at
 https://skia-tree-status.appspot.com/cq/jcgregorio@google.com/310253003/180001

[commit-bot: I haz the power, 2014-06-16 13:29:19]

Change committed as bdfa827ef0bcb15f2d11333072b29274190eb377

[tfarina, 2014-09-23 04:02:46]

https://codereview.chromium.org/310253003/diff/180001/compute_engine_scripts/...
File compute_engine_scripts/monitoring/graphite_setup.sh (right):

https://codereview.chromium.org/310253003/diff/180001/compute_engine_scripts/...
compute_engine_scripts/monitoring/graphite_setup.sh:23: sudo mkdir
/home/www-data
could this be simplified to the following command line?

sudo adduser www-data

[jcgregorio, 2014-09-23 11:24:08]

https://codereview.chromium.org/310253003/diff/180001/compute_engine_scripts/...
File compute_engine_scripts/monitoring/graphite_setup.sh (right):

https://codereview.chromium.org/310253003/diff/180001/compute_engine_scripts/...
compute_engine_scripts/monitoring/graphite_setup.sh:23: sudo mkdir
/home/www-data
If I recall correctly this didn't work because the www-data user already exists
because Apache created it.

We will shortly be switching to InfluxDB so much of this will be going away.

On 2014/09/23 04:02:45, tfarina wrote:
> could this be simplified to the following command line?
> 
> sudo adduser www-data