Downloading Node and NPM deps via gclient sync.

third_party/node/update_node_binaries

Index: third_party/node/update_node_binaries
diff --git a/third_party/node/update_node_binaries b/third_party/node/update_node_binaries
new file mode 100755
index 0000000000000000000000000000000000000000..9b92f82703f5ce462a763cdd74aaf02ccde860b1
--- /dev/null
+++ b/third_party/node/update_node_binaries
@@ -0,0 +1,97 @@
+#!/bin/bash
+
+# Copyright 2017 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+# Script for updating Node binaries.
+# 1) Update NODE_VERSION variable below to the desired version.
+# 2) Run this script.
+# 3) Upload the binaries to the Google Storage bucket (commands to upload
+#    binaries are printed at step 2, look for "gsutil.py").
+# 4) Land a CL with the changes generated by this script.
+
+cd "$(dirname "$0")"
+
+BASE_URL="https://nodejs.org/dist"
+NODE_VERSION="v6.9.4"
+
+#######################################
+# Updates Node binary for Mac
+#######################################
+update_mac() {
+  local SUFFIX="darwin-x64"
+  local FILENAME="node-${NODE_VERSION}-${SUFFIX}.tar.gz"
+  local FOLDER="mac"
+  local URL="${BASE_URL}/${NODE_VERSION}/${FILENAME}"
+  rm -f "${FOLDER}/${FILENAME}"
+  wget -P "${FOLDER}/" "${URL}"

[palmer, 2017/01/13 01:02:27]

Might be good to check if wget fails, for whatever reason (including failure to
check the nodejs.org HTTPS certificate). 1 thing you can do is

    set -e

at the top of the script, to have it just bail out on the first error. If you
don't want to do that you can just do the usual

    if ! wget ...; then
      exit 1
    fi

or whatever.

[M-A Ruel, 2017/01/13 02:25:04]

I highly recomment "set -eu". It's more annoying to write the script when you
expect subcommands to normally exit non zero but it's much safer.

[dpapad, 2017/01/13 03:28:10]

Done.

+
+  # Unpack temporarily, delete NPM symlink and re-pack.
+  tar xfz "${FOLDER}/${FILENAME}" -C "${FOLDER}/"
+  rm "${FOLDER}/${FILENAME}"
+  rm "${FOLDER}/node-${NODE_VERSION}-${SUFFIX}/bin/npm"
+
+  # Drop the version info from the name, since it is redundant and would make
+  # rolling new versions more involved.
+  mv "${FOLDER}/node-${NODE_VERSION}-${SUFFIX}/" "${FOLDER}/node-${SUFFIX}/"
+  tar cfz "${FOLDER}/node-${SUFFIX}.tar.gz" -C "${FOLDER}" "node-${SUFFIX}/"
+  local sha1
+  sha1="$(sha1sum ${FOLDER}/node-${SUFFIX}.tar.gz | cut -d ' ' -f1)"
+  echo "${sha1}" > "${FOLDER}/node-${SUFFIX}.tar.gz.sha1"

[palmer, 2017/01/13 01:02:28]

Maybe check the checksum that Node distributes? E.g.
https://nodejs.org/dist/latest-v6.x/SHASUMS256.txt.

If gsutil needs SHA-1, it's ok to use that, but we should at least do the
minimal check before assuming that the archive we got is good.

It'd be even better if we could get the archive and the SHA-256s from different
sources, in case 1 or the other gets compromised. (The real solution to this
problem is Binary Transparency, but for now let's just dot our i's and cross our
t's.)

[dpapad, 2017/01/13 03:28:10]

It's download_from_google_storage script that requires SHA1 (see
https://cs.chromium.org/chromium/tools/depot_tools/download_from_google_stora...
and all over that file).

I added checks to ensure that the downloaded binaries match the SHA256 hashes.

+  echo "Please execute manually the following:"
+  echo "> gsutil.py cp ${FOLDER}/node-${SUFFIX}.tar.gz gs://chromium-nodejs/${NODE_VERSION:1}/${sha1}"
+  echo "DONE updating Mac."
+}
+
+#######################################
+# Updates Node binary for Linux
+#######################################
+update_linux() {
+  local SUFFIX="linux-x64"
+  local FILENAME="node-${NODE_VERSION}-${SUFFIX}.tar.xz"
+  local FOLDER="linux"
+  local URL="${BASE_URL}/${NODE_VERSION}/${FILENAME}"
+  rm -f "${FOLDER}/${FILENAME}"
+  wget -P "${FOLDER}/" "${URL}"
+
+  # Unpack temporarily, delete NPM symlink and re-pack as gz (not xz).
+  tar xf "${FOLDER}/${FILENAME}" -C "${FOLDER}/"
+  rm "${FOLDER}/${FILENAME}"
+  rm "${FOLDER}/node-${NODE_VERSION}-${SUFFIX}/bin/npm"
+
+  # Drop the version info from the name, since it is redundant and would make
+  # rolling new versions more involved.
+  mv "${FOLDER}/node-${NODE_VERSION}-${SUFFIX}/" "${FOLDER}/node-${SUFFIX}/"
+  tar cfz "${FOLDER}/node-${SUFFIX}.tar.gz" -C "${FOLDER}" "node-${SUFFIX}/"
+  local sha1
+  sha1="$(sha1sum ${FOLDER}/node-${SUFFIX}.tar.gz | cut -d ' ' -f1)"
+  echo "${sha1}" > "${FOLDER}/node-${SUFFIX}.tar.gz.sha1"
+  echo "Please execute manually the following:"
+  echo "> gsutil.py cp ${FOLDER}/node-${SUFFIX}.tar.gz gs://chromium-nodejs/${NODE_VERSION:1}/${sha1}"
+  echo "DONE updating Linux."
+}
+
+#######################################
+# Updates Node binary for Windows
+#######################################

[palmer, 2017/01/13 01:02:27]

Nit: I'd argue these comment blocks are unnecessary; the function names explain
themselves well enough.

[dpapad, 2017/01/13 03:28:10]

Removed.

+update_win() {
+  local FILENAME="node.exe"
+  local FOLDER="win"
+  local WINDOWS_URL="${BASE_URL}/${NODE_VERSION}/win-x64/${FILENAME}"
+  rm -f "${FOLDER}/${FILENAME}"
+  wget -P "${FOLDER}/" "${WINDOWS_URL}"
+  local sha1
+  sha1="$(sha1sum ${FOLDER}/node.exe | cut -d ' ' -f1)"
+  echo "${sha1}" > "${FOLDER}/node.exe.sha1"
+  echo "Please execute manually the following:"
+  echo "> gsutil.py cp ${FOLDER}/node.exe gs://chromium-nodejs/${NODE_VERSION:1}/${sha1}"
+  echo "DONE updating Windows."
+}
+
+update_linux
+update_mac
+update_win
+
+# Update DEPS to point to the new Google Storage bucket subfolder.
+sed -i "s@\(chromium-nodejs/\)\([0-9\.]\)\+@\1${NODE_VERSION:1}@" ../../DEPS