bin/sync-and-gyp: sh->py (make more cross-platform)

bin/sync-and-gyp

Index: bin/sync-and-gyp
diff --git a/bin/sync-and-gyp b/bin/sync-and-gyp
index c944079938c771e5b85c8b548e768ed4b63c5545..2eee852596b4f89b9dee1e870ca47f6196761f8e 100755
--- a/bin/sync-and-gyp
+++ b/bin/sync-and-gyp
@@ -1,58 +1,81 @@
-#!/bin/sh
+#!/usr/bin/env python
 
 # Copyright 2015 Google Inc.
 #
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
-# This script will update Skia's dependenciess as necessary and run
+# This script will update Skia's dependencies as necessary and run
 # gyp if needed.
 
-# Depends on: Posix-compliant shell, Python, and Git.
+# Depends on: Python, and Git.

[bungeman-skia, 2015/11/09 15:30:09]

Extra comma. Also, aside from the call to git-sync-deps, this doesn't appear to
need git.

 #
 # Example usage:
 #
 #   git clone https://skia.googlesource.com/skia
 #   cd skia
-#   bin/sync-and-gyp
+#   python bin/sync-and-gyp
 #   ninja -C out/Debug && out/Debug/dm
 #
 # Once changes are made to DEPS or gyp/ or the source, call:
 #
-#   bin/sync-and-gyp
-
-if [ "$SKIA_OUT" ]; then
-    mkdir -p "$SKIA_OUT" || exit
-    # get non-relative path of $SKIA_OUT before changing directory.
-    SKIA_OUT="$(cd "$SKIA_OUT"; pwd)"
-fi
-
-cd "$(dirname "$0")/.."
-
-if ! [ -f DEPS ]; then
-    echo DEPS file missing >&2
-    exit 1
-fi
-
-GIT_SYNC_DEPS_QUIET=1 python tools/git-sync-deps || exit
-
-catifexists() { if [ -f "$1" ]; then cat "$1"; fi; }
-
-gyp_hasher() {
-    {
-        echo "$CC"
-        echo "$CXX"
-        echo "$GYP_GENERATORS"
-        echo "$GYP_DEFINES"
-        find gyp -type f -print -exec git hash-object {} \;
-        find bench gm tests -name '*.c*' | LANG= sort
-    } | git hash-object --stdin
-}
-
-: ${SKIA_OUT:=out}
-GYP_HASH=$(gyp_hasher)
-HASH_PATH="${SKIA_OUT}/gyp_hash"
-if [ "$GYP_HASH" != "$(catifexists "$HASH_PATH")" ]; then
-    python ./gyp_skia || exit
-    echo "$GYP_HASH" > "$HASH_PATH"
-fi
+#   python bin/sync-and-gyp
+
+import fnmatch
+import hashlib
+import subprocess
+import os
+
+skia_dir = os.path.join(os.path.dirname(__file__), os.pardir)
+
+skia_out = os.environ.get("SKIA_OUT")
+if skia_out:
+  skia_out = os.path.abspath(skia_out)
+else:
+  skia_out = 'out'

[mtklein, 2015/11/09 15:14:20]

Seems like we'd want to abspath this too?

[hal.canary, 2015/11/09 15:21:51]

No need.  It will be relative to `skia_dir`.

+
+os.chdir(skia_dir)
+
+if not os.path.isfile('DEPS'):
+  sys.stderr.write('DEPS file missing')
+  exit(1)
+
+env = os.environ.copy()
+env["GIT_SYNC_DEPS_QUIET"] = "1"
+subprocess.call(['python', 'tools/git-sync-deps'], env=env)
+
+hasher = hashlib.sha1()
+
+for var in ['CC', 'CXX', 'GYP_GENERATORS', 'GYP_DEFINES']:

[mtklein, 2015/11/09 15:14:20]

Might add in CFLAGS, CPPFLAGS, CXXFLAGS?

(Alternatively, the entire environment?)

[hal.canary, 2015/11/09 15:21:51]

Does gyp respect those?  If so, yes.
Seems like overkill.

[bungeman-skia, 2015/11/09 15:30:08]

The environment variables which at least the ninja generator bakes in should be
used here.

CC, CXX, CPPFLAGS, CFLAGS, CPPFLAGS_host, CFLAGS_host, CC_host, CC_target,
CXX_host, CXX_target, AR, AR_host, AR_target, NM, NM_host, NM_target, READELF,
READELF_host, READELF_target

That's the quick list, there are probably more.

+  hasher.update(os.environ.get(var, '') + '\n')
+
+def listfiles(folder, filter):
+  for root, folders, files in os.walk(folder):
+    for filename in folders + files:
+      if fnmatch.fnmatch(filename, filter):
+        yield os.path.join(root, filename)
+
+for filename in sorted(listfiles('gyp', '*')):
+  with open(filename, 'r') as f:
+    hasher.update(f.read())

[bungeman-skia, 2015/11/09 15:30:08]

It seems unfortunate to need to read the content, as opposed to using an already
computed hash.

[hal.canary, 2015/11/09 16:16:26]

I don't have a better solution.  How does git do this?

+
+for dir in ['bench', 'gm', 'tests']:

[mtklein, 2015/11/09 15:14:20]

Think it makes sense to generalize this to all directories?  You're probably
going to need to rerun GYP if a new source file shows up anywhere.

[bungeman-skia, 2015/11/09 15:30:08]

It does seem unfortunate that this file needs to be kept in sync with the
globbing. Also, is scanning all the files worth it? Re-gyping Skia doesn't
exactly take a long time. Most of the win of this script is only pulling DEPS
when needed? Or is it the principle of the thing?

[mtklein, 2015/11/09 15:36:55]

Being able to skip re-Gyping Skia on Windows seems like the selling point of the
whole CL.

+  for filename in sorted(listfiles(dir, '*.c*')):
+    hasher.update(filename + '\n')
+
+gyp_hash = hasher.hexdigest()
+
+hash_path = os.path.join(skia_out, 'gyp_hash')
+
+def cat_if_exists(path):
+  if os.path.exists(path):
+    with open(path, 'r') as f:
+      return f.read()
+  return ''
+
+if cat_if_exists(hash_path).strip() != gyp_hash:
+  env = os.environ.copy()
+  env['SKIA_OUT'] = skia_out

[bungeman-skia, 2015/11/09 15:30:08]

Before this created this directory, but this no longer seems to be the case.
Does this directory need to be created here?

[hal.canary, 2015/11/09 16:16:26]

I created the directory because $(cd "$dir"; pwd) is the posix-shell way of
doing os.path.abspath(dir).  If successful, `gyp_skia` will create the
directory.

+  subprocess.call(['python', './gyp_skia'], env=env)
+  with open(hash_path, 'w') as o:
+    o.write(gyp_hash)