Make the editor build callable from tools/build.py

tools/create_editor.py

Index: tools/create_editor.py
===================================================================
--- tools/create_editor.py	(revision 0)
+++ tools/create_editor.py	(revision 0)
@@ -0,0 +1,171 @@
+#!/usr/bin/env python
+#
+# Copyright (c) 2012, the Dart project authors.  Please see the AUTHORS file

[ricow1, 2013/01/08 07:08:26]

Since this is a new file - 2013

[devoncarew, 2013/01/09 21:08:09]

Done.

+# for details. All rights reserved. Use of this source code is governed by a
+# BSD-style license that can be found in the LICENSE file.
+#
+# A script which will be invoked from gyp to create a build of the editor.
+#
+# Usage: ./tools/build.py editor

[ricow1, 2013/01/08 07:08:26]

I do not think this is valid with the current cl (but I am actually more than
fine with us adding that in a follow up)

[devoncarew, 2013/01/09 21:08:09]

I'll add a TODO: here saying that it doesn't work that way yet, but that's our
goal.

+#        ./tools/build_editor.py <output>
+
+import glob

[ricow1, 2013/01/08 07:08:26]

could we order these alphabetically

[devoncarew, 2013/01/09 21:08:09]

I think they are already.

+import os
+import shutil
+import subprocess
+import sys
+import utils
+import zipfile
+
+from os.path import join
+
+OUTPUT = None
+BUILDER_CLOBBER = 'BUILDBOT_CLOBBER'
+
+OS_CONFIG = {
+  'win32': 'win32, win32',
+  'linux': 'linux, gtk',
+  'macos': 'macosx, cocoa'
+}
+
+ARCH_CONFIG = {
+  'ia32': 'x86',
+  'x64': 'x86_64'
+}
+
+def Main(argv):

[ricow1, 2013/01/08 07:08:26]

move Main method down right above
if __name__ == '__main__':
  sys.exit(Main(sys.argv))

[devoncarew, 2013/01/09 21:08:09]

Done.

+  global OUTPUT
+  
+  if len(argv) != 2:
+    print 'usage: tools/create_editor.py <output>'
+    return 0
+  
+  OUTPUT = argv[1]

[ricow1, 2013/01/08 07:08:26]

why not use utils.GetBuildDir (see below, we also need to import this) and then
create a editor directory in there

+  if not os.path.isabs(OUTPUT):
+    OUTPUT = os.path.abspath(OUTPUT)

[ricow1, 2013/01/08 07:08:26]

Verify that this is a valid directory (or even better, use utils.GetBuildDir

+  
+  osName = utils.GuessOS()

[ricow1, 2013/01/08 07:08:26]

can you do this, I think you need to import our utils lib for this to work

[devoncarew, 2013/01/09 21:08:09]

We are importing it...

+  mode = ('release', 'debug')['Debug' in OUTPUT]

[ricow1, 2013/01/08 07:08:26]

could we add a --mode option instead, defaulting to Debug, this is standard for
our normal build script

[devoncarew, 2013/01/09 21:08:09]

My goal is for this to be callable from our normal build script. and for it to
determine the mode and arch options automatically based on the path. My template
is how create_sdk is called from the dart.gyp file.

[ricow1, 2013/01/10 13:23:12]

My only concern here is: when we change the name of the output directory this is
going to break (I would really like us to only have to change stuff like that in
one place). 

[devoncarew, 2013/01/11 17:08:48]

Gotcha, and that makes perfect sense to me. I'll keep the output path name
scrapping, but add --mode and --arch flags which will be used if passed in. I'll
also add a TODO: that we'll want to revisit this when the script is callable
from dart.gyp.

+  arch = ('ia32', 'x64')['X64' in OUTPUT]

[ricow1, 2013/01/08 07:08:26]

same as for --mode

+
+  print "\nBuilding the editor"
+  print "  config : %s, %s, %s" % (osName, arch, mode)
+  print "  output : %s" % OUTPUT
+
+  if GetShouldClobber():

[ricow1, 2013/01/08 07:08:26]

I think this still belongs in the annotated steps script, not here. This is
buildbot specific functionality, but I see that there are some additional stuff
being deleted, so maybe we need to update the annotated steps to actually do
that?

[devoncarew, 2013/01/09 21:08:09]

Ah, from looking at tools/clean_output_directory.py, it looks like it does
everything we need on a clobber. I'll remove this if statement.

+    shutil.rmtree(OUTPUT, True)
+    shutil.rmtree(GetDownloadCache(), True)
+    shutil.rmtree(GetEditorTemp(), True)
+    shutil.rmtree(GetEclipseBuildRoot(), True)
+  else:
+    DeleteDirContents(OUTPUT)

[danrubel, 2013/01/07 23:07:54]

shutil.rmtree(OUTPUT, True) instead of the above line?

[devoncarew, 2013/01/07 23:21:22]

Done.

[ricow1, 2013/01/08 07:08:26]

Could we add a TODO here to say to remove when we make this incremental

[ricow1, 2013/01/08 07:08:26]

I would probably also print that I am deleting the directory, seems like
relevant info

[devoncarew, 2013/01/09 21:08:09]

I added a printout for when we clean the out directory.

Even when the build becomes incremental, we'll still want to clean the output
directory before a build. The eclipse build artifacts are named based on
timestamps and svn rev ids. If we don't clean before a build we'll have lots of
plugins accumulate very fast.

+
+  # macosx, cocoa, x86 & macosx, cocoa, x86_64

[ricow1, 2013/01/08 07:08:26]

I have a hard time reading these comments

[devoncarew, 2013/01/09 21:08:09]

I'll add some explaination.

+  # win32, win32, x86 & win32, win32, x86_64
+  # linux, gtk, x86 & linux, gtk, x86_64
+
+  buildConfig = OS_CONFIG[osName] + ', ' + ARCH_CONFIG[arch]
+
+  print "\ninvoking build_rcp.xml with buildConfig = [%s]\n" % buildConfig
+
+  sys.stdout.flush()
+  sys.stderr.flush()
+
+  buildScript = join('editor', 'tools', 'features', 
+                     'com.google.dart.tools.deploy.feature_releng', 
+                     'build_rcp.xml')
+
+  buildRcpStatus = subprocess.call(
+      [AntPath(), 
+      '-lib',
+      join('third_party', 'bzip2', 'bzip2.jar'),
+      '-Dbuild.out=' + OUTPUT,
+      '-Dbuild.configs=' + buildConfig,
+      '-Dbuild.revision=' + utils.GetSVNRevision(),
+      '-Ddart.version.full=' + utils.GetVersion(),
+      '-Dbuild.root=' + GetEclipseBuildRoot(),
+      '-Dbuild.downloads=' + GetDownloadCache(),
+      '-Dbuild.source=' + os.path.abspath('editor'),
+      '-Dbuild.dart.sdk=' + GetSdkPath(),
+      '-Dbuild.no.properties=true',
+      '-buildfile',
+      buildScript],
+    shell=utils.IsWindows())
+
+  if buildRcpStatus != 0:
+    sys.exit(buildRcpStatus)
+
+  # build_rcp.xml will put the built editor archive in the OUTPUT directory
+  # (dart-editor-macosx.cocoa.x86.zip). It contains the editor application in a
+  # dart/ subdirectory. We unzip the contents of the archive into OUTPUT. It
+  # will use the ../dart-sdk directory as its SDK.
+  archives = glob.glob(join(OUTPUT, '*.zip'))
+  
+  if (archives):
+    ProcessEditorArchive(archives[0], OUTPUT)
+
+  if os.path.exists(GetEditorTemp()):
+    shutil.rmtree(GetEditorTemp())
+
+  print('\nEditor build successful')
+
+
+def AntPath():
+  parent = join('third_party', 'apache_ant', 'v1_7_1', 'bin')
+  if utils.IsWindows():
+    return join(parent, 'ant.bat')
+  else:
+    return join(parent, 'ant')
+
+
+def ProcessEditorArchive(archive, outDir):
+  tempDir = join(GetEditorTemp(), 'editor.out')
+  os.makedirs(tempDir)
+
+  if utils.IsWindows():
+    f = zipfile.ZipFile(archive)
+    f.extractall(tempDir)
+  else:
+    subprocess.call(['unzip', '-q', archive, '-d', tempDir])
+
+  for src in glob.glob(join(tempDir, 'dart', '*')):
+    shutil.move(src, outDir)
+
+  shutil.rmtree(tempDir)
+  os.unlink(archive)
+
+
+def GetEditorTemp():
+  return join(GetOutputParent(), 'editor.build.temp')
+
+
+def GetDownloadCache():
+  return join(os.path.expanduser('~'), '.editor.download.cache')

[ricow1, 2013/01/08 07:08:26]

Any reason why not to put this into the output directory?

[devoncarew, 2013/01/09 21:08:09]

No, that sounds reasonable - done.

+
+
+def GetEclipseBuildRoot():
+  return join(GetOutputParent(), 'editor.build.cache')
+
+
+def GetSdkPath():
+  return join(os.path.dirname(OUTPUT), 'dart-sdk')
+
+
+def GetOutputParent():
+  return os.path.dirname(os.path.dirname(OUTPUT))
+
+
+def DeleteDirContents(directory):
+  for root, dirs, files in os.walk(directory):
+    for f in files:
+      os.unlink(join(root, f))
+    for d in dirs:
+      shutil.rmtree(join(root, d))
+
+
+def GetShouldClobber():
+  return os.environ.get(BUILDER_CLOBBER) == "1"
+
+
+if __name__ == '__main__':
+  sys.exit(Main(sys.argv))