Skeleton code for running the forthcoming async/await compiler on pub.

sdk/lib/_internal/pub/bin/async_compile.dart

Index: sdk/lib/_internal/pub/bin/async_compile.dart
diff --git a/sdk/lib/_internal/pub/bin/async_compile.dart b/sdk/lib/_internal/pub/bin/async_compile.dart
new file mode 100644
index 0000000000000000000000000000000000000000..a8cedc5aa0296e86ec58f867f6a35da259101126
--- /dev/null
+++ b/sdk/lib/_internal/pub/bin/async_compile.dart
@@ -0,0 +1,94 @@
+// Copyright (c) 2014, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'dart:io';
+
+import 'package:path/path.dart' as p;
+
+/// The path to pub's root directory (sdk/lib/_internal/pub) in the Dart repo.
+///
+/// This assumes this script is itself being run from within the repo.
+final sourceDir = p.dirname(p.dirname(p.fromUri(Platform.script)));
+
+/// The directory that compiler output should be written to.
+String buildDir;
+
+/// This runs the async/await compiler on all of the pub source code.
+///
+/// It reads from the repo and writes the compiled output into the given build
+/// directory (using the same file names and relative layout). Does not
+/// compile files that haven't changed since the last time they were compiled.
+// TODO(rnystrom): Remove this when #104 is fixed.
+void main(List<String> arguments) {

[nweiz, 2014/08/22 19:29:15]

This needs to be more resilient to being run in parallel. test.dart will run
many test suites at once, which will all try to invoke this. Maybe it should
take out some sort of lock in the build output directory?

[Bob Nystrom, 2014/08/25 17:40:00]

Done.

It doesn't check the directory for existence before creating it since that's a
potential race condition. Instead, it just always calls
directory.create(recursive: true), which will do nothing if the directory
already exists.

When it writes the output file, it catches and ignores an IO exception there
since it likely means another compiler is writing the same file.

I think that covers all of the cases. I haven't seen any issues running the
tests locally, though that doesn't necessarily mean that much. I'll keep an eye
on the bots after this lands.

[nweiz, 2014/08/25 19:40:56]

This is probably sufficient to avoid crashes, but multiple instances of
async_compile will still probably do a lot of redundant work.

+  _validate(arguments.isNotEmpty, "Missing build directory.");
+  _validate(arguments.length <= 2, "Unexpected arguments.");
+  if (arguments.length == 2) {
+    _validate(arguments[1] == "--silent",
+        "Invalid argument '${arguments[1]}");

[nweiz, 2014/08/22 19:29:15]

Why not just use args? It would make this parsing clearer and make using this
more flexible from the command line.

[Bob Nystrom, 2014/08/25 17:40:00]

It felt like more trouble than it's worth. This script is temporary and isn't
something I expect any end users to use.

+  }
+
+  // Create the build output directory if not already there.

[nweiz, 2014/08/22 19:29:15]

Nit: "if not" -> "if it's not"

[Bob Nystrom, 2014/08/25 17:40:00]

Done.

+  buildDir = p.join(p.normalize(arguments[0]), "pub_async");
+  if (!new Directory(buildDir).existsSync()) {
+    new Directory(buildDir).createSync(recursive: true);
+  }

[nweiz, 2014/08/22 19:29:15]

This script lives in pub; I'd just import pub's io library and use ensureDir
here.

[Bob Nystrom, 2014/08/25 17:40:00]

We'll probably want to use async syntax in io.dart before too long. :)

This script, by design, has minimal dependencies since anything it imports can't
use async/await.

+
+  var silent = arguments.length == 2 && arguments[1] == "--silent";
+  var numFiles = 0;
+  var numCompiled = 0;
+
+  // Compile any modified or missing files.
+  for (var entry in new Directory(sourceDir).listSync(recursive: true)) {
+    if (p.extension(entry.path) != ".dart") continue;
+
+    // Skip tests.
+    // TODO(rnystrom): Do we want to use this for tests too?
+    if (p.isWithin(p.join(sourceDir, "test"), entry.path)) continue;
+
+    numFiles++;
+    var relative = p.relative(entry.path, from: sourceDir);
+
+    var sourceFile = entry as File;
+    var destPath = p.join(buildDir, relative);
+    var destFile = new File(destPath);
+    if (!destFile.existsSync() ||
+        entry.lastModifiedSync().isAfter(destFile.lastModifiedSync())) {
+      compile(sourceFile.path, sourceFile.readAsStringSync(), destPath);
+      numCompiled++;
+      if (!silent) print("Compiled ${sourceFile.path}");

[nweiz, 2014/08/22 19:29:15]

Nit: add a period.

[Bob Nystrom, 2014/08/25 17:40:00]

Done.

+    }
+  }
+
+  if (!silent) print("Compiled $numCompiled out of $numFiles files.");
+}
+
+final _compilerPattern = new RegExp(r'(\.\./)+compiler');

[nweiz, 2014/08/22 19:29:15]

I'd feel a little better if this at least started with ^import.

[Bob Nystrom, 2014/08/25 17:40:00]

Done.

+
+void compile(String sourcePath, String source, String destPath) {
+  var destDir = new Directory(p.dirname(destPath));
+  if (!destDir.existsSync()) destDir.createSync(recursive: true);
+
+  // TODO(rnystrom): Do real async/await transformation here!
+  source = source.replaceAll("ASYNC!", "");
+
+  // Pub imports dart2js using relative imports that reach outside of pub's
+  // source tree. Since the build directory is in a different location, we need
+  // to fix those to be valid relative imports from the build directory.
+  var compilerDir = p.join(sourceDir, "../compiler");
+  var relative = p.relative(compilerDir, from: destDir.path);

[nweiz, 2014/08/22 19:29:15]

Use p.url for manipulating import paths.

[Bob Nystrom, 2014/08/25 17:40:00]

Done.

+  source = source.replaceAll(_compilerPattern, relative);
+
+  new File(destPath).writeAsStringSync(source);
+}
+
+/// Validates command-line argument usage and exits with [message] if [valid]
+/// is `false`.
+void _validate(bool valid, String message) {
+  if (valid) return;
+
+  stderr.writeln(message);
+  stderr.writeln();
+  stderr.writeln("Usage: dart async_compile.dart <build dir> [--silent]");
+  exit(64);
+}