add bazel worker for summary generation

pkg/front_end/tool/bazel/worker.dart

+// Copyright (c) 2017, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+import 'dart:async';
+import 'dart:io';
+
+import 'package:args/args.dart';
+import 'package:bazel_worker/bazel_worker.dart';
+import 'package:front_end/front_end.dart' hide FileSystemException;
+import 'package:front_end/src/fasta/command_line_reporting.dart';
+import 'package:kernel/target/targets.dart';
+
+main(List<String> args) async {

[Siggi Cherem (dart-lang), 2017/08/21 20:16:49]

I'm still contemplating whether to put this here or in a new `bazel_integration`
package. I personally prefer the latter, but we can debate further if you prefer
(I'm also OK making it a TODO for now).

[jakemac, 2017/08/21 21:16:37]

I don't feel that strongly, but analyzer/ddc both natively support bazel in
their main binary, and it is useful (pub uses the bazel worker protocol for
ddc/analyzer as well).

Since there isn't really a canonical entrypoint for front_end (that I could
find) I just created this one but ideally it would be merged into some more
general one at some point?

+  args = preprocessArgs(args);
+
+  if (args.contains('--persistent_worker')) {
+    if (args.length != 1) {
+      throw new StateError(
+          "unexpected args, expected only --persistent-worker but got: $args");
+    }
+    await new SummaryWorker().run();
+  } else {
+    await computeSummary(args);
+  }
+}
+
+/// A bazel worker loop that can compute summaries.
+class SummaryWorker extends AsyncWorkerLoop {
+  Future<WorkResponse> performRequest(WorkRequest request) async {
+    var outputBuffer = new StringBuffer();
+    var response = new WorkResponse()..exitCode = 0;
+    try {
+      await computeSummary(request.arguments,
+          isWorker: true, outputBuffer: outputBuffer);
+    } catch (_, s) {
+      outputBuffer.writeln(s);
+      response.exitCode = 15;
+    }
+    response.output = outputBuffer.toString();
+    return response;
+  }
+}
+
+/// If the last arg starts with `@`, this reads the file it points to and treats
+/// each line as an additional arg.
+///
+/// This is how individual work request args are differentiated from startup
+/// args in bazel (inidividual work request args go in that file).
+List<String> preprocessArgs(List<String> args) {
+  args = new List.from(args);
+  if (args.isEmpty) {
+    return args;
+  }
+  String lastArg = args.last;
+  if (lastArg.startsWith('@')) {
+    File argsFile = new File(lastArg.substring(1));
+    try {
+      args.removeLast();
+      args.addAll(argsFile.readAsLinesSync());
+    } on FileSystemException catch (e) {
+      throw new Exception('Failed to read file specified by $lastArg : $e');
+    }
+  }
+  return args;
+}
+
+/// An [ArgParser] for generating kernel summaries.
+final summaryArgsParser = new ArgParser()
+  ..addOption('dart-sdk-summary')
+  ..addOption('input-summary', allowMultiple: true)
+  ..addOption('multi-root', allowMultiple: true)
+  ..addOption('packages-file')
+  ..addOption('source', allowMultiple: true)
+  ..addOption('output');
+
+/// Computes a kernel summary based on [args].
+///
+/// If [isWorker] is true then exit codes will not be set on failure.
+///
+/// If [outputBuffer] is provided then messages will be written to that buffer
+/// instead of printed to the console.
+Future computeSummary(List<String> args,
+    {bool isWorker: false, StringBuffer outputBuffer}) async {
+  var parsedArgs = summaryArgsParser.parse(args);
+  var options = new CompilerOptions()
+    ..packagesFileUri = Uri.parse(parsedArgs['packages-file'])
+    ..inputSummaries = parsedArgs['input-summary'].map(Uri.parse).toList()
+    ..sdkSummary = Uri.parse(parsedArgs['dart-sdk-summary'])
+    ..multiRoots = parsedArgs['multi-root'].map(Uri.parse).toList()
+    ..target = new NoneTarget(new TargetFlags());
+
+  if (isWorker) {
+    options.setExitCodeOnProblem = false;

[Siggi Cherem (dart-lang), 2017/08/21 20:16:49]

this is already false by default :)

In your case I recommend to not use this option entirely, instead just rely on
the onError handler below like you do in the worker case.

[jakemac, 2017/08/21 21:16:37]

done

+  }
+
+  if (outputBuffer != null) {
+    options.onError = (CompilationMessage error) {
+      var severityString = severityName(error.severity, capitalized: true);
+      outputBuffer.writeln('$severityString: ${error.message}');
+      if (error.severity != Severity.nit) {
+        throw error;

[Siggi Cherem (dart-lang), 2017/08/21 20:16:49]

`throw` might not be the most user-friendly response when using this as a bazel
aspect. How about using an onError handler that records all errors and sets an
exit code instead?

Same below for the `else` case (instead of throwOnWarnigns)

[jakemac, 2017/08/21 21:16:37]

The bazel worker handles errors and sends a proper response etc automatically.

+      }
+    };
+  } else {
+    options.throwOnWarnings = true;

[Siggi Cherem (dart-lang), 2017/08/21 20:16:49]

our default error reporting throws and prints to the console. The printing tries
to use colors, but if you want to tune this further, you'd need to use the
onError callabck.

[jakemac, 2017/08/21 21:16:37]

I think the default handling is fine

+  }
+
+  var sources = parsedArgs['source'].map(Uri.parse).toList();

[Siggi Cherem (dart-lang), 2017/08/21 20:16:49]

we intend to fix this, but we've had issues in the past when some URis don't
have a scheme. I believe it is not an issue with summaries, but it can be an
issue with the sdk-root and source files. In that case, you may want to use
(Uri.base.resolve) here.

I have a feeling thought that you are always passing Uris as arguments anyways
(package:* or multi-root:///*), correct? If so, feel free to ignore this.

[jakemac, 2017/08/21 21:16:37]

These will be real uris with schemes yes

+  var program = await summaryFor(sources, options);
+
+  var outputFile = new File(parsedArgs['output']);
+  outputFile.createSync(recursive: true);
+  outputFile.writeAsBytesSync(program);
+}