Initial API for the Dart front_end package.

pkg/front_end/lib/options.dart

Index: pkg/front_end/lib/options.dart
diff --git a/pkg/front_end/lib/options.dart b/pkg/front_end/lib/options.dart
new file mode 100644
index 0000000000000000000000000000000000000000..e7f48f32579b6e5348a35d11cd6e10d25891fe80
--- /dev/null
+++ b/pkg/front_end/lib/options.dart
@@ -0,0 +1,79 @@
+// Copyright (c) 2016, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+library front_end.options;
+
+import 'compilation_error.dart';
+
+/// Callback used to report errors encountered during compilation.
+typedef void ErrorHandler(CompilationError error);
+
+/// Front-end Analysis options relevant to compiler back ends.

[Brian Wilkerson, 2016/10/17 15:08:14]

nit: Remove "Analysis"? It doesn't appear to be adding anything.

[Paul Berry, 2016/10/17 17:04:27]

Done.

+///
+/// TODO(paulberry): add a mechanism to allow file system operations to be
+/// stubbed out for testing.
+class Options {

[Brian Wilkerson, 2016/10/17 15:08:14]

"Options" is really generic. I doubt that you want to add all possible options
for all of the API's that the front end will eventually support to one class, so
consider qualifying the name.

[Paul Berry, 2016/10/17 17:04:27]

Ok, renamed to "CompilerOptions".

+  /// The path to the Dart SDK.
+  ///
+  /// If `null`, the SDK will be searched for using
+  /// [Platform.resolvedExecutable] as a starting point.
+  String sdkPath;
+
+  /// Callback to which compilation errors should be delivered.
+  ///
+  /// If `null`, the first error will be reported by throwing an exception of
+  /// type [CompilationError].
+  ErrorHandler onError;
+
+  /// Path to the ".packages" file.
+  ///
+  /// If `null`, the ".packages" file will be found via the standard
+  /// package_config search algorithm.
+  String packagesFilePath;
+
+  /// Paths to the input summary files (excluding the SDK summary).
+  List<String> inputSummaries = [];

[Brian Wilkerson, 2016/10/17 15:08:14]

Is there a strong connection between inputSummaries and packagesFilePath? I'm
assuming that the summary files in this list are like the sdk summary file in
that they don't contain any information about the bodies of functions. If that's
true, then we need to be able to locate the source associated with any given
library. Is that done via the .packages file (for non-bazel packages) and
bazelRoots (for bazel packages)?

If so, I think this needs to be made explicit. If not, then how are sources for
file in summaries found?

[Siggi Cherem (dart-lang), 2016/10/17 16:27:29]

I don't believe there should be any connection.
Correct.
Yes - summaries will have a resolved URL of the library (either a package:url or
a file:/ url for which we know where to locate on disk (either via the
uriOverrides or via the bazel-root mechanism below).
Would it be fair to say that summaries can contain a URI for the sources where
they are derived from and that these URIs can be resolved using the same
resolution mechanism that we would use if that URI was imported in the code?

[Paul Berry, 2016/10/17 17:04:27]

Yes, that's true.  Although in practice I'm not sure how important this is.  In
the modular compilation case, the compiler isn't going to need to look at the
sources corresponding to a summary, because the information in the summary file
will be sufficient.  In the whole program compilation case, we're not going to
gain much benefit from using summaries anyhow.

I'm leaving the comment as is for now.  If it's still unclear, I'm happy to
discuss in person and we can decide exactly what we want to say in the comment.

[Brian Wilkerson, 2016/10/17 17:09:16]

So, the packagesFilePath is used to resolve package: URI's found in source code
(unless a uriOverride is provided), but not those in summaries? Are
packageFilePath and uriOverride mutually exclusive?
Given that we don't know, at the time we're producing a summary, where the file
containing that summary will be located, all of the URI's in a summary file
would need to be absolute. Given that all of the URI's in a summary file are
absolute, then, yes, I think that would work.

I still think this should be carefully explained in the docs, because it isn't
obvious to everyone.

[Paul Berry, 2016/10/17 17:32:06]

Correct.
No, packageFilePath and uriOverride are not mutually exclusive.  One overrides
the other.

Here's the resolution process as I understand it in pseudocode:

resolveUriToLibrary(uri) {
  if (any summary refers to uri) {
    return resynthesizeLibraryFromSummary(uri);
  }
  String filePath;
  if (uriOverride.containsKey(uri)) {
    filePath = uriOverride[uri];
  } else {
    if (uri.scheme == 'package') {
      uri = resolve package URI using packagesFilePath;
    }
    assert(uri.scheme == 'file');
    filePath = uri.path; // Neglecting Windows subtleties
    if (filePath starts with "/bazel-root/") {
      String pathRelativeToRoot = remainder of filePath after "/bazel-root/";
      for (bazelRoot in bazelRoots) {
        String candidatePath = path.join(bazelRoot, pathRelativeToRoot);
        if (a file exists at candidatePath) {
          filePath = candidatePath;
          break;
        }
      }
    }
  }
  return createElementModelForLibrary(filePath);
}
Would you mind sitting with me at my desk and helping me craft the necessary
documentation?  I want to make sure I write something that will adequately
explain the situation, and I suspect the most efficient way to do that would be
to do it "pair programming" style, in person, so that we can reduce the amount
of back and forth review.

[Siggi Cherem (dart-lang), 2016/10/17 18:25:31]

Thanks for writing the pseudocode, it reminds me that the use of URIs in
summaries is even more limited than I thought: we don't even try to resolve it
to a disk location. It's mainly a lookup to check whether or not an (unresolved)
URI is covered by a specific summary.
Lets also consider whether all this information needs to be in this frontend API
or documented to somewhere else (e.g. design docs for summaries?). In
particular, I'd only include here what the user of this API needs to know or
provide.

[Paul Berry, 2016/10/17 18:38:43]

Acknowledged.  In the interest of trying to get this CL landed, I'd prefer to
leave things documented here for now.  If/when we publish an official design doc
for summaries, I'm happy to change the comments here to point to the published
doc.

+
+  /// Path to the SDK summary file.
+  ///
+  /// If `null`, the SDK summary will be searched for at a default location
+  /// within [sdkPath].
+  String sdkSummary;

[Brian Wilkerson, 2016/10/17 15:08:14]

Perhaps mention that it is critical that the sdkSummary (if set) is consistent
with the contents of the SDK in sdkPath.

Given the tight coupling, should this even be an option? We will always find the
SDK sources; do we really need to allow a summary file outside the SDK to be
used?

[Siggi Cherem (dart-lang), 2016/10/17 16:27:29]

I think this option is important to have:
 - it helps on certain build systems to be able to store the sdk-summary file in
a different location than the SDK itself.
 - for modular compilation we'll use an sdk summary and generate kernel that
refer to sdk symbols but will not include the kernel code for the sdk itself. In
those cases I'd like to only specify where to find this file.

In the past, when integrating with bazel, we have had to create a fake sdk
folder structure just to be able to load the sdk summaries. I'd like to avoid
that in the future.

[Paul Berry, 2016/10/17 17:04:27]

Actually this is not critical.  As with the situation above, in the modular
compilation case, if the sdk summary is set, then the SDK source files won't
need to be used.  In the whole program compilation case, the summaries are of
little benefit.
In addition to the reasons Siggi stated, we need this to cover the case where
the version of the front end being used does not match the version published in
the SDK; in that case we will need to use the front end to create its own
summary of the SDK which is independent of the summary stored in the SDK itself;
then we will need to pass this summary into dependent build steps.

[Brian Wilkerson, 2016/10/17 17:09:16]

Are we planning on shipping a version of the SDK that doesn't include a summary?
If there's a reason why we can't get the SDK summary from the SDK, then it makes
sense, but I don't understand why would we be unable to get the summary from the
SDK. Sounds like we should talk.
Yeah, that sounds pretty awful. I don't understand why that was necessary,
though.

[Brian Wilkerson, 2016/10/17 17:20:44]

It sounds like the two are mutually exclusive. Is there a case where we need
both to be set? If not, then the mutual exclusivity of the two options should be
documented.

[Paul Berry, 2016/10/17 17:32:06]

Some of these are Google-internal use cases.  Let's discuss in person.

+
+  /// URI override map.
+  ///
+  /// This is a map from Uri to file path which overrides the normal URI
+  /// resolution algorithm.  If not set, the normal URI resolution algorithm
+  /// will always be used.
+  ///
+  /// TODO(paulberry): transition analyzer and dev_compiler to use the
+  /// "file:///bazel-root" mechanism, and then remove this.

[Brian Wilkerson, 2016/10/17 15:08:14]

Is this mechanism one that you created, or is this in use elsewhere? (Perhaps
best discussed off-line.)

[Siggi Cherem (dart-lang), 2016/10/17 16:27:29]

This is a mechanism we introduced in dart2js when integrating our modular mode
with bazel. I'm happy to share it more details offline, but it's well summarized
in the bazelRoots dart doc below.

[Brian Wilkerson, 2016/10/17 17:09:16]

Yes, I would like to get more info. Thanks!

Analyzer has a different way of dealing with this, so I'd like to understand the
trade-offs of the two approaches.

[Paul Berry, 2016/10/17 17:32:06]

Since we already have some discussion to do, I'm happy to explain this.

[Siggi Cherem (dart-lang), 2016/10/17 18:25:31]

Sounds good - I'm happy to join on VC if you like. Just let me know.

+  @deprecated
+  Map<Uri, String> uriOverride = {};
+
+  /// Bazel roots.
+  ///
+  /// Any Uri that resolves to "file:///bazel-root/$rest" will be searched for
+  /// at "$root/$rest" ("$root\\$rest" in Windows), where "$root" is drawn from
+  /// this list.  If the file is not found at any of those locations, the URI
+  /// "file:///bazel-root/$rest" will be used directly.
+  ///
+  /// Intended use: if the Bazel workspace is located at path "$workspace", this
+  /// could be set to `['$workspace', '$workspace/bazel-bin',
+  /// '$workspace/bazel-genfiles']`, effectively overlaying source and generated
+  /// files.
+  List<String> bazelRoots = [];
+
+  /// Sets the platform bit, which determines which patch files should be
+  /// applied to the SDK.
+  ///
+  /// The value should be a power of two, and should match the `PLATFORM` bit
+  /// flags in sdk/lib/_internal/sdk_library_metadata/lib/libraries.dart.  If
+  /// zero, no patch files will be applied.
+  int platformBit;
+
+  /// The declared variables for use by configurable imports and constant
+  /// evaluation.
+  Map<String, String> declaredVariables;
+}