OLD | NEW |
1 // Copyright (c) 2012, the Dart project authors. Please see the AUTHORS file | 1 // Copyright (c) 2012, the Dart project authors. Please see the AUTHORS file |
2 // for details. All rights reserved. Use of this source code is governed by a | 2 // for details. All rights reserved. Use of this source code is governed by a |
3 // BSD-style license that can be found in the LICENSE file. | 3 // BSD-style license that can be found in the LICENSE file. |
4 | 4 |
5 library fasta.parser.parser; | 5 library fasta.parser.parser; |
6 | 6 |
7 import '../fasta_codes.dart' | 7 import '../fasta_codes.dart' |
8 show | 8 show |
9 FastaCode, | 9 FastaCode, |
10 FastaMessage, | 10 FastaMessage, |
(...skipping 164 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
175 /// [keyword.dart](../scanner/keyword.dart) and ensure the identifier is added | 175 /// [keyword.dart](../scanner/keyword.dart) and ensure the identifier is added |
176 /// to the keyword table. | 176 /// to the keyword table. |
177 /// | 177 /// |
178 /// As a consequence of this, one should not use `==` to compare strings in the | 178 /// As a consequence of this, one should not use `==` to compare strings in the |
179 /// parser. One should favor the methods [optional] and [expected] to recognize | 179 /// parser. One should favor the methods [optional] and [expected] to recognize |
180 /// keywords or identifiers. In some cases, it's possible to compare a token's | 180 /// keywords or identifiers. In some cases, it's possible to compare a token's |
181 /// `stringValue` using [identical], but normally [optional] will suffice. | 181 /// `stringValue` using [identical], but normally [optional] will suffice. |
182 /// | 182 /// |
183 /// Historically, we over-used identical, and when identical is used on other | 183 /// Historically, we over-used identical, and when identical is used on other |
184 /// objects than strings, it can often be replaced by `==`. | 184 /// objects than strings, it can often be replaced by `==`. |
| 185 /// |
| 186 /// ## Flexibility, Extensibility, and Specification |
| 187 /// |
| 188 /// The parser is designed to be flexible and extensible. Its methods are |
| 189 /// designed to be overridden in subclasses, so it can be extended to handle |
| 190 /// unspecified language extension or experiments while everything in this file |
| 191 /// attempts to follow the specification (unless when it interferes with error |
| 192 /// recovery). |
| 193 /// |
| 194 /// We achieve flexibily, extensible, and specification compliance by following |
| 195 /// a few rules-of-thumb: |
| 196 /// |
| 197 /// 1. All methods in the parser should be public. |
| 198 /// |
| 199 /// 2. The methods follow the specified grammar, and do not implement custom |
| 200 /// extensions, for example, `native`. |
| 201 /// |
| 202 /// 3. The parser doesn't rewrite the token stream (when dealing with `>>`). |
| 203 /// |
| 204 /// ### Implementing Extensions |
| 205 /// |
| 206 /// For various reasons, some Dart language implementations have used |
| 207 /// custom/unspecified extensions to the Dart grammar. Examples of this |
| 208 /// includes diet parsing, patch files, `native` keyword, and generic |
| 209 /// comments. This class isn't supposed to implement any of these |
| 210 /// features. Instead it provides hooks for those extensions to be implemented |
| 211 /// in subclasses or listeners. Let's examine how diet parsing and `native` |
| 212 /// keyword is currently supported by Fasta. |
| 213 /// |
| 214 /// #### Implementation of `native` Keyword |
| 215 /// |
| 216 /// Both dart2js and the Dart VM have used the `native` keyword to mark methods |
| 217 /// that couldn't be implemented in the Dart language and needed to be |
| 218 /// implemented in JavaScript or C++, respectively. An example of the syntax |
| 219 /// extension used by the Dart VM is: |
| 220 /// |
| 221 /// nativeFunction() native "NativeFunction"; |
| 222 /// |
| 223 /// When attempting to parse this function, the parser eventually calls |
| 224 /// [parseFunctionBody]. This method will report an unrecoverable error to the |
| 225 /// listener with the code [codeExpectedFunctionBody]. The listener can then |
| 226 /// look at the error code and the token and use the methods in |
| 227 /// [dart_vm_native.dart](dart_vm_native.dart) to parse the native syntax. |
| 228 /// |
| 229 /// #### Implementation of Diet Parsing |
| 230 /// |
| 231 /// We call it _diet_ _parsing_ when the parser skips parts of a file. Both |
| 232 /// dart2js and the Dart VM have been relying on this from early on as it allows |
| 233 /// them to more quickly compile small programs that use small parts of big |
| 234 /// libraries. It's also become an integrated part of how Fasta builds up |
| 235 /// outlines before starting to parse method bodies. |
| 236 /// |
| 237 /// When looking through this parser, you'll find a number of unused methods |
| 238 /// starting with `skip`. These methods are only used by subclasses, such as |
| 239 /// [ClassMemberParser](class_member_parser.dart) and |
| 240 /// [TopLevelParser](top_level_parser.dart). These methods violate the |
| 241 /// principle above about following the specified grammar, and originally lived |
| 242 /// in subclasses. However, we realized that these methods were so widely used |
| 243 /// and hard to maintain in subclasses, that it made sense to move them here. |
| 244 /// |
| 245 /// ### Specification and Error Recovery |
| 246 /// |
| 247 /// To improve error recovery, the parser will inform the listener of |
| 248 /// recoverable errors and continue to parse. An example of a recoverable |
| 249 /// error is: |
| 250 /// |
| 251 /// Error: Asynchronous for-loop can only be used in 'async' or 'async*'... |
| 252 /// main() { await for (var x in []) {} } |
| 253 /// ^^^^^ |
| 254 /// |
| 255 /// For unrecoverable errors, the parser will ask the listener for help to |
| 256 /// recover from the error. We haven't made much progress on these kinds of |
| 257 /// errors, so in most cases, the parser aborts by skipping to the end of file. |
| 258 /// |
| 259 /// Historically, this parser has been rather lax in what it allows, and |
| 260 /// deferred the enforcement of some syntactical rules to subsequent phases. It |
| 261 /// doesn't matter how we got there, only that we've identified that it's |
| 262 /// easier if the parser reports as many errors it can, but informs the |
| 263 /// listener if the error is recoverable or not. |
| 264 /// |
| 265 /// Currently, the parser is particularly lax when it comes to the order of |
| 266 /// modifiers such as `abstract`, `final`, `static`, etc. Historically, dart2js |
| 267 /// would handle such errors in later phases. We hope that these cases will go |
| 268 /// away as Fasta matures. |
185 class Parser { | 269 class Parser { |
186 final Listener listener; | 270 final Listener listener; |
187 | 271 |
188 Uri get uri => listener.uri; | 272 Uri get uri => listener.uri; |
189 | 273 |
190 bool mayParseFunctionExpressions = true; | 274 bool mayParseFunctionExpressions = true; |
191 | 275 |
192 bool parseGenericMethodComments = false; | 276 bool parseGenericMethodComments = false; |
193 | 277 |
194 /// Represents parser state: what asynchronous syntax is allowed in the | 278 /// Represents parser state: what asynchronous syntax is allowed in the |
(...skipping 3660 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
3855 previous.setNext(firstToken); | 3939 previous.setNext(firstToken); |
3856 beforeToken = firstToken; | 3940 beforeToken = firstToken; |
3857 } | 3941 } |
3858 } | 3942 } |
3859 | 3943 |
3860 typedef FastaMessage NoArgument(Uri uri, int charOffset); | 3944 typedef FastaMessage NoArgument(Uri uri, int charOffset); |
3861 | 3945 |
3862 typedef FastaMessage TokenArgument(Uri uri, int charOffset, Token token); | 3946 typedef FastaMessage TokenArgument(Uri uri, int charOffset, Token token); |
3863 | 3947 |
3864 typedef FastaMessage StringArgument(Uri uri, int charOffset, String string); | 3948 typedef FastaMessage StringArgument(Uri uri, int charOffset, String string); |
OLD | NEW |