| OLD | NEW |
| 1 # The dart2js compiler | 1 # The dart2js compiler |
| 2 | 2 |
| 3 Welcome to the sources of the dart2js compiler! | 3 Welcome to the sources of the dart2js compiler! |
| 4 | 4 |
| 5 ## Architecture | 5 ## Architecture |
| 6 | 6 |
| 7 The compiler is currently undergoing a long refactoring process. As you navigate | 7 The compiler is currently undergoing a long refactoring process. As you navigate |
| 8 this code you may find it helpful to understand how the compiler used to be, | 8 this code you may find it helpful to understand how the compiler used to be, |
| 9 where it is going, and where it is today. | 9 where it is going, and where it is today. |
| 10 | 10 |
| 11 ### The near future architecture | 11 ### The near future architecture |
| 12 | 12 |
| 13 The compiler will operate in these general phases: | 13 The compiler will operate in these general phases: |
| 14 | 14 |
| 15 1. **load kernel**: Load all the code as kernel | 15 1. **load kernel**: Load all the code as kernel |
| 16 * Collect dart sources transtively | 16 * Collect dart sources transtively |
| 17 * Convert to kernel AST | 17 * Convert to kernel AST |
| 18 | 18 |
| 19 (this is will handled by invoking the front-end package) | 19 (this will be handled by invoking the front-end package) |
| 20 | 20 |
| 21 Alternatively, the compiler can start compilation directly from a kernel | 21 Alternatively, the compiler can start compilation directly from kernel files. |
| 22 file(s). | |
| 23 | 22 |
| 24 2. **model**: Create a Dart model of the program | 23 2. **model**: Create a Dart model of the program |
| 25 * The kernel ASTs could be used as a model, so this might be a no-op or jus
t | 24 * The kernel ASTs could be used as a model, so this might be a no-op or jus
t |
| 26 creating a thin wrapper on top of kernel. | 25 creating a thin wrapper on top of kernel. |
| 27 | 26 |
| 28 3. **tree-shake and create world**: Build world of reachable code | 27 3. **tree-shake and create world**: Build world of reachable code |
| 29 * For each reachable piece of code: | 28 * For each reachable piece of code: |
| 30 * Compute impact (i1) from kernel AST | 29 * Compute impact (i1) from kernel AST |
| 31 * Build a closed world (w1) | 30 * Build a closed world (w1) |
| 32 | 31 |
| (...skipping 166 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 199 growing, in the past the term *universe* was used to describe this growing | 198 growing, in the past the term *universe* was used to describe this growing |
| 200 world. While the term is not completely deleted from the codebase, a lot of | 199 world. While the term is not completely deleted from the codebase, a lot of |
| 201 progress has been made to rename *universe* into *world builders*. | 200 progress has been made to rename *universe* into *world builders*. |
| 202 | 201 |
| 203 * **model**: there are many models in the compiler: | 202 * **model**: there are many models in the compiler: |
| 204 | 203 |
| 205 * **element model**: this is an abstraction describing the elements seen in | 204 * **element model**: this is an abstraction describing the elements seen in |
| 206 Dart programs, like "libraries", "classes", "methods", etc. | 205 Dart programs, like "libraries", "classes", "methods", etc. |
| 207 | 206 |
| 208 * **entity model**: also describes elements seen in Dart programs, but it is | 207 * **entity model**: also describes elements seen in Dart programs, but it is |
| 209 meant to be minimalistic and a super-hierarcy above the *element models*. | 208 meant to be minimalistic and a super-hierarchy above the *element models*. |
| 210 This is a newer addition, is an added abstraction to make it possible to | 209 This is a newer addition, is an added abstraction to make it possible to |
| 211 refactor our code from our old frontend to the kernel frontend. | 210 refactor our code from our old frontend to the kernel frontend. |
| 212 | 211 |
| 213 * **Dart vs JS models**: the compiler in the past had a single model to | 212 * **Dart vs JS models**: the compiler in the past had a single model to |
| 214 describe elements in the source and elements that were being compiled. In | 213 describe elements in the source and elements that were being compiled. In |
| 215 the future we plan to have two. Both input model and output models will be | 214 the future we plan to have two. Both input model and output models will be |
| 216 implementations of the *entity model*. The JS model is intended to have | 215 implementations of the *entity model*. The JS model is intended to have |
| 217 concepts specific about generating code in JS (like constructor-bodies as | 216 concepts specific about generating code in JS (like constructor-bodies as |
| 218 a separate entity than the constructor, closure classes, etc). | 217 a separate entity than the constructor, closure classes, etc). |
| 219 | 218 |
| 220 * **emitter model**: this is a model just used for dumping out the structure | 219 * **emitter model**: this is a model just used for dumping out the structure |
| 221 of the program in a .js text file. It doesn't have enough semantic meaning | 220 of the program in a .js text file. It doesn't have enough semantic meaning |
| 222 to be a JS model for compilation at this moment. | 221 to be a JS model for compilation at this moment. |
| 223 | 222 |
| 224 * **enqueuer**: a work-queue used to achieve tree-shaking (or more precisely | 223 * **enqueuer**: a work-queue used to achieve tree-shaking (or more precisely |
| 225 tree-growing): elements are added to the enqueuer as we recognize that they | 224 tree-growing): elements are added to the enqueuer as we recognize that they |
| 226 are needed in a given application. Note that we even track how elements are | 225 are needed in a given application. Note that we even track how elements are |
| 227 used, since some ways of using an element require more code than others. | 226 used, since some ways of using an element require more code than others. |
| 228 | 227 |
| 229 ### Code layout | 228 ### Code layout |
| 230 | 229 |
| 231 Here are some details of our current code layout and what's on each file. This | 230 Here are some details of our current code layout and what's in each file. This |
| 232 list also includes some action items (labeled AI below), which are mainly | 231 list also includes some action items (labeled AI below), which are mainly |
| 233 cleanup tasks that we have been discussing for a while: | 232 cleanup tasks that we have been discussing for a while: |
| 234 | 233 |
| 235 **bin folder**: some experimental command-line entrypoints, these need to be | 234 **bin folder**: some experimental command-line entrypoints, these need to be |
| 236 revisited | 235 revisited |
| 237 | 236 |
| 238 * `bin/dart2js.dart`: is a dart2js entry point, not used today other than | 237 * `bin/dart2js.dart`: is a dart2js entry point, not used today other than |
| 239 locally for development, most of our tools launch dart2js from | 238 locally for development, most of our tools launch dart2js from |
| 240 `lib/src/dart2js.dart` instead. | 239 `lib/src/dart2js.dart` instead. |
| 241 | 240 |
| 242 AI: change how we build the SDK to launch dart2js from here, most logic might | 241 AI: change how we build the SDK to launch dart2js from here, most logic might |
| 243 remain inside `lib/src/dart2js.dart` for testing purposes. | 242 remain inside `lib/src/dart2js.dart` for testing purposes. |
| 244 | 243 |
| 245 * `bin/resolver.dart`: an experimental binary we used to run the resolver and | |
| 246 serializer. As we are moving to work on top of kernel this is deprecated and | |
| 247 should be deleted. | |
| 248 | |
| 249 AI: delete this file. | |
| 250 | |
| 251 **lib folder**: API to use dart2js as a library. This is used by our | 244 **lib folder**: API to use dart2js as a library. This is used by our |
| 252 command-line tool to launch dart2js, but also by pub to invoke dart2js as a | 245 command-line tool to launch dart2js, but also by pub to invoke dart2js as a |
| 253 library during `pub-build` and `pub-serve`. | 246 library during `pub-build` and `pub-serve`. |
| 254 | 247 |
| 255 * `lib/compiler_new.dart`: the current API. This API is used by our command-line | 248 * `lib/compiler_new.dart`: the current API. This API is used by our command-line |
| 256 tool to spawn the dart2js compiler. This API (and everything that is | 249 tool to spawn the dart2js compiler. This API (and everything that is |
| 257 transitively created from it) has no dependencies on `dart:io` so that the | 250 transitively created from it) has no dependencies on `dart:io` so that the |
| 258 compiler can be used in contexts where `dart:io` is not available (e.g. | 251 compiler can be used in contexts where `dart:io` is not available (e.g. |
| 259 running in a browser worker) or where `dart:io` is not used explicitly (e.g. | 252 running in a browser worker) or where `dart:io` is not used explicitly (e.g. |
| 260 running as a pub transformer). | 253 running as a pub transformer). |
| (...skipping 190 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 451 * `lib/src/mirrors_used.dart`: task that analyzes `@MirrorsUsed` annotations, | 444 * `lib/src/mirrors_used.dart`: task that analyzes `@MirrorsUsed` annotations, |
| 452 which let the compiler continue to do tree-shaking even when code is used via | 445 which let the compiler continue to do tree-shaking even when code is used via |
| 453 `dart:mirrors`. | 446 `dart:mirrors`. |
| 454 | 447 |
| 455 * Input/output: the compiler is designed to avoid all dependencies on dart:io. | 448 * Input/output: the compiler is designed to avoid all dependencies on dart:io. |
| 456 Most data is consumed and emitted via provider APIs. | 449 Most data is consumed and emitted via provider APIs. |
| 457 | 450 |
| 458 * `lib/src/compiler_new.dart`: defines the interface of these providers (see | 451 * `lib/src/compiler_new.dart`: defines the interface of these providers (see |
| 459 `CompilerInput` and `CompilerOutput`). | 452 `CompilerInput` and `CompilerOutput`). |
| 460 | 453 |
| 461 * `lib/src/null_compiler_output.dart`: an `CompilerOutput` that discards all | 454 * `lib/src/null_compiler_output.dart`: a `CompilerOutput` that discards all |
| 462 data written to it (name derives from /dev/null). | 455 data written to it (name derives from /dev/null). |
| 463 | 456 |
| 464 * `lib/src/source_file_provider.dart`: _TODO: add details_. | 457 * `lib/src/source_file_provider.dart`: _TODO: add details_. |
| 465 | 458 |
| 466 * Parsing: most of the parsing logic is now in the `front_end` package, | 459 * Parsing: most of the parsing logic is now in the `front_end` package, |
| 467 currently under `pkg/front_end/lib/src/fasta/scanner` and | 460 currently under `pkg/front_end/lib/src/fasta/scanner` and |
| 468 `pkg/front_end/lib/src/fasta/parser`. The `front_end` parser is AST agnostic | 461 `pkg/front_end/lib/src/fasta/parser`. The `front_end` parser is AST agnostic |
| 469 and uses listeners to create on the side what they want as the result of | 462 and uses listeners to create on the side what they want as the result of |
| 470 parsing. The logic to create dart2js' ASTs is defined in listeners within the | 463 parsing. The logic to create dart2js' ASTs is defined in listeners within the |
| 471 compiler package: | 464 compiler package: |
| (...skipping 84 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 556 * `invoke_dynamic_specializers.dart` | 549 * `invoke_dynamic_specializers.dart` |
| 557 * `builder.dart` | 550 * `builder.dart` |
| 558 * `ssa_branch_builder.dart` | 551 * `ssa_branch_builder.dart` |
| 559 * `value_range_analyzer.dart` | 552 * `value_range_analyzer.dart` |
| 560 * `ssa_tracer.dart` | 553 * `ssa_tracer.dart` |
| 561 * `loop_handler.dart` | 554 * `loop_handler.dart` |
| 562 | 555 |
| 563 * `tool`: some helper scripts, some of these could be deleted | 556 * `tool`: some helper scripts, some of these could be deleted |
| 564 | 557 |
| 565 * `tool/perf.dart`: used by our benchmark runners to measure performance of | 558 * `tool/perf.dart`: used by our benchmark runners to measure performance of |
| 566 some frontend pieces of dart2js. We shuld be able to delete it in the near | 559 some frontend pieces of dart2js. We should be able to delete it in the near |
| 567 future once the front end code is moved into `fasta`. | 560 future once the front end code is moved into `fasta`. |
| 568 | 561 |
| 569 * `tool/perf_test.dart`: small test to ensure we don't break `perf.dart`. | 562 * `tool/perf_test.dart`: small test to ensure we don't break `perf.dart`. |
| 570 | 563 |
| 571 * `tool/track_memory.dart`: a helper script to see memory usage of dart2js | 564 * `tool/track_memory.dart`: a helper script to see memory usage of dart2js |
| 572 while it's running. Used in the past to profile the global analysis phases | 565 while it's running. Used in the past to profile the global analysis phases |
| 573 when run on very large apps. | 566 when run on very large apps. |
| 574 | 567 |
| 575 * `tool/dart2js_stress.dart` and `tool/dart2js_profile_many.dart`: other | 568 * `tool/dart2js_stress.dart` and `tool/dart2js_profile_many.dart`: other |
| 576 helper wrappers to make it easier to profile dart2js with Observatory. | 569 helper wrappers to make it easier to profile dart2js with Observatory. |
| (...skipping 236 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 813 `lib/src/types/forwarding_type_mask.dart` | 806 `lib/src/types/forwarding_type_mask.dart` |
| 814 `lib/src/types/container_type_mask.dart` | 807 `lib/src/types/container_type_mask.dart` |
| 815 `lib/src/types/constants.dart` | 808 `lib/src/types/constants.dart` |
| 816 `lib/src/types/flat_type_mask.dart` | 809 `lib/src/types/flat_type_mask.dart` |
| 817 `lib/src/types/masks.dart` | 810 `lib/src/types/masks.dart` |
| 818 `lib/src/types/value_type_mask.dart` | 811 `lib/src/types/value_type_mask.dart` |
| 819 `lib/src/types/union_type_mask.dart` | 812 `lib/src/types/union_type_mask.dart` |
| 820 | 813 |
| 821 `lib/src/hash` | 814 `lib/src/hash` |
| 822 `lib/src/hash/sha1.dart` | 815 `lib/src/hash/sha1.dart` |
| OLD | NEW |