OLD | NEW |
1 # The dart2js compiler | 1 # The dart2js compiler |
2 | 2 |
3 Welcome to the sources of the dart2js compiler! | 3 Welcome to the sources of the dart2js compiler! |
4 | 4 |
5 ## Architecture | 5 ## Architecture |
6 | 6 |
7 The compiler is currently undergoing a long refactoring process. As you navigate | 7 The compiler is currently undergoing a long refactoring process. As you navigate |
8 this code you may find it helpful to understand how the compiler used to be, | 8 this code you may find it helpful to understand how the compiler used to be, |
9 where it is going, and where it is today. | 9 where it is going, and where it is today. |
10 | 10 |
11 ### The near future architecture | 11 ### The near future architecture |
12 | 12 |
13 The compiler will operate in these general phases: | 13 The compiler will operate in these general phases: |
14 | 14 |
15 1. **load kernel**: Load all the code as kernel | 15 1. **load kernel**: Load all the code as kernel |
16 * Collect dart sources transtively | 16 * Collect dart sources transtively |
17 * Convert to kernel AST | 17 * Convert to kernel AST |
18 | 18 |
19 (this is will handled by invoking the front-end package) | 19 (this will be handled by invoking the front-end package) |
20 | 20 |
21 Alternatively, the compiler can start compilation directly from a kernel | 21 Alternatively, the compiler can start compilation directly from kernel files. |
22 file(s). | |
23 | 22 |
24 2. **model**: Create a Dart model of the program | 23 2. **model**: Create a Dart model of the program |
25 * The kernel ASTs could be used as a model, so this might be a no-op or jus
t | 24 * The kernel ASTs could be used as a model, so this might be a no-op or jus
t |
26 creating a thin wrapper on top of kernel. | 25 creating a thin wrapper on top of kernel. |
27 | 26 |
28 3. **tree-shake and create world**: Build world of reachable code | 27 3. **tree-shake and create world**: Build world of reachable code |
29 * For each reachable piece of code: | 28 * For each reachable piece of code: |
30 * Compute impact (i1) from kernel AST | 29 * Compute impact (i1) from kernel AST |
31 * Build a closed world (w1) | 30 * Build a closed world (w1) |
32 | 31 |
(...skipping 166 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
199 growing, in the past the term *universe* was used to describe this growing | 198 growing, in the past the term *universe* was used to describe this growing |
200 world. While the term is not completely deleted from the codebase, a lot of | 199 world. While the term is not completely deleted from the codebase, a lot of |
201 progress has been made to rename *universe* into *world builders*. | 200 progress has been made to rename *universe* into *world builders*. |
202 | 201 |
203 * **model**: there are many models in the compiler: | 202 * **model**: there are many models in the compiler: |
204 | 203 |
205 * **element model**: this is an abstraction describing the elements seen in | 204 * **element model**: this is an abstraction describing the elements seen in |
206 Dart programs, like "libraries", "classes", "methods", etc. | 205 Dart programs, like "libraries", "classes", "methods", etc. |
207 | 206 |
208 * **entity model**: also describes elements seen in Dart programs, but it is | 207 * **entity model**: also describes elements seen in Dart programs, but it is |
209 meant to be minimalistic and a super-hierarcy above the *element models*. | 208 meant to be minimalistic and a super-hierarchy above the *element models*. |
210 This is a newer addition, is an added abstraction to make it possible to | 209 This is a newer addition, is an added abstraction to make it possible to |
211 refactor our code from our old frontend to the kernel frontend. | 210 refactor our code from our old frontend to the kernel frontend. |
212 | 211 |
213 * **Dart vs JS models**: the compiler in the past had a single model to | 212 * **Dart vs JS models**: the compiler in the past had a single model to |
214 describe elements in the source and elements that were being compiled. In | 213 describe elements in the source and elements that were being compiled. In |
215 the future we plan to have two. Both input model and output models will be | 214 the future we plan to have two. Both input model and output models will be |
216 implementations of the *entity model*. The JS model is intended to have | 215 implementations of the *entity model*. The JS model is intended to have |
217 concepts specific about generating code in JS (like constructor-bodies as | 216 concepts specific about generating code in JS (like constructor-bodies as |
218 a separate entity than the constructor, closure classes, etc). | 217 a separate entity than the constructor, closure classes, etc). |
219 | 218 |
220 * **emitter model**: this is a model just used for dumping out the structure | 219 * **emitter model**: this is a model just used for dumping out the structure |
221 of the program in a .js text file. It doesn't have enough semantic meaning | 220 of the program in a .js text file. It doesn't have enough semantic meaning |
222 to be a JS model for compilation at this moment. | 221 to be a JS model for compilation at this moment. |
223 | 222 |
224 * **enqueuer**: a work-queue used to achieve tree-shaking (or more precisely | 223 * **enqueuer**: a work-queue used to achieve tree-shaking (or more precisely |
225 tree-growing): elements are added to the enqueuer as we recognize that they | 224 tree-growing): elements are added to the enqueuer as we recognize that they |
226 are needed in a given application. Note that we even track how elements are | 225 are needed in a given application. Note that we even track how elements are |
227 used, since some ways of using an element require more code than others. | 226 used, since some ways of using an element require more code than others. |
228 | 227 |
229 ### Code layout | 228 ### Code layout |
230 | 229 |
231 Here are some details of our current code layout and what's on each file. This | 230 Here are some details of our current code layout and what's in each file. This |
232 list also includes some action items (labeled AI below), which are mainly | 231 list also includes some action items (labeled AI below), which are mainly |
233 cleanup tasks that we have been discussing for a while: | 232 cleanup tasks that we have been discussing for a while: |
234 | 233 |
235 **bin folder**: some experimental command-line entrypoints, these need to be | 234 **bin folder**: some experimental command-line entrypoints, these need to be |
236 revisited | 235 revisited |
237 | 236 |
238 * `bin/dart2js.dart`: is a dart2js entry point, not used today other than | 237 * `bin/dart2js.dart`: is a dart2js entry point, not used today other than |
239 locally for development, most of our tools launch dart2js from | 238 locally for development, most of our tools launch dart2js from |
240 `lib/src/dart2js.dart` instead. | 239 `lib/src/dart2js.dart` instead. |
241 | 240 |
242 AI: change how we build the SDK to launch dart2js from here, most logic might | 241 AI: change how we build the SDK to launch dart2js from here, most logic might |
243 remain inside `lib/src/dart2js.dart` for testing purposes. | 242 remain inside `lib/src/dart2js.dart` for testing purposes. |
244 | 243 |
245 * `bin/resolver.dart`: an experimental binary we used to run the resolver and | |
246 serializer. As we are moving to work on top of kernel this is deprecated and | |
247 should be deleted. | |
248 | |
249 AI: delete this file. | |
250 | |
251 **lib folder**: API to use dart2js as a library. This is used by our | 244 **lib folder**: API to use dart2js as a library. This is used by our |
252 command-line tool to launch dart2js, but also by pub to invoke dart2js as a | 245 command-line tool to launch dart2js, but also by pub to invoke dart2js as a |
253 library during `pub-build` and `pub-serve`. | 246 library during `pub-build` and `pub-serve`. |
254 | 247 |
255 * `lib/compiler_new.dart`: the current API. This API is used by our command-line | 248 * `lib/compiler_new.dart`: the current API. This API is used by our command-line |
256 tool to spawn the dart2js compiler. This API (and everything that is | 249 tool to spawn the dart2js compiler. This API (and everything that is |
257 transitively created from it) has no dependencies on `dart:io` so that the | 250 transitively created from it) has no dependencies on `dart:io` so that the |
258 compiler can be used in contexts where `dart:io` is not available (e.g. | 251 compiler can be used in contexts where `dart:io` is not available (e.g. |
259 running in a browser worker) or where `dart:io` is not used explicitly (e.g. | 252 running in a browser worker) or where `dart:io` is not used explicitly (e.g. |
260 running as a pub transformer). | 253 running as a pub transformer). |
(...skipping 190 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
451 * `lib/src/mirrors_used.dart`: task that analyzes `@MirrorsUsed` annotations, | 444 * `lib/src/mirrors_used.dart`: task that analyzes `@MirrorsUsed` annotations, |
452 which let the compiler continue to do tree-shaking even when code is used via | 445 which let the compiler continue to do tree-shaking even when code is used via |
453 `dart:mirrors`. | 446 `dart:mirrors`. |
454 | 447 |
455 * Input/output: the compiler is designed to avoid all dependencies on dart:io. | 448 * Input/output: the compiler is designed to avoid all dependencies on dart:io. |
456 Most data is consumed and emitted via provider APIs. | 449 Most data is consumed and emitted via provider APIs. |
457 | 450 |
458 * `lib/src/compiler_new.dart`: defines the interface of these providers (see | 451 * `lib/src/compiler_new.dart`: defines the interface of these providers (see |
459 `CompilerInput` and `CompilerOutput`). | 452 `CompilerInput` and `CompilerOutput`). |
460 | 453 |
461 * `lib/src/null_compiler_output.dart`: an `CompilerOutput` that discards all | 454 * `lib/src/null_compiler_output.dart`: a `CompilerOutput` that discards all |
462 data written to it (name derives from /dev/null). | 455 data written to it (name derives from /dev/null). |
463 | 456 |
464 * `lib/src/source_file_provider.dart`: _TODO: add details_. | 457 * `lib/src/source_file_provider.dart`: _TODO: add details_. |
465 | 458 |
466 * Parsing: most of the parsing logic is now in the `front_end` package, | 459 * Parsing: most of the parsing logic is now in the `front_end` package, |
467 currently under `pkg/front_end/lib/src/fasta/scanner` and | 460 currently under `pkg/front_end/lib/src/fasta/scanner` and |
468 `pkg/front_end/lib/src/fasta/parser`. The `front_end` parser is AST agnostic | 461 `pkg/front_end/lib/src/fasta/parser`. The `front_end` parser is AST agnostic |
469 and uses listeners to create on the side what they want as the result of | 462 and uses listeners to create on the side what they want as the result of |
470 parsing. The logic to create dart2js' ASTs is defined in listeners within the | 463 parsing. The logic to create dart2js' ASTs is defined in listeners within the |
471 compiler package: | 464 compiler package: |
(...skipping 84 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
556 * `invoke_dynamic_specializers.dart` | 549 * `invoke_dynamic_specializers.dart` |
557 * `builder.dart` | 550 * `builder.dart` |
558 * `ssa_branch_builder.dart` | 551 * `ssa_branch_builder.dart` |
559 * `value_range_analyzer.dart` | 552 * `value_range_analyzer.dart` |
560 * `ssa_tracer.dart` | 553 * `ssa_tracer.dart` |
561 * `loop_handler.dart` | 554 * `loop_handler.dart` |
562 | 555 |
563 * `tool`: some helper scripts, some of these could be deleted | 556 * `tool`: some helper scripts, some of these could be deleted |
564 | 557 |
565 * `tool/perf.dart`: used by our benchmark runners to measure performance of | 558 * `tool/perf.dart`: used by our benchmark runners to measure performance of |
566 some frontend pieces of dart2js. We shuld be able to delete it in the near | 559 some frontend pieces of dart2js. We should be able to delete it in the near |
567 future once the front end code is moved into `fasta`. | 560 future once the front end code is moved into `fasta`. |
568 | 561 |
569 * `tool/perf_test.dart`: small test to ensure we don't break `perf.dart`. | 562 * `tool/perf_test.dart`: small test to ensure we don't break `perf.dart`. |
570 | 563 |
571 * `tool/track_memory.dart`: a helper script to see memory usage of dart2js | 564 * `tool/track_memory.dart`: a helper script to see memory usage of dart2js |
572 while it's running. Used in the past to profile the global analysis phases | 565 while it's running. Used in the past to profile the global analysis phases |
573 when run on very large apps. | 566 when run on very large apps. |
574 | 567 |
575 * `tool/dart2js_stress.dart` and `tool/dart2js_profile_many.dart`: other | 568 * `tool/dart2js_stress.dart` and `tool/dart2js_profile_many.dart`: other |
576 helper wrappers to make it easier to profile dart2js with Observatory. | 569 helper wrappers to make it easier to profile dart2js with Observatory. |
(...skipping 236 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
813 `lib/src/types/forwarding_type_mask.dart` | 806 `lib/src/types/forwarding_type_mask.dart` |
814 `lib/src/types/container_type_mask.dart` | 807 `lib/src/types/container_type_mask.dart` |
815 `lib/src/types/constants.dart` | 808 `lib/src/types/constants.dart` |
816 `lib/src/types/flat_type_mask.dart` | 809 `lib/src/types/flat_type_mask.dart` |
817 `lib/src/types/masks.dart` | 810 `lib/src/types/masks.dart` |
818 `lib/src/types/value_type_mask.dart` | 811 `lib/src/types/value_type_mask.dart` |
819 `lib/src/types/union_type_mask.dart` | 812 `lib/src/types/union_type_mask.dart` |
820 | 813 |
821 `lib/src/hash` | 814 `lib/src/hash` |
822 `lib/src/hash/sha1.dart` | 815 `lib/src/hash/sha1.dart` |
OLD | NEW |