Second step towards loading the core library scripts directly from the sources

Second step towards loading core library scripts directly from the sources
- Modify the library source generator to generate a source mapping array
  The generated array is of the following format:
  const char* dart::Bootstrap::corelib_source_paths_[] = {
    "dart:core", "/workspace1/dart-all/dart/sdk/lib/core/core.dart", 

    "bool.dart", "/workspace1/dart-all/dart/sdk/lib/core/bool.dart", 
    ...
    ...
  };

- Read the source file using the source mapping array instead of relying on
  a generated buffer containing the sources

- Modify the gyp files to ensure that all libraries are read directly
  from the sources. This CL does not change the patch part yet.
  Remove the source concatentation step in the gypi files for all the core
  libraries.

R=hausner@google.com, iposva@google.com

Committed: https://code.google.com/p/dart/source/detail?r=22380

[siva, 2013-05-01 22:09:56]

Anton, there is a small API change to Dart_Initialize and would require
co-ordination with you.

[hausner, 2013-05-01 23:07:30]

lgtm

[Ivan Posva, 2013-05-02 21:42:58]

Can we make it in such a way that the different source files have the effective
URI of for example dart:core/bool.dart to avoid conflicting URIs when looking at
the source URIs?

-Ivan

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/bin/run_vm...
File runtime/bin/run_vm_tests.cc (right):

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/bin/run_vm...
runtime/bin/run_vm_tests.cc:14: using dart::bin::DartUtils;
No using, please...

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/bin/run_vm...
runtime/bin/run_vm_tests.cc:110: DartUtils::OpenFile,
Do we need these here? Can we run the VM tests from the snapshot?

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/tools/gen_...
File runtime/tools/gen_library_src_paths.py (right):

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/tools/gen_...
runtime/tools/gen_library_src_paths.py:27: if line.startswith('library '):
This will iterate all the way through the file and if anything starts the line
with "library " then it is deemed to be the main_file.

Shouldn't it just be the first file in the list?

In any case we should be able to stop the iteration once we have found "library
" or "part of ". This would at least minimize the possibility of hitting some
other line starting with "library " and getting a false positive.

[Ivan Posva, 2013-05-02 22:23:21]

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/vm/bootstr...
File runtime/vm/bootstrap.cc (right):

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/vm/bootstr...
runtime/vm/bootstrap.cc:207: // In the boot strap loader we do not try and do
any canonicalization.
We should actually prepend the lib URI if this is a relative URI (in our special
case that means it does not start with "dart:") so that we canonicalize to
"dart:core/bool.dart" when including the "part 'bool.dart';" from the library
"dart:core".

[siva, 2013-05-03 01:58:32]

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/bin/run_vm...
File runtime/bin/run_vm_tests.cc (right):

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/bin/run_vm...
runtime/bin/run_vm_tests.cc:14: using dart::bin::DartUtils;
On 2013/05/02 21:42:58, Ivan Posva wrote:
> No using, please...

Done.

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/bin/run_vm...
runtime/bin/run_vm_tests.cc:110: DartUtils::OpenFile,
There are some basic tests which cannot assume the presence of a snapshot. I
don't think it is possible to eliminate basic boot strapping.

On 2013/05/02 21:42:58, Ivan Posva wrote:
> Do we need these here? Can we run the VM tests from the snapshot?

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/tools/gen_...
File runtime/tools/gen_library_src_paths.py (right):

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/tools/gen_...
runtime/tools/gen_library_src_paths.py:27: if line.startswith('library '):
I didn't want to make the assumption that the first file is
always the main file.

Added code to breakout once the main file is identified.

On 2013/05/02 21:42:58, Ivan Posva wrote:
> This will iterate all the way through the file and if anything starts the line
> with "library " then it is deemed to be the main_file.
> 
> Shouldn't it just be the first file in the list?
> 
> In any case we should be able to stop the iteration once we have found
"library
> " or "part of ". This would at least minimize the possibility of hitting some
> other line starting with "library " and getting a false positive.

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/vm/bootstr...
File runtime/vm/bootstrap.cc (right):

https://chromiumcodereview.appspot.com/14786012/diff/28001/runtime/vm/bootstr...
runtime/vm/bootstrap.cc:207: // In the boot strap loader we do not try and do
any canonicalization.
On 2013/05/02 22:23:21, Ivan Posva wrote:
> We should actually prepend the lib URI if this is a relative URI (in our
special
> case that means it does not start with "dart:") so that we canonicalize to
> "dart:core/bool.dart" when including the "part 'bool.dart';" from the library
> "dart:core".

Instead of prepending the lib URI here, as we would get called back with the
prepended string when we need to actually read the file, I prepend the lib URI
when we create the script object for the parts in LoadPartSource above.

[Ivan Posva, 2013-05-03 05:21:35]

LGTM -ip

https://chromiumcodereview.appspot.com/14786012/diff/43001/runtime/vm/bootstr...
File runtime/vm/bootstrap.cc (right):

https://chromiumcodereview.appspot.com/14786012/diff/43001/runtime/vm/bootstr...
runtime/vm/bootstrap.cc:193: strings.SetAt(1, String::Handle(isolate,
String::New("/")));
Symbols::Slash()

[siva, 2013-05-03 17:03:08]

https://codereview.chromium.org/14786012/diff/43001/runtime/vm/bootstrap.cc
File runtime/vm/bootstrap.cc (right):

https://codereview.chromium.org/14786012/diff/43001/runtime/vm/bootstrap.cc#n...
runtime/vm/bootstrap.cc:193: strings.SetAt(1, String::Handle(isolate,
String::New("/")));
On 2013/05/03 05:21:35, Ivan Posva wrote:
> Symbols::Slash()

Done.

[siva, 2013-05-03 18:12:37]

Committed patchset #6 manually as r22380 (presubmit successful).