Generate docgen output along with api_docs as part of the build

utils/apidoc/docgen.gyp

 # Copyright (c) 2012, the Dart project authors.  Please see the AUTHORS file
 # for details. All rights reserved. Use of this source code is governed by a
 # BSD-style license that can be found in the LICENSE file.
 
 {
   'variables' : {
     'script_suffix%': '',
   },
   'conditions' : [
     ['OS=="win"', {
       'variables' : {
         'script_suffix': '.bat',
       },
     }],
   ],
   'targets': [
     {
-      'target_name': 'api_docs',
+      'target_name': 'docgen',
       'type': 'none',
       'dependencies': [
         '../../utils/compiler/compiler.gyp:dart2js',
         '../../runtime/dart-runtime.gyp:dart',
         '../../pkg/pkg.gyp:pkg_packages',

[kustermann, 2013/11/19 16:40:37]

I'd still depend on the old apidocs target here.

[Alan Knight, 2013/11/19 19:14:47]

Done.

       ],
       'includes': [
         '../../sdk/lib/core/corelib_sources.gypi',
       ],
       'actions': [
         {
-          'action_name': 'run_apidoc',
+          'action_name': 'run_docgen',
           # The 'inputs' list records the files whose timestamps are
           # compared to the files listed in 'outputs'.  If a file
           # 'outputs' doesn't exist or if a file in 'inputs' is newer
           # than a file in 'outputs', this action is executed.  Notice
           # that the dependencies listed above has nothing to do with
           # when this action is executed.  You must list a file in
           # 'inputs' to make sure that it exists before the action is
           # executed, or to make sure this action is re-run.
           #
           # We want to build the platform documentation whenever
           # dartdoc, apidoc, or its dependency changes.  This prevents
           # people from accidentally breaking apidoc when making
           # changes to the platform libraries and or when modifying
           # dart2js or the VM.
           #
           # In addition, we want to make sure that the platform
           # documentation is regenerated when the platform sources
           # changes.
           #
           # So we want this action to be re-run when a dart file
           # changes in this directory, or in the SDK library (we may
           # no longer need to list the files in ../../runtime/lib and
           # ../../runtime/bin, as most of them has moved to
           # ../../sdk/lib).
           #
-          # In addition, we want to make sure the documentation is
-          # regenerated when a resource file (CSS, PNG, etc) is
-          # updated.  This is because these files are also copied to
-          # the output directory.
           'inputs': [
             '<(PRODUCT_DIR)/<(EXECUTABLE_PREFIX)dart<(EXECUTABLE_SUFFIX)',
             '<(SHARED_INTERMEDIATE_DIR)/utils_wrapper.dart.snapshot',
             '<!@(["python", "../../tools/list_files.py", "\\.(css|ico|js|json|png|sh|txt|yaml|py)$", ".", "../../sdk/lib/_internal/dartdoc"])',
             '<!@(["python", "../../tools/list_files.py", "\\.dart$", ".", "../../sdk/lib", "../../runtime/lib", "../../runtime/bin"])',
             '../../sdk/bin/dart',
             '../../sdk/bin/dart.bat',
             '../../sdk/bin/dart2js',
             '../../sdk/bin/dart2js.bat',
             '../../tools/only_in_release_mode.py',
+            '<(PRODUCT_DIR)/api_docs/index.html',

[kustermann, 2013/11/19 16:40:37]

Add a short comment, that this is used for ensuring that "docgen" is executed
after "apidocs".

[Alan Knight, 2013/11/19 19:14:47]

Done.

           ],
           'outputs': [
-            '<(PRODUCT_DIR)/api_docs/index.html',
-            '<(PRODUCT_DIR)/api_docs/client-static.js',
+            '<(PRODUCT_DIR)/api_docs/docgen/index.json',
           ],
           'action': [
             'python',
             '../../tools/only_in_release_mode.py',
             '<@(_outputs)',
             '--',
             '<(PRODUCT_DIR)/<(EXECUTABLE_PREFIX)dart<(EXECUTABLE_SUFFIX)',
+            '--old_gen_heap_size=1024',
             '--package-root=<(PRODUCT_DIR)/packages/',
-            'apidoc.dart',
-            '--out=<(PRODUCT_DIR)/api_docs',
-            '--version=<!@(["python", "../../tools/print_version.py"])',
+            '../../pkg/docgen/bin/docgen.dart',
+            '--out=<(PRODUCT_DIR)/api_docs/docgen',
+            '--json',
+            '--include-sdk',
             '--package-root=<(PRODUCT_DIR)/packages',
-            '--mode=static',
-            '--exclude-lib=analyzer',
             '--exclude-lib=async_helper',
-            '--exclude-lib=barback',
-            '--exclude-lib=browser',
-            '--exclude-lib=dartdoc',
+            '--exclude-lib=expect',
             '--exclude-lib=docgen',

[kustermann, 2013/11/19 16:40:37]

I'm wondering why you removed so many '--exclude-lib' flags here.

[Alan Knight, 2013/11/19 19:14:47]

From what I could tell there were two rationales for excluding things. One is
things that, while they were in the pkg directory, weren't being published as
standalone packages. This is a short blacklist found in
tools/publish_all_pkgs.py. The other were things that people at some point
didn't want displayed by default in the sdk docs, but this seems to have been
applied non-systematically. 

The new viewer's default view does not show anything in pkg and allows showing
the documentation for just a single package, so it seems reasonable to generate
all the docs and be able to provide documentation for individual packages that
are published on pub (i.e. everything in pkg not in . Ideally, we'd move to
generating fully individual documentation sets for each package and hosting that
differently, but this is a useful stopgap.

I did add docgen to the list. It should arguably be excluded, since it doesn't
work outside of the SDK (imports dartj2s mirrors via ../../.. paths). But it's
been published already, so if we stop we should probably also remove it and that
seems like a separate CL.

-            '--exclude-lib=expect',
-            '--exclude-lib=http',
-            '--exclude-lib=oauth2',
-            '--exclude-lib=scheduled_test',
-            '--exclude-lib=stack_trace',
-            '--exclude-lib=watcher',
-            '--exclude-lib=webdriver',
-            '--exclude-lib=yaml',
-            '--include-lib=matcher',
-            '--extra-lib=pkg/unittest/lib/mock.dart',
+            '../../pkg',          
           ],
-          'message': 'Running apidoc: <(_action)',
+          'message': 'Running docgen: <(_action)',
         },
       ],
     }
   ],
 }