Provide a way to suppress implicit MIDL generation rules for actions

Provide a way to suppress implicit MIDL generation rules for actions

For IDL files, GYP ninja normally generates rules to build MIDL
unless there are explicit rules. This special casing only works
for rules, but not actions.
In blink, we are going to add an action which runs blink's IDL compiler
to generate IDL dictionary implementation .h/.cpp files, which means
that we need a way to similar logic for actions.
This CL introduces 'explicit_idl_action' flag for such purpose.

Patch from bashi@chromium.org.

R=scottmg@chromium.org

Committed: https://code.google.com/p/gyp/source/detail?r=1964

[bashi, 2014-08-06 02:16:33]

Hi Nico, Could you take a look at this?

We want to have a way to suppress gyp's default IDL handling for actions
(Context: https://codereview.chromium.org/429853002/).

I don't think this is an ideal solution, but I can't figure out a better
solution.

WDYT?

[Nico, 2014-08-06 04:06:18]

ninja/win -> scottmg

[scottmg, 2014-08-06 16:22:28]

Can you add a test please? See e.g. test/win/*idl*.

[bashi, 2014-08-07 02:22:33]

On 2014/08/06 16:22:28, scottmg wrote:
> Can you add a test please? See e.g. test/win/*idl*.

Added a target in test/win/idl-rules/basic-idl.gyp. This target has
'explicit_idl_action' flag.

[scottmg, 2014-08-07 03:43:28]

On 2014/08/07 02:22:33, bashi1 wrote:
> On 2014/08/06 16:22:28, scottmg wrote:
> > Can you add a test please? See e.g. test/win/*idl*.
> 
> Added a target in test/win/idl-rules/basic-idl.gyp. This target has
> 'explicit_idl_action' flag.

It looks like you forgot to add the .py and .idl that the test use.

[bashi, 2014-08-08 01:09:44]

On 2014/08/07 03:43:28, scottmg wrote:
> On 2014/08/07 02:22:33, bashi1 wrote:
> > On 2014/08/06 16:22:28, scottmg wrote:
> > > Can you add a test please? See e.g. test/win/*idl*.
> > 
> > Added a target in test/win/idl-rules/basic-idl.gyp. This target has
> > 'explicit_idl_action' flag.
> 
> It looks like you forgot to add the .py and .idl that the test use.

Ah, sorry. Added. However, I got following errors when I ran the test. Do I need
VS2013 (not express)?

PYTHONPATH=C:\src\chromium\src\tools\gyp\test\lib
TESTGYP_FORMAT=msvs
C:\python_27_amd64\files\python.exe .\test\win\gyptest-midl-rules.py
Error: could not find devenv
Exception AttributeError: "'TestGypMSVS' object has no attribute '_dirlist'" in
<bound method TestGypMSVS.__del__ of 2a8
fdd8> ignored
TESTGYP_FORMAT=ninja
C:\python_27_amd64\files\python.exe .\test\win\gyptest-midl-rules.py
Error: could not find devenv
Exception AttributeError: "'TestGypNinja' object has no attribute '_dirlist'" in
<bound method TestGypNinja.__del__ of 2
82f630> ignored

Failed the following 2 tests:
        .\test\win\gyptest-midl-rules.py
        .\test\win\gyptest-midl-rules.py

[scottmg, 2014-08-08 23:20:02]

On 2014/08/08 01:09:44, bashi1 wrote:
> On 2014/08/07 03:43:28, scottmg wrote:
> > On 2014/08/07 02:22:33, bashi1 wrote:
> > > On 2014/08/06 16:22:28, scottmg wrote:
> > > > Can you add a test please? See e.g. test/win/*idl*.
> > > 
> > > Added a target in test/win/idl-rules/basic-idl.gyp. This target has
> > > 'explicit_idl_action' flag.
> > 
> > It looks like you forgot to add the .py and .idl that the test use.
> 
> Ah, sorry. Added. However, I got following errors when I ran the test. Do I
need
> VS2013 (not express)?
> 
> PYTHONPATH=C:\src\chromium\src\tools\gyp\test\lib
> TESTGYP_FORMAT=msvs
> C:\python_27_amd64\files\python.exe .\test\win\gyptest-midl-rules.py
> Error: could not find devenv
> Exception AttributeError: "'TestGypMSVS' object has no attribute '_dirlist'"
in
> <bound method TestGypMSVS.__del__ of 2a8
> fdd8> ignored
> TESTGYP_FORMAT=ninja
> C:\python_27_amd64\files\python.exe .\test\win\gyptest-midl-rules.py
> Error: could not find devenv
> Exception AttributeError: "'TestGypNinja' object has no attribute '_dirlist'"
in
> <bound method TestGypNinja.__del__ of 2
> 82f630> ignored
> 
> Failed the following 2 tests:
>         .\test\win\gyptest-midl-rules.py
>         .\test\win\gyptest-midl-rules.py

I'm not sure, maybe yes. I patched it in, and get a failure on ninja per below.
You can run just ninja (which will maybe work with Express?) by passing -f ninja
to gyptest.py. It's also helpful to change "if params['parallel']:" in ninja.py
to "if False:" to get better stack traces.


[bashi-midl]d:\src\cr3\src\tools\gyp>python gyptest.py
test\win\gyptest-midl-rules.py
PYTHONPATH=d:\src\cr3\src\tools\gyp\test\lib
TESTGYP_FORMAT=msvs
d:\src\depot_tools\python276_bin\python.exe test\win\gyptest-midl-rules.py
PASSED
TESTGYP_FORMAT=ninja
d:\src\depot_tools\python276_bin\python.exe test\win\gyptest-midl-rules.py
None returned 1
STDOUT =========================================================================

STDERR =========================================================================
Traceback (most recent call last):
  File "d:\src\cr3\src\tools\gyp\gyp_main.py", line 18, in <module>
    sys.exit(gyp.script_main())
  File "d:\src\cr3\src\tools\gyp\pylib\gyp\__init__.py", line 544, in
script_main
    return main(sys.argv[1:])
  File "d:\src\cr3\src\tools\gyp\pylib\gyp\__init__.py", line 537, in main
    return gyp_main(args)
  File "d:\src\cr3\src\tools\gyp\pylib\gyp\__init__.py", line 522, in gyp_main
    generator.GenerateOutput(flat_list, targets, data, params)
  File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 2239, in
GenerateOutput
    config_name)
  File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 2157, in
GenerateOutputForConfig
    target = writer.WriteSpec(spec, config_name, generator_flags)
  File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 440, in
WriteSpec
    spec, extra_sources, actions_depends, mac_bundle_depends)
  File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 553, in
WriteActionsRulesCopies
    extra_mac_bundle_resources)
  File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 611, in
WriteActions
    outputs = [self.GypPathToNinja(o, env) for o in action['outputs']]
  File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 312, in
GypPathToNinja
    assert '$' not in path, path
AssertionError: ${root}.cpp

FAILED test at line 423 of d:\src\cr3\src\tools\gyp\test\lib\TestCommon.py
(_complete)
        from line 532 of d:\src\cr3\src\tools\gyp\test\lib\TestCommon.py (run)
        from line 304 of d:\src\cr3\src\tools\gyp\test\lib\TestGyp.py (run)
        from line 291 of d:\src\cr3\src\tools\gyp\test\lib\TestGyp.py (run_gyp)
        from line 986 of d:\src\cr3\src\tools\gyp\test\lib\TestGyp.py (run_gyp)
        from line 19 of test\win\gyptest-midl-rules.py

Failed the following test:
        test\win\gyptest-midl-rules.py

[bashi, 2014-08-12 01:07:15]

On 2014/08/08 23:20:02, scottmg wrote:
> On 2014/08/08 01:09:44, bashi1 wrote:
> > On 2014/08/07 03:43:28, scottmg wrote:
> > > On 2014/08/07 02:22:33, bashi1 wrote:
> > > > On 2014/08/06 16:22:28, scottmg wrote:
> > > > > Can you add a test please? See e.g. test/win/*idl*.
> > > > 
> > > > Added a target in test/win/idl-rules/basic-idl.gyp. This target has
> > > > 'explicit_idl_action' flag.
> > > 
> > > It looks like you forgot to add the .py and .idl that the test use.
> > 
> > Ah, sorry. Added. However, I got following errors when I ran the test. Do I
> need
> > VS2013 (not express)?
> > 
> > PYTHONPATH=C:\src\chromium\src\tools\gyp\test\lib
> > TESTGYP_FORMAT=msvs
> > C:\python_27_amd64\files\python.exe .\test\win\gyptest-midl-rules.py
> > Error: could not find devenv
> > Exception AttributeError: "'TestGypMSVS' object has no attribute '_dirlist'"
> in
> > <bound method TestGypMSVS.__del__ of 2a8
> > fdd8> ignored
> > TESTGYP_FORMAT=ninja
> > C:\python_27_amd64\files\python.exe .\test\win\gyptest-midl-rules.py
> > Error: could not find devenv
> > Exception AttributeError: "'TestGypNinja' object has no attribute
'_dirlist'"
> in
> > <bound method TestGypNinja.__del__ of 2
> > 82f630> ignored
> > 
> > Failed the following 2 tests:
> >         .\test\win\gyptest-midl-rules.py
> >         .\test\win\gyptest-midl-rules.py
> 
> I'm not sure, maybe yes. I patched it in, and get a failure on ninja per
below.
> You can run just ninja (which will maybe work with Express?) by passing -f
ninja
> to gyptest.py. It's also helpful to change "if params['parallel']:" in
ninja.py
> to "if False:" to get better stack traces.
> 
> 
> [bashi-midl]d:\src\cr3\src\tools\gyp>python gyptest.py
> test\win\gyptest-midl-rules.py
> PYTHONPATH=d:\src\cr3\src\tools\gyp\test\lib
> TESTGYP_FORMAT=msvs
> d:\src\depot_tools\python276_bin\python.exe test\win\gyptest-midl-rules.py
> PASSED
> TESTGYP_FORMAT=ninja
> d:\src\depot_tools\python276_bin\python.exe test\win\gyptest-midl-rules.py
> None returned 1
> STDOUT
=========================================================================
> 
> STDERR
=========================================================================
> Traceback (most recent call last):
>   File "d:\src\cr3\src\tools\gyp\gyp_main.py", line 18, in <module>
>     sys.exit(gyp.script_main())
>   File "d:\src\cr3\src\tools\gyp\pylib\gyp\__init__.py", line 544, in
> script_main
>     return main(sys.argv[1:])
>   File "d:\src\cr3\src\tools\gyp\pylib\gyp\__init__.py", line 537, in main
>     return gyp_main(args)
>   File "d:\src\cr3\src\tools\gyp\pylib\gyp\__init__.py", line 522, in gyp_main
>     generator.GenerateOutput(flat_list, targets, data, params)
>   File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 2239, in
> GenerateOutput
>     config_name)
>   File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 2157, in
> GenerateOutputForConfig
>     target = writer.WriteSpec(spec, config_name, generator_flags)
>   File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 440, in
> WriteSpec
>     spec, extra_sources, actions_depends, mac_bundle_depends)
>   File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 553, in
> WriteActionsRulesCopies
>     extra_mac_bundle_resources)
>   File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 611, in
> WriteActions
>     outputs = [self.GypPathToNinja(o, env) for o in action['outputs']]
>   File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 312, in
> GypPathToNinja
>     assert '$' not in path, path
> AssertionError: ${root}.cpp
> 
> FAILED test at line 423 of d:\src\cr3\src\tools\gyp\test\lib\TestCommon.py
> (_complete)
>         from line 532 of d:\src\cr3\src\tools\gyp\test\lib\TestCommon.py (run)
>         from line 304 of d:\src\cr3\src\tools\gyp\test\lib\TestGyp.py (run)
>         from line 291 of d:\src\cr3\src\tools\gyp\test\lib\TestGyp.py
(run_gyp)
>         from line 986 of d:\src\cr3\src\tools\gyp\test\lib\TestGyp.py
(run_gyp)
>         from line 19 of test\win\gyptest-midl-rules.py
> 
> Failed the following test:
>         test\win\gyptest-midl-rules.py

Thank you so much for testing this CL! I'm still getting the "could not find
devenv" errors even specifying -f ninja options.

I uploaded a speculative fix (PatchSet #4). I submitted try jobs, bot it seems
not working?

[scottmg, 2014-08-12 03:32:44]

On 2014/08/12 01:07:15, bashi1 wrote:
> On 2014/08/08 23:20:02, scottmg wrote:
> > On 2014/08/08 01:09:44, bashi1 wrote:
> > > On 2014/08/07 03:43:28, scottmg wrote:
> > > > On 2014/08/07 02:22:33, bashi1 wrote:
> > > > > On 2014/08/06 16:22:28, scottmg wrote:
> > > > > > Can you add a test please? See e.g. test/win/*idl*.
> > > > > 
> > > > > Added a target in test/win/idl-rules/basic-idl.gyp. This target has
> > > > > 'explicit_idl_action' flag.
> > > > 
> > > > It looks like you forgot to add the .py and .idl that the test use.
> > > 
> > > Ah, sorry. Added. However, I got following errors when I ran the test. Do
I
> > need
> > > VS2013 (not express)?
> > > 
> > > PYTHONPATH=C:\src\chromium\src\tools\gyp\test\lib
> > > TESTGYP_FORMAT=msvs
> > > C:\python_27_amd64\files\python.exe .\test\win\gyptest-midl-rules.py
> > > Error: could not find devenv
> > > Exception AttributeError: "'TestGypMSVS' object has no attribute
'_dirlist'"
> > in
> > > <bound method TestGypMSVS.__del__ of 2a8
> > > fdd8> ignored
> > > TESTGYP_FORMAT=ninja
> > > C:\python_27_amd64\files\python.exe .\test\win\gyptest-midl-rules.py
> > > Error: could not find devenv
> > > Exception AttributeError: "'TestGypNinja' object has no attribute
> '_dirlist'"
> > in
> > > <bound method TestGypNinja.__del__ of 2
> > > 82f630> ignored
> > > 
> > > Failed the following 2 tests:
> > >         .\test\win\gyptest-midl-rules.py
> > >         .\test\win\gyptest-midl-rules.py
> > 
> > I'm not sure, maybe yes. I patched it in, and get a failure on ninja per
> below.
> > You can run just ninja (which will maybe work with Express?) by passing -f
> ninja
> > to gyptest.py. It's also helpful to change "if params['parallel']:" in
> ninja.py
> > to "if False:" to get better stack traces.
> > 
> > 
> > [bashi-midl]d:\src\cr3\src\tools\gyp>python gyptest.py
> > test\win\gyptest-midl-rules.py
> > PYTHONPATH=d:\src\cr3\src\tools\gyp\test\lib
> > TESTGYP_FORMAT=msvs
> > d:\src\depot_tools\python276_bin\python.exe test\win\gyptest-midl-rules.py
> > PASSED
> > TESTGYP_FORMAT=ninja
> > d:\src\depot_tools\python276_bin\python.exe test\win\gyptest-midl-rules.py
> > None returned 1
> > STDOUT
> =========================================================================
> > 
> > STDERR
> =========================================================================
> > Traceback (most recent call last):
> >   File "d:\src\cr3\src\tools\gyp\gyp_main.py", line 18, in <module>
> >     sys.exit(gyp.script_main())
> >   File "d:\src\cr3\src\tools\gyp\pylib\gyp\__init__.py", line 544, in
> > script_main
> >     return main(sys.argv[1:])
> >   File "d:\src\cr3\src\tools\gyp\pylib\gyp\__init__.py", line 537, in main
> >     return gyp_main(args)
> >   File "d:\src\cr3\src\tools\gyp\pylib\gyp\__init__.py", line 522, in
gyp_main
> >     generator.GenerateOutput(flat_list, targets, data, params)
> >   File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 2239,
in
> > GenerateOutput
> >     config_name)
> >   File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 2157,
in
> > GenerateOutputForConfig
> >     target = writer.WriteSpec(spec, config_name, generator_flags)
> >   File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 440, in
> > WriteSpec
> >     spec, extra_sources, actions_depends, mac_bundle_depends)
> >   File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 553, in
> > WriteActionsRulesCopies
> >     extra_mac_bundle_resources)
> >   File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 611, in
> > WriteActions
> >     outputs = [self.GypPathToNinja(o, env) for o in action['outputs']]
> >   File "d:\src\cr3\src\tools\gyp\pylib\gyp\generator\ninja.py", line 312, in
> > GypPathToNinja
> >     assert '$' not in path, path
> > AssertionError: ${root}.cpp
> > 
> > FAILED test at line 423 of d:\src\cr3\src\tools\gyp\test\lib\TestCommon.py
> > (_complete)
> >         from line 532 of d:\src\cr3\src\tools\gyp\test\lib\TestCommon.py
(run)
> >         from line 304 of d:\src\cr3\src\tools\gyp\test\lib\TestGyp.py (run)
> >         from line 291 of d:\src\cr3\src\tools\gyp\test\lib\TestGyp.py
> (run_gyp)
> >         from line 986 of d:\src\cr3\src\tools\gyp\test\lib\TestGyp.py
> (run_gyp)
> >         from line 19 of test\win\gyptest-midl-rules.py
> > 
> > Failed the following test:
> >         test\win\gyptest-midl-rules.py
> 
> Thank you so much for testing this CL! I'm still getting the "could not find
> devenv" errors even specifying -f ninja options.
> 
> I uploaded a speculative fix (PatchSet #4). I submitted try jobs, bot it seems
> not working?

Yes, explicit names seems to work.

[scottmg, 2014-08-12 03:34:02]

LGTM with that. I can land it.

https://codereview.chromium.org/440293002/diff/40001/test/win/idl-rules/basic...
File test/win/idl-rules/basic-idl.gyp (right):

https://codereview.chromium.org/440293002/diff/40001/test/win/idl-rules/basic...
test/win/idl-rules/basic-idl.gyp:49: 'explicit_idl_action': 1,
you should add

'msvs_cygwin_shell': 0,

here.

[scottmg, 2014-08-12 03:36:07]

Committed patchset #4 manually as 1964 (presubmit successful).

[bashi, 2014-08-12 03:57:22]

On 2014/08/12 03:36:07, scottmg wrote:
> Committed patchset #4 manually as 1964 (presubmit successful).

Thank you! I'll use the flag with msvs_cygwin_shell: 0.

Is it ok to create a CL to roll GYP r1964 in Chromium by myself?

[bashi, 2014-08-12 04:17:12]

On 2014/08/12 03:57:22, bashi1 wrote:
> On 2014/08/12 03:36:07, scottmg wrote:
> > Committed patchset #4 manually as 1964 (presubmit successful).
> 
> Thank you! I'll use the flag with msvs_cygwin_shell: 0.
> 
> Is it ok to create a CL to roll GYP r1964 in Chromium by myself?

Prepared a roll. https://codereview.chromium.org/467483002/
I'll land this if bots happy.