Moving version script and gypi file to build\util.

Moving version script and gypi file to build\util.

BUG=343792
OK, it's been almost a week now, with regular pings to OWNERs, so I guess it's OK to TBR... It's only Gyp/script changes anyway... Feel free to uncheck the CQ box if you disagree...

TBR=beng@chromium.org, darin@chromium.org

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=252481

[MAD, 2014-02-14 04:10:39]

Can you please validate that I did this right, before I ask for OWNERS review?

I think the chrome\VERSION file should also be moved to build\util, but since
this file is generated from other tools that I'm not familiar with, I prefer not
to touch it just yet. I think this initial patch at least resolvea the issue of
using base without Chrome, as long as the version script doesn't get invoked,
which can be avoided by setting the GYP variable test_isolation_mode to noop,
right?

Thanks!

BYE
MAD...

[brettw, 2014-02-14 05:47:57]

lgtm

[MAD, 2014-02-14 15:38:37]

Darin: OWNERS review please?

Thanks!

BYE
MAD...

[grt (UTC plus 2), 2014-02-14 16:44:10]

https://codereview.chromium.org/165893002/diff/210001/build/util/version.gypi
File build/util/version.gypi (right):

https://codereview.chromium.org/165893002/diff/210001/build/util/version.gypi...
build/util/version.gypi:9: 'version_path': '<(DEPTH)/chrome/VERSION',
since other projects may pull in build and base via DEPS without pulling in
Chrome, please allow this to be overridden with:
      'version_path%': '<(DEPTH)/chrome/VERSION',

[MAD, 2014-02-14 16:52:06]

https://codereview.chromium.org/165893002/diff/210001/build/util/version.gypi
File build/util/version.gypi (right):

https://codereview.chromium.org/165893002/diff/210001/build/util/version.gypi...
build/util/version.gypi:9: 'version_path': '<(DEPTH)/chrome/VERSION',
On 2014/02/14 16:44:11, grt wrote:
> since other projects may pull in build and base via DEPS without pulling in
> Chrome, please allow this to be overridden with:
>       'version_path%': '<(DEPTH)/chrome/VERSION',

Good idea... 

Note that pulling base doesn't trigger this if the GYP variable
test_isolation_mode is set to "noop"...

But I like your trick bettert...

Thanks!

Done.

[MAD, 2014-02-14 17:35:45]

Actually, I take that back... It doesn't work... When I do this, I get this
error, and I can't find where it comes from:
===============================================
________ running 'd:\src\depot_tools\python276_bin\python.exe
src/build/gyp_chromium' in 'd:\src\git-chrome1'
Enabled Psyco JIT.
Generating gyp files from GN...
Updating projects from gyp files...
Traceback (most recent call last):
  File "../build/util/version.py", line 197, in <module>
    sys.exit(main())
  File "../build/util/version.py", line 169, in main
    values = fetch_values(variable_files)
  File "../build/util/version.py", line 55, in fetch_values
    fetch_values_from_file(values, file_name)
  File "../build/util/version.py", line 30, in fetch_values_from_file
    for line in open(file_name, 'r').readlines():
IOError: [Errno 2] No such file or directory: 'VERSION'
gyp: Call to 'python ../build/util/version.py -f VERSION -t
"@BUILD@.@PATCH_HI@.@PATCH_LO@" -e "PATCH_HI=int(PATCH)/256"  -e
"PATCH_LO=int(PATCH)%256"' returned exit status 1.
Error: Command d:\src\depot_tools\python276_bin\python.exe
src/build/gyp_chromium returned non-zero exit status 1 in d:\src\git-chrome1
Hook ''d:\src\depot_tools\python276_bin\python.exe' src/build/gyp_chromium' took
10.79 secs
===============================================

I tried to look for places in gyp files where VERSION is set without a path and
even though I change them I still get this error... 

Any clue what could cause this?

Thanks!

BYE
MAD

[grt (UTC plus 2), 2014-02-14 17:42:10]

On 2014/02/14 17:35:45, MAD wrote:
> Actually, I take that back... It doesn't work... When I do this, I get this
> error, and I can't find where it comes from:
> ===============================================
> ________ running 'd:\src\depot_tools\python276_bin\python.exe
> src/build/gyp_chromium' in 'd:\src\git-chrome1'
> Enabled Psyco JIT.
> Generating gyp files from GN...
> Updating projects from gyp files...
> Traceback (most recent call last):
>   File "../build/util/version.py", line 197, in <module>
>     sys.exit(main())
>   File "../build/util/version.py", line 169, in main
>     values = fetch_values(variable_files)
>   File "../build/util/version.py", line 55, in fetch_values
>     fetch_values_from_file(values, file_name)
>   File "../build/util/version.py", line 30, in fetch_values_from_file
>     for line in open(file_name, 'r').readlines():
> IOError: [Errno 2] No such file or directory: 'VERSION'
> gyp: Call to 'python ../build/util/version.py -f VERSION -t
> "@BUILD@.@PATCH_HI@.@PATCH_LO@" -e "PATCH_HI=int(PATCH)/256"  -e
> "PATCH_LO=int(PATCH)%256"' returned exit status 1.
> Error: Command d:\src\depot_tools\python276_bin\python.exe
> src/build/gyp_chromium returned non-zero exit status 1 in d:\src\git-chrome1
> Hook ''d:\src\depot_tools\python276_bin\python.exe' src/build/gyp_chromium'
took
> 10.79 secs
> ===============================================
> 
> I tried to look for places in gyp files where VERSION is set without a path
and
> even though I change them I still get this error... 
> 
> Any clue what could cause this?

I wasn't suggesting using VERSION without a path. I was suggesting letting the
embedder of version.gpyi override the location of the VERSION file so that they
can have their own VERSION independent of Chrome's. I'm not sure what you've
tried that leads to the failure you're seeing. Perhaps upload your change?

[MAD, 2014-02-14 18:20:44]

I just added the % to both version_path of that path (GYP failed when I
added it just to one :-)... I left the path as is...

So my guess is that somewhere else, someone overrides this value without
the path and it affects this... I tried finding other gyp files that use
VERSION without the path, and adding the path there didn't change
anything...

Do you still want me to upload a patch...



On Fri, Feb 14, 2014 at 12:42 PM, <grt@chromium.org> wrote:

> On 2014/02/14 17:35:45, MAD wrote:
>
>> Actually, I take that back... It doesn't work... When I do this, I get
>> this
>> error, and I can't find where it comes from:
>> ===============================================
>> ________ running 'd:\src\depot_tools\python276_bin\python.exe
>> src/build/gyp_chromium' in 'd:\src\git-chrome1'
>> Enabled Psyco JIT.
>> Generating gyp files from GN...
>> Updating projects from gyp files...
>> Traceback (most recent call last):
>>    File "../build/util/version.py", line 197, in <module>
>>      sys.exit(main())
>>    File "../build/util/version.py", line 169, in main
>>      values = fetch_values(variable_files)
>>    File "../build/util/version.py", line 55, in fetch_values
>>      fetch_values_from_file(values, file_name)
>>    File "../build/util/version.py", line 30, in fetch_values_from_file
>>      for line in open(file_name, 'r').readlines():
>> IOError: [Errno 2] No such file or directory: 'VERSION'
>> gyp: Call to 'python ../build/util/version.py -f VERSION -t
>> "@BUILD@.@PATCH_HI@.@PATCH_LO@" -e "PATCH_HI=int(PATCH)/256"  -e
>> "PATCH_LO=int(PATCH)%256"' returned exit status 1.
>> Error: Command d:\src\depot_tools\python276_bin\python.exe
>> src/build/gyp_chromium returned non-zero exit status 1 in
>> d:\src\git-chrome1
>> Hook ''d:\src\depot_tools\python276_bin\python.exe'
>> src/build/gyp_chromium'
>>
> took
>
>> 10.79 secs
>> ===============================================
>>
>
>  I tried to look for places in gyp files where VERSION is set without a
>> path
>>
> and
>
>> even though I change them I still get this error...
>>
>
>  Any clue what could cause this?
>>
>
> I wasn't suggesting using VERSION without a path. I was suggesting letting
> the
> embedder of version.gpyi override the location of the VERSION file so that
> they
> can have their own VERSION independent of Chrome's. I'm not sure what
> you've
> tried that leads to the failure you're seeing. Perhaps upload your change?
>
> https://codereview.chromium.org/165893002/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[grt (UTC plus 2), 2014-02-14 19:14:14]

On 2014/02/14 18:20:44, MAD wrote:
> I just added the % to both version_path of that path (GYP failed when I
> added it just to one :-)... I left the path as is...

Hmm. Have a look at build\common.gypi for inspiration. It uses % to allow
variables to be overridden. Notice that it jumps through hoops to copy variables
up scopes. Maybe this verson .gypi needs to do the same thing.

[MAD, 2014-02-14 19:56:36]

It's already two level deep, so if I use:
  'variables': {
    'variables': {
      'version_py_path': '<(DEPTH)/build/util/version.py',
      'version_path%': '<(DEPTH)/chrome/VERSION',
    },
    'version_py_path': '<(version_py_path)',
    'version_path%': '<(version_path)',

I get:
Updating projects from gyp files...
Traceback (most recent call last):
  File "../build/util/version.py", line 197, in <module>
    sys.exit(main())
  File "../build/util/version.py", line 169, in main
    values = fetch_values(variable_files)
  File "../build/util/version.py", line 55, in fetch_values
    fetch_values_from_file(values, file_name)
  File "../build/util/version.py", line 30, in fetch_values_from_file
    for line in open(file_name, 'r').readlines():
IOError: [Errno 2] No such file or directory: 'VERSION'
gyp: Call to 'python ../build/util/version.py -f VERSION -t "@BUILD@
.@PATCH_HI@.@PATCH_LO@" -e "PATCH_HI=int(PATCH)/256" -e
"PATCH_LO=int(PATCH)%256"' returned exit status 1.
Error: Command d:\src\depot_tools\python276_bin\python.exe
src/build/gyp_chromium returned non-zero exit status 1 in d:\src\git-chrome1
Hook ''d:\src\depot_tools\python276_bin\python.exe' src/build/gyp_chromium'
took 10.91 secs

And then, if I add a level,
  'variables': {
    'variables': {
      'variables': {
        'version_py_path': '<(DEPTH)/build/util/version.py',
        'version_path%': '<(DEPTH)/chrome/VERSION',
      },
      'version_path%': '<(version_path)',
    },
    'version_py_path': '<(version_py_path)',
    'version_path%': '<(version_path)',

I get a whole bunch of:
  File "d:\src\git-chrome1\src\tools\gyp\pylib\gyp\input.py", line 994, in
ExpandVariables
    output = ExpandVariables(output, phase, variables, build_file)
  File "d:\src\git-chrome1\src\tools\gyp\pylib\gyp\input.py", line 994, in
ExpandVariables
    output = ExpandVariables(output, phase, variables, build_file)
  File "d:\src\git-chrome1\src\tools\gyp\pylib\gyp\input.py", line 994, in
ExpandVariables
    output = ExpandVariables(output, phase, variables, build_file)
  File "d:\src\git-chrome1\src\tools\gyp\pylib\gyp\input.py", line 994, in
ExpandVariables
    output = ExpandVariables(output, phase, variables, build_file)


Am I missing something?



On Fri, Feb 14, 2014 at 2:14 PM, <grt@chromium.org> wrote:

> On 2014/02/14 18:20:44, MAD wrote:
>
>> I just added the % to both version_path of that path (GYP failed when I
>> added it just to one :-)... I left the path as is...
>>
>
> Hmm. Have a look at build\common.gypi for inspiration. It uses % to allow
> variables to be overridden. Notice that it jumps through hoops to copy
> variables
> up scopes. Maybe this verson .gypi needs to do the same thing.
>
> https://codereview.chromium.org/165893002/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[Ken Russell (switch to Gerrit), 2014-02-14 20:46:35]

This seems fine overall, though it looks to me like it's pushing the problem a
little further down the road, since it just makes build/util/version.gypi refer
to chrome/version.py.

Maybe M-A will have another suggestion on restructuring this.

For the record, the reason this was necessary was to be able to properly include
the manifest alongside Chrome's Windows executable in chrome.isolate, and that
was needed for chrome_elf.dll for some reason. If a different solution were
found for that dependency then isolate.gypi wouldn't need to include
version.gypi at all.

[grt (UTC plus 2), 2014-02-14 21:04:52]

i suspect a name collision. the variable version_path is likely defined in
other .gypi files that are also being included/used.


On Fri, Feb 14, 2014 at 2:56 PM, Marc-Andre Decoste <mad@chromium.org>wrote:

> It's already two level deep, so if I use:
>   'variables': {
>     'variables': {
>       'version_py_path': '<(DEPTH)/build/util/version.py',
>       'version_path%': '<(DEPTH)/chrome/VERSION',
>     },
>     'version_py_path': '<(version_py_path)',
>     'version_path%': '<(version_path)',
>
> I get:
> Updating projects from gyp files...
> Traceback (most recent call last):
>   File "../build/util/version.py", line 197, in <module>
>     sys.exit(main())
>   File "../build/util/version.py", line 169, in main
>     values = fetch_values(variable_files)
>   File "../build/util/version.py", line 55, in fetch_values
>     fetch_values_from_file(values, file_name)
>   File "../build/util/version.py", line 30, in fetch_values_from_file
>     for line in open(file_name, 'r').readlines():
> IOError: [Errno 2] No such file or directory: 'VERSION'
> gyp: Call to 'python ../build/util/version.py -f VERSION -t "@BUILD@
> .@PATCH_HI@.@PATCH_LO@" -e "PATCH_HI=int(PATCH)/256" -e
> "PATCH_LO=int(PATCH)%256"' returned exit status 1.
> Error: Command d:\src\depot_tools\python276_bin\python.exe
> src/build/gyp_chromium returned non-zero exit status 1 in d:\
> src\git-chrome1
> Hook ''d:\src\depot_tools\python276_bin\python.exe'
> src/build/gyp_chromium' took 10.91 secs
>
> And then, if I add a level,
>   'variables': {
>     'variables': {
>       'variables': {
>         'version_py_path': '<(DEPTH)/build/util/version.py',
>         'version_path%': '<(DEPTH)/chrome/VERSION',
>       },
>       'version_path%': '<(version_path)',
>     },
>     'version_py_path': '<(version_py_path)',
>     'version_path%': '<(version_path)',
>
> I get a whole bunch of:
>   File "d:\src\git-chrome1\src\tools\gyp\pylib\gyp\input.py", line 994, in
> ExpandVariables
>     output = ExpandVariables(output, phase, variables, build_file)
>   File "d:\src\git-chrome1\src\tools\gyp\pylib\gyp\input.py", line 994, in
> ExpandVariables
>     output = ExpandVariables(output, phase, variables, build_file)
>   File "d:\src\git-chrome1\src\tools\gyp\pylib\gyp\input.py", line 994, in
> ExpandVariables
>     output = ExpandVariables(output, phase, variables, build_file)
>   File "d:\src\git-chrome1\src\tools\gyp\pylib\gyp\input.py", line 994, in
> ExpandVariables
>     output = ExpandVariables(output, phase, variables, build_file)
>
>
> Am I missing something?
>
>
>
> On Fri, Feb 14, 2014 at 2:14 PM, <grt@chromium.org> wrote:
>
>> On 2014/02/14 18:20:44, MAD wrote:
>>
>>> I just added the % to both version_path of that path (GYP failed when I
>>> added it just to one :-)... I left the path as is...
>>>
>>
>> Hmm. Have a look at build\common.gypi for inspiration. It uses % to allow
>> variables to be overridden. Notice that it jumps through hoops to copy
>> variables
>> up scopes. Maybe this verson .gypi needs to do the same thing.
>>
>> https://codereview.chromium.org/165893002/
>>
>
>  To unsubscribe from this group and stop receiving emails from it, send an
> email to chromium-reviews+unsubscribe@chromium.org.
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[MAD, 2014-02-14 21:27:48]

Actually, version.py is not under chrome anymore, it alongside
version.gypi, it's only VERSION that is left under chrome because I don't
know where are the scripts that update it...

It does solve the issue that gyp parsing for base doesn't force the need
for a chrome folder if the test_isolation_mode GYP variable is set to
noop... Which resolves my use case for now...

I suggest that the people who now how chrome/VERSION is generated that they
move it to build/util too...



On Fri, Feb 14, 2014 at 3:46 PM, <kbr@chromium.org> wrote:

> This seems fine overall, though it looks to me like it's pushing the
> problem a
> little further down the road, since it just makes build/util/version.gypi
> refer
> to chrome/version.py.
>
> Maybe M-A will have another suggestion on restructuring this.
>
> For the record, the reason this was necessary was to be able to properly
> include
> the manifest alongside Chrome's Windows executable in chrome.isolate, and
> that
> was needed for chrome_elf.dll for some reason. If a different solution were
> found for that dependency then isolate.gypi wouldn't need to include
> version.gypi at all.
>
>
> https://codereview.chromium.org/165893002/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[MAD, 2014-02-17 13:38:53]

Ping Darin?

Should I ask Ben instead?

Thanks!

BYE
MAD

[MAD, 2014-02-17 13:38:53]

Ping Darin?

Should I ask Ben instead?

Thanks!

BYE
MAD

[M-A Ruel, 2014-02-18 01:02:53]

On 2014/02/14 20:46:35, Ken Russell wrote:
> This seems fine overall, though it looks to me like it's pushing the problem a
> little further down the road, since it just makes build/util/version.gypi
refer
> to chrome/version.py.
> 
> Maybe M-A will have another suggestion on restructuring this.
> 
> For the record, the reason this was necessary was to be able to properly
include
> the manifest alongside Chrome's Windows executable in chrome.isolate, and that
> was needed for chrome_elf.dll for some reason. If a different solution were
> found for that dependency then isolate.gypi wouldn't need to include
> version.gypi at all.

An option would be to have 2 isolate.gypi, one that specifies more flags than
the 'base' one. But it's really a can of worms and would recommend against it.

I don't have a better idea than moving all the versioning related stuff into
build/.

[MAD, 2014-02-19 03:39:29]

Ping Ben or Darin? Can I get an OWNER review for this?

Thanks!

BYE
MAD

[Ken Russell (switch to Gerrit), 2014-02-19 08:22:23]

LGTM FWIW

https://codereview.chromium.org/165893002/diff/210001/chrome/chrome_common.gypi
File chrome/chrome_common.gypi (right):

https://codereview.chromium.org/165893002/diff/210001/chrome/chrome_common.gy...
chrome/chrome_common.gypi:595: 'version_py_path':
'<(DEPTH)/build/util//version.py',
Double '/'

[MAD, 2014-02-20 14:39:28]

Thanks Ken... Done...

So? Should I find other OWNERS? Darin? Ben?

BYE
MAD

https://codereview.chromium.org/165893002/diff/210001/chrome/chrome_common.gypi
File chrome/chrome_common.gypi (right):

https://codereview.chromium.org/165893002/diff/210001/chrome/chrome_common.gy...
chrome/chrome_common.gypi:595: 'version_py_path':
'<(DEPTH)/build/util//version.py',
On 2014/02/19 08:22:24, Ken Russell wrote:
> Double '/'

Done.

[MAD, 2014-02-20 18:19:45]

The CQ bit was checked by mad@chromium.org

[commit-bot: I haz the power, 2014-02-20 18:24:03]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/mad@chromium.org/165893002/500001

[commit-bot: I haz the power, 2014-02-20 18:54:11]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-02-20 18:54:12]

Retried try job too often on win_rel for step(s) base_unittests, browser_tests,
interactive_ui_tests, net_unittests, unit_tests
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=win_rel&nu...

[darin (slow to review), 2014-02-20 18:57:53]

LGTM (I defer to brettw+kbr)

[MAD, 2014-02-20 20:09:31]

The CQ bit was checked by mad@chromium.org

[commit-bot: I haz the power, 2014-02-20 20:11:52]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/mad@chromium.org/165893002/760001

[commit-bot: I haz the power, 2014-02-20 23:47:59]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-02-20 23:47:59]

Retried try job too often on win_rel for step(s) browser_tests
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=win_rel&nu...

[MAD, 2014-02-20 23:53:55]

The CQ bit was checked by mad@chromium.org

[commit-bot: I haz the power, 2014-02-20 23:56:39]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/mad@chromium.org/165893002/760001

[commit-bot: I haz the power, 2014-02-21 03:14:10]

Change committed as 252481