Allow creation and modification of scopes in GN.

Allow creation and modification of scopes in GN.

Previously scope variables could only be created by exec_script() and read_file() functions. This patch adds the following syntax:
  foo = { a = 1 b = 2 }
  foo.a += 1
  foo.c = 3
This will be used in a followup for toolchain args for non-default toolchains, which will allow us to generalize the templates as the number of configurations grows over time (currently every variable has be to be forwarded in the root toolchain templates).

It also allows mutations of list elements which isn't actually necessary for this but the implementation is the same as for scope mutations:
  list[0] = "foo"

This does a significant rework of the operator implementation. This was required for the more general "destination" for assignments and mutations.

It also updates the operator functions to use move semantics more. Previously:
  list3 = list1 + list2
Would be implemented as:
  1. Copy list1 into new new variable.
  2. Copy list2 into a new variable.
  3. Copy list1's copy to a new variable.
  4. Append elements of list2 to list1's copy by copying.
  5. Copy that list to list3.
That's a lot of copies! The new implementation does:
  1. Copy list1 into a new variable.
  2. Copy list2 into a new variable.
  3. Append elements of list2's copy to list1's copy by moving.
  4. Move list2 to list3.
Assuming the lists contain strings, each string should now be copied exactly once rather than three times.

Added a lot of documentation to the grammar help on how the language works (not strictly grammar but it seems like the best place to put it.

BUG=

Committed: https://crrev.com/01cdea1ed9ad32a7a595ef1a5adfa384006e3097
Cr-Commit-Position: refs/heads/master@{#409672}

[brettw, 2016-08-01 23:40:41]

.

[brettw, 2016-08-02 00:02:26]

.

[brettw, 2016-08-02 05:26:32]

The CQ bit was checked by brettw@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-08-02 05:26:38]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-08-02 05:31:47]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-08-02 05:31:48]

Dry run: Try jobs failed on following builders:
  chromium_presubmit on master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[brettw, 2016-08-02 16:36:34]

Description was changed from

==========
Allow creation and modification of scopes in GN

BUG=
==========

to

==========
Allow creation and modification of scopes in GN.

Previously scope variables could only be created by exec_script() and
read_file() functions. This patch adds the following syntax:
  foo = { a = 1 b = 2 }
  foo.a += 1
  foo.c = 3

It also allows mutations of list elements which isn't actually necessary for
this but the implementation is the same as for scope mutations:
  list[0] = "foo"

This does a significant rework of the operator implementation. This was required
for the more general "destination" for assignments and mutations.

It also updates the operator functions to use move semantics more. Previously:
  list3 = list1 + list2
Would be implemented as:
  1. Copy list1 into new new variable.
  2. Copy list2 into a new variable.
  3. Copy list1's copy to a new variable.
  4. Append elements of list2 to list1's copy by copying.
  5. Copy that list to list3.
That's a lot of copies! The new implementation does:
  1. Copy list1 into a new variable.
  2. Copy list2 into a new variable.
  3. Append elements of list2's copy to list1's copy by moving.
  4. Move list2 to list3.
Assuming the lists contain strings, each string should now be copied exactly
once rather than three times.


BUG=
==========

[brettw, 2016-08-02 16:44:25]

Description was changed from

==========
Allow creation and modification of scopes in GN.

Previously scope variables could only be created by exec_script() and
read_file() functions. This patch adds the following syntax:
  foo = { a = 1 b = 2 }
  foo.a += 1
  foo.c = 3

It also allows mutations of list elements which isn't actually necessary for
this but the implementation is the same as for scope mutations:
  list[0] = "foo"

This does a significant rework of the operator implementation. This was required
for the more general "destination" for assignments and mutations.

It also updates the operator functions to use move semantics more. Previously:
  list3 = list1 + list2
Would be implemented as:
  1. Copy list1 into new new variable.
  2. Copy list2 into a new variable.
  3. Copy list1's copy to a new variable.
  4. Append elements of list2 to list1's copy by copying.
  5. Copy that list to list3.
That's a lot of copies! The new implementation does:
  1. Copy list1 into a new variable.
  2. Copy list2 into a new variable.
  3. Append elements of list2's copy to list1's copy by moving.
  4. Move list2 to list3.
Assuming the lists contain strings, each string should now be copied exactly
once rather than three times.


BUG=
==========

to

==========
Allow creation and modification of scopes in GN.

Previously scope variables could only be created by exec_script() and
read_file() functions. This patch adds the following syntax:
  foo = { a = 1 b = 2 }
  foo.a += 1
  foo.c = 3

It also allows mutations of list elements which isn't actually necessary for
this but the implementation is the same as for scope mutations:
  list[0] = "foo"

This does a significant rework of the operator implementation. This was required
for the more general "destination" for assignments and mutations.

It also updates the operator functions to use move semantics more. Previously:
  list3 = list1 + list2
Would be implemented as:
  1. Copy list1 into new new variable.
  2. Copy list2 into a new variable.
  3. Copy list1's copy to a new variable.
  4. Append elements of list2 to list1's copy by copying.
  5. Copy that list to list3.
That's a lot of copies! The new implementation does:
  1. Copy list1 into a new variable.
  2. Copy list2 into a new variable.
  3. Append elements of list2's copy to list1's copy by moving.
  4. Move list2 to list3.
Assuming the lists contain strings, each string should now be copied exactly
once rather than three times.

Added a lot of documentation to the grammar help on how the language works (not
strictly grammar but it seems like the best place to put it.

BUG=
==========

[brettw, 2016-08-02 16:58:41]

Cleanup

[brettw, 2016-08-02 18:03:49]

The CQ bit was checked by brettw@chromium.org to run a CQ dry run

[brettw, 2016-08-02 18:03:56]

brettw@chromium.org changed reviewers:
+ dpranke@chromium.org

[commit-bot: I haz the power, 2016-08-02 18:04:15]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[brettw, 2016-08-02 18:08:12]

Description was changed from

==========
Allow creation and modification of scopes in GN.

Previously scope variables could only be created by exec_script() and
read_file() functions. This patch adds the following syntax:
  foo = { a = 1 b = 2 }
  foo.a += 1
  foo.c = 3

It also allows mutations of list elements which isn't actually necessary for
this but the implementation is the same as for scope mutations:
  list[0] = "foo"

This does a significant rework of the operator implementation. This was required
for the more general "destination" for assignments and mutations.

It also updates the operator functions to use move semantics more. Previously:
  list3 = list1 + list2
Would be implemented as:
  1. Copy list1 into new new variable.
  2. Copy list2 into a new variable.
  3. Copy list1's copy to a new variable.
  4. Append elements of list2 to list1's copy by copying.
  5. Copy that list to list3.
That's a lot of copies! The new implementation does:
  1. Copy list1 into a new variable.
  2. Copy list2 into a new variable.
  3. Append elements of list2's copy to list1's copy by moving.
  4. Move list2 to list3.
Assuming the lists contain strings, each string should now be copied exactly
once rather than three times.

Added a lot of documentation to the grammar help on how the language works (not
strictly grammar but it seems like the best place to put it.

BUG=
==========

to

==========
Allow creation and modification of scopes in GN.

Previously scope variables could only be created by exec_script() and
read_file() functions. This patch adds the following syntax:
  foo = { a = 1 b = 2 }
  foo.a += 1
  foo.c = 3
This will be used in a followup for toolchain args for non-default toolchains,
which will allow us to generalize the templates as the number of configurations
grows over time (currently every variable has be to be forwarded in the root
toolchain templates).

It also allows mutations of list elements which isn't actually necessary for
this but the implementation is the same as for scope mutations:
  list[0] = "foo"

This does a significant rework of the operator implementation. This was required
for the more general "destination" for assignments and mutations.

It also updates the operator functions to use move semantics more. Previously:
  list3 = list1 + list2
Would be implemented as:
  1. Copy list1 into new new variable.
  2. Copy list2 into a new variable.
  3. Copy list1's copy to a new variable.
  4. Append elements of list2 to list1's copy by copying.
  5. Copy that list to list3.
That's a lot of copies! The new implementation does:
  1. Copy list1 into a new variable.
  2. Copy list2 into a new variable.
  3. Append elements of list2's copy to list1's copy by moving.
  4. Move list2 to list3.
Assuming the lists contain strings, each string should now be copied exactly
once rather than three times.

Added a lot of documentation to the grammar help on how the language works (not
strictly grammar but it seems like the best place to put it.

BUG=
==========

[commit-bot: I haz the power, 2016-08-02 18:50:34]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-08-02 18:50:35]

Dry run: This issue passed the CQ dry run.

[Dirk Pranke, 2016-08-03 16:51:31]

Can you point to a CL or something that shows how the toolchain_args() code will
change? It would be good to see a practical illustration of this.

[brettw, 2016-08-03 16:58:11]

I don't have a CL yet, but it's easy. The toolchain args will look like this:
  
  toolchain_args = {
    current_cpu = "foo"
    current_os = "os360"
  }

and the gcc_template will look like:

  forward_variables_from(invoker, [ "toolchain_args" ])

  # Disable sanitizers for non-default toolchains.
  toolchain_args.is_asan = false
  toolchain_args.is_cfi = false
  ...

[Dirk Pranke, 2016-08-03 23:06:20]

lgtm w/ nits.

https://codereview.chromium.org/2187523003/diff/160001/tools/gn/operators.cc
File tools/gn/operators.cc (right):

https://codereview.chromium.org/2187523003/diff/160001/tools/gn/operators.cc#...
tools/gn/operators.cc:45: Value* GetExistingMutableValue(const ParseNode*
origin);
Nit: would GetExistingValueIfMutable() be a better name ? Does this return null
if the value exists but isn't writable (and, if so, when would that happen)?

https://codereview.chromium.org/2187523003/diff/160001/tools/gn/parser.cc
File tools/gn/parser.cc (right):

https://codereview.chromium.org/2187523003/diff/160001/tools/gn/parser.cc#new...
tools/gn/parser.cc:147: "   - Lists: Lists are arbitrary-length ordered lists of
any variable.\n"
nit: "lists of values" rather than "lists of any variable" ?

https://codereview.chromium.org/2187523003/diff/160001/tools/gn/parser.cc#new...
tools/gn/parser.cc:172: "  exist. This is to prevent common typos and to detect
dead code that\n"
nit: "doesn't exist" might be clearer as "not in the list"

https://codereview.chromium.org/2187523003/diff/160001/tools/gn/parser.cc#new...
tools/gn/parser.cc:190: "  current state like variables. With the exception of
loops and\n"
nit: add parens around "like variables"?

[brettw, 2016-08-03 23:23:00]

https://codereview.chromium.org/2187523003/diff/160001/tools/gn/operators.cc
File tools/gn/operators.cc (right):

https://codereview.chromium.org/2187523003/diff/160001/tools/gn/operators.cc#...
tools/gn/operators.cc:45: Value* GetExistingMutableValue(const ParseNode*
origin);
On 2016/08/03 23:06:19, Dirk Pranke wrote:
> Nit: would GetExistingValueIfMutable() be a better name ? Does this return
null
> if the value exists but isn't writable (and, if so, when would that happen)?

Done.

https://codereview.chromium.org/2187523003/diff/160001/tools/gn/parser.cc
File tools/gn/parser.cc (right):

https://codereview.chromium.org/2187523003/diff/160001/tools/gn/parser.cc#new...
tools/gn/parser.cc:147: "   - Lists: Lists are arbitrary-length ordered lists of
any variable.\n"
On 2016/08/03 23:06:19, Dirk Pranke wrote:
> nit: "lists of values" rather than "lists of any variable" ?

Done.

https://codereview.chromium.org/2187523003/diff/160001/tools/gn/parser.cc#new...
tools/gn/parser.cc:172: "  exist. This is to prevent common typos and to detect
dead code that\n"
On 2016/08/03 23:06:19, Dirk Pranke wrote:
> nit: "doesn't exist" might be clearer as "not in the list"

Done.

https://codereview.chromium.org/2187523003/diff/160001/tools/gn/parser.cc#new...
tools/gn/parser.cc:190: "  current state like variables. With the exception of
loops and\n"
On 2016/08/03 23:06:19, Dirk Pranke wrote:
> nit: add parens around "like variables"?

Done.

[brettw, 2016-08-03 23:23:52]

Review comments

[brettw, 2016-08-03 23:23:53]

The CQ bit was checked by brettw@chromium.org

[brettw, 2016-08-03 23:23:53]

The patchset sent to the CQ was uploaded after l-g-t-m from dpranke@chromium.org
Link to the patchset: https://codereview.chromium.org/2187523003/#ps180001
(title: "Review comments")

[commit-bot: I haz the power, 2016-08-03 23:24:21]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-08-04 00:00:49]

Description was changed from

==========
Allow creation and modification of scopes in GN.

Previously scope variables could only be created by exec_script() and
read_file() functions. This patch adds the following syntax:
  foo = { a = 1 b = 2 }
  foo.a += 1
  foo.c = 3
This will be used in a followup for toolchain args for non-default toolchains,
which will allow us to generalize the templates as the number of configurations
grows over time (currently every variable has be to be forwarded in the root
toolchain templates).

It also allows mutations of list elements which isn't actually necessary for
this but the implementation is the same as for scope mutations:
  list[0] = "foo"

This does a significant rework of the operator implementation. This was required
for the more general "destination" for assignments and mutations.

It also updates the operator functions to use move semantics more. Previously:
  list3 = list1 + list2
Would be implemented as:
  1. Copy list1 into new new variable.
  2. Copy list2 into a new variable.
  3. Copy list1's copy to a new variable.
  4. Append elements of list2 to list1's copy by copying.
  5. Copy that list to list3.
That's a lot of copies! The new implementation does:
  1. Copy list1 into a new variable.
  2. Copy list2 into a new variable.
  3. Append elements of list2's copy to list1's copy by moving.
  4. Move list2 to list3.
Assuming the lists contain strings, each string should now be copied exactly
once rather than three times.

Added a lot of documentation to the grammar help on how the language works (not
strictly grammar but it seems like the best place to put it.

BUG=
==========

to

==========
Allow creation and modification of scopes in GN.

Previously scope variables could only be created by exec_script() and
read_file() functions. This patch adds the following syntax:
  foo = { a = 1 b = 2 }
  foo.a += 1
  foo.c = 3
This will be used in a followup for toolchain args for non-default toolchains,
which will allow us to generalize the templates as the number of configurations
grows over time (currently every variable has be to be forwarded in the root
toolchain templates).

It also allows mutations of list elements which isn't actually necessary for
this but the implementation is the same as for scope mutations:
  list[0] = "foo"

This does a significant rework of the operator implementation. This was required
for the more general "destination" for assignments and mutations.

It also updates the operator functions to use move semantics more. Previously:
  list3 = list1 + list2
Would be implemented as:
  1. Copy list1 into new new variable.
  2. Copy list2 into a new variable.
  3. Copy list1's copy to a new variable.
  4. Append elements of list2 to list1's copy by copying.
  5. Copy that list to list3.
That's a lot of copies! The new implementation does:
  1. Copy list1 into a new variable.
  2. Copy list2 into a new variable.
  3. Append elements of list2's copy to list1's copy by moving.
  4. Move list2 to list3.
Assuming the lists contain strings, each string should now be copied exactly
once rather than three times.

Added a lot of documentation to the grammar help on how the language works (not
strictly grammar but it seems like the best place to put it.

BUG=
==========

[commit-bot: I haz the power, 2016-08-04 00:00:51]

Committed patchset #10 (id:180001)

[commit-bot: I haz the power, 2016-08-04 00:04:03]

Description was changed from

==========
Allow creation and modification of scopes in GN.

Previously scope variables could only be created by exec_script() and
read_file() functions. This patch adds the following syntax:
  foo = { a = 1 b = 2 }
  foo.a += 1
  foo.c = 3
This will be used in a followup for toolchain args for non-default toolchains,
which will allow us to generalize the templates as the number of configurations
grows over time (currently every variable has be to be forwarded in the root
toolchain templates).

It also allows mutations of list elements which isn't actually necessary for
this but the implementation is the same as for scope mutations:
  list[0] = "foo"

This does a significant rework of the operator implementation. This was required
for the more general "destination" for assignments and mutations.

It also updates the operator functions to use move semantics more. Previously:
  list3 = list1 + list2
Would be implemented as:
  1. Copy list1 into new new variable.
  2. Copy list2 into a new variable.
  3. Copy list1's copy to a new variable.
  4. Append elements of list2 to list1's copy by copying.
  5. Copy that list to list3.
That's a lot of copies! The new implementation does:
  1. Copy list1 into a new variable.
  2. Copy list2 into a new variable.
  3. Append elements of list2's copy to list1's copy by moving.
  4. Move list2 to list3.
Assuming the lists contain strings, each string should now be copied exactly
once rather than three times.

Added a lot of documentation to the grammar help on how the language works (not
strictly grammar but it seems like the best place to put it.

BUG=
==========

to

==========
Allow creation and modification of scopes in GN.

Previously scope variables could only be created by exec_script() and
read_file() functions. This patch adds the following syntax:
  foo = { a = 1 b = 2 }
  foo.a += 1
  foo.c = 3
This will be used in a followup for toolchain args for non-default toolchains,
which will allow us to generalize the templates as the number of configurations
grows over time (currently every variable has be to be forwarded in the root
toolchain templates).

It also allows mutations of list elements which isn't actually necessary for
this but the implementation is the same as for scope mutations:
  list[0] = "foo"

This does a significant rework of the operator implementation. This was required
for the more general "destination" for assignments and mutations.

It also updates the operator functions to use move semantics more. Previously:
  list3 = list1 + list2
Would be implemented as:
  1. Copy list1 into new new variable.
  2. Copy list2 into a new variable.
  3. Copy list1's copy to a new variable.
  4. Append elements of list2 to list1's copy by copying.
  5. Copy that list to list3.
That's a lot of copies! The new implementation does:
  1. Copy list1 into a new variable.
  2. Copy list2 into a new variable.
  3. Append elements of list2's copy to list1's copy by moving.
  4. Move list2 to list3.
Assuming the lists contain strings, each string should now be copied exactly
once rather than three times.

Added a lot of documentation to the grammar help on how the language works (not
strictly grammar but it seems like the best place to put it.

BUG=

Committed: https://crrev.com/01cdea1ed9ad32a7a595ef1a5adfa384006e3097
Cr-Commit-Position: refs/heads/master@{#409672}
==========

[commit-bot: I haz the power, 2016-08-04 00:04:04]

Patchset 10 (id:??) landed as
https://crrev.com/01cdea1ed9ad32a7a595ef1a5adfa384006e3097
Cr-Commit-Position: refs/heads/master@{#409672}