Update third_party/protobuf to protobuf-v3.0.0-beta-3

Update third_party/protobuf to protobuf-v3.0.0-beta-3

This is a minor update that incorporates some of our local patches. It also
introduces lite support for proto3 language.

It also incorporates a patch that reduces amount of memory
used by proto messages, at a cost of increased binary size.

For mobile, the binary size grows by ~20 KB with significant memory usage reduction:

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05-27_17-22-13

For desktop, the binary size grows by ~200 KB with similar reduction.

BUG=607751
Committed: https://crrev.com/a5b3d51ac5c84564b9671fbdb6afb54a527ebd90
Cr-Commit-Position: refs/heads/master@{#397491}

[xyzzyz, 2016-05-20 21:23:01]

Description was changed from

==========
Update third_party/protobuf to protobuf-v3.0.0-beta-3

This is a minor update that incorporates some of our local patches. It also
introduces lite support for proto3 language.

BUG=607751
==========

to

==========
Hi Peter, I'd like to ask you to review the protobuf update change:

Update third_party/protobuf to protobuf-v3.0.0-beta-3

This is a minor update that incorporates some of our local patches. It also
introduces lite support for proto3 language.

This CL differs from the upstream release only by the virtue of having applied
patches from patches/, and the following files:

xyzzyz@xyzzyz:/tmp$ diff -ur protobuf-3.0.0-beta-3/
~/chromium/src/third_party/protobuf
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
BUILD.gn
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf: DEPS
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
__init__.py
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
protobuf.gyp
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
protobuf_lite.gypi
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
protobuf_nacl.gyp
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
proto_library.gni
Only in
/usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf/python/google/protobuf:
descriptor_pb2.py
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
README.chromium
Only in protobuf-3.0.0-beta-3/src/google/protobuf/testdata: bad_utf8_string
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
third_party

I updated README.chromium, removed some of our local patches from the previous
update that have been fixed in upstream, and added patches for bugs introduced
in new release.

Files that are worth reviewing are new patches in patches/, README.chromium, and
changes in protobuf.gyp, protobuf_lite.gypi and BUILD.gn

BUG=607751
==========

[xyzzyz, 2016-05-20 21:23:05]

xyzzyz@chromium.org changed reviewers:
+ pkasting@chromium.org

[xyzzyz, 2016-05-20 22:20:23]

Description was changed from

==========
Hi Peter, I'd like to ask you to review the protobuf update change:

Update third_party/protobuf to protobuf-v3.0.0-beta-3

This is a minor update that incorporates some of our local patches. It also
introduces lite support for proto3 language.

This CL differs from the upstream release only by the virtue of having applied
patches from patches/, and the following files:

xyzzyz@xyzzyz:/tmp$ diff -ur protobuf-3.0.0-beta-3/
~/chromium/src/third_party/protobuf
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
BUILD.gn
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf: DEPS
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
__init__.py
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
protobuf.gyp
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
protobuf_lite.gypi
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
protobuf_nacl.gyp
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
proto_library.gni
Only in
/usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf/python/google/protobuf:
descriptor_pb2.py
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
README.chromium
Only in protobuf-3.0.0-beta-3/src/google/protobuf/testdata: bad_utf8_string
Only in /usr/local/google/home/xyzzyz/chromium/src/third_party/protobuf:
third_party

I updated README.chromium, removed some of our local patches from the previous
update that have been fixed in upstream, and added patches for bugs introduced
in new release.

Files that are worth reviewing are new patches in patches/, README.chromium, and
changes in protobuf.gyp, protobuf_lite.gypi and BUILD.gn

BUG=607751
==========

to

==========
Update third_party/protobuf to protobuf-v3.0.0-beta-3

This is a minor update that incorporates some of our local patches. It also
introduces lite support for proto3 language.

BUG=607751
==========

[Peter Kasting, 2016-05-24 23:45:34]

LGTM

https://codereview.chromium.org/1983203003/diff/70001/third_party/protobuf/RE...
File third_party/protobuf/README.chromium (right):

https://codereview.chromium.org/1983203003/diff/70001/third_party/protobuf/RE...
third_party/protobuf/README.chromium:59: Chrome workloads don't seem to be
benefitting from this.
Did you determine the actual memory savings from this?

[xyzzyz, 2016-05-25 00:15:28]

https://codereview.chromium.org/1983203003/diff/70001/third_party/protobuf/RE...
File third_party/protobuf/README.chromium (right):

https://codereview.chromium.org/1983203003/diff/70001/third_party/protobuf/RE...
third_party/protobuf/README.chromium:59: Chrome workloads don't seem to be
benefitting from this.
On 2016/05/24 23:45:34, Peter Kasting wrote:
> Did you determine the actual memory savings from this?

I tried, but the benchmarking results are really fuzzy.

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...
http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...

One is with that patch kept, the other with reverted. Try guessing which is
which (be sure to also click on "memory" at the top to show all cases). Note
that ToT itself has bigger variance.

[Peter Kasting, 2016-05-25 00:19:50]

https://codereview.chromium.org/1983203003/diff/70001/third_party/protobuf/RE...
File third_party/protobuf/README.chromium (right):

https://codereview.chromium.org/1983203003/diff/70001/third_party/protobuf/RE...
third_party/protobuf/README.chromium:59: Chrome workloads don't seem to be
benefitting from this.
On 2016/05/25 00:15:28, xyzzyz wrote:
> On 2016/05/24 23:45:34, Peter Kasting wrote:
> > Did you determine the actual memory savings from this?
> 
> I tried, but the benchmarking results are really fuzzy.
> 
>
http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...
>
http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...
> 
> One is with that patch kept, the other with reverted. Try guessing which is
> which (be sure to also click on "memory" at the top to show all cases). Note
> that ToT itself has bigger variance.

To me the second link (05-20) looks fairly consistently better than the first
link (05-19).

Which is which?

[xyzzyz, 2016-05-25 00:25:47]

https://codereview.chromium.org/1983203003/diff/70001/third_party/protobuf/RE...
File third_party/protobuf/README.chromium (right):

https://codereview.chromium.org/1983203003/diff/70001/third_party/protobuf/RE...
third_party/protobuf/README.chromium:59: Chrome workloads don't seem to be
benefitting from this.
On 2016/05/25 00:19:50, Peter Kasting wrote:
> On 2016/05/25 00:15:28, xyzzyz wrote:
> > On 2016/05/24 23:45:34, Peter Kasting wrote:
> > > Did you determine the actual memory savings from this?
> > 
> > I tried, but the benchmarking results are really fuzzy.
> > 
> >
>
http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...
> >
>
http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...
> > 
> > One is with that patch kept, the other with reverted. Try guessing which is
> > which (be sure to also click on "memory" at the top to show all cases). Note
> > that ToT itself has bigger variance.
> 
> To me the second link (05-20) looks fairly consistently better than the first
> link (05-19).
> 
> Which is which?

The 05-20 is the optimize-memory-with-larger-binary-size one. Why would you say
it's better, though?

[Peter Kasting, 2016-05-25 01:17:56]

https://codereview.chromium.org/1983203003/diff/70001/third_party/protobuf/RE...
File third_party/protobuf/README.chromium (right):

https://codereview.chromium.org/1983203003/diff/70001/third_party/protobuf/RE...
third_party/protobuf/README.chromium:59: Chrome workloads don't seem to be
benefitting from this.
On 2016/05/25 00:25:47, xyzzyz wrote:
> On 2016/05/25 00:19:50, Peter Kasting wrote:
> > On 2016/05/25 00:15:28, xyzzyz wrote:
> > > On 2016/05/24 23:45:34, Peter Kasting wrote:
> > > > Did you determine the actual memory savings from this?
> > > 
> > > I tried, but the benchmarking results are really fuzzy.
> > > 
> > >
> >
>
http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...
> > >
> >
>
http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...
> > > 
> > > One is with that patch kept, the other with reverted. Try guessing which
is
> > > which (be sure to also click on "memory" at the top to show all cases).
Note
> > > that ToT itself has bigger variance.
> > 
> > To me the second link (05-20) looks fairly consistently better than the
first
> > link (05-19).
> > 
> > Which is which?
> 
> The 05-20 is the optimize-memory-with-larger-binary-size one. Why would you
say
> it's better, though? 

Every relevant browser memory stat is improved compared to the other run:
* Commit charge is 5 MB lower
* Peak RSS is 1.25 MB lower
* Final VM size is 8 MB lower
* Peak VM size is 6 MB lower
* Dirty private VM size is 1.5 MB lower
* Final VM RSS size is 1.5 MB lower

Most of these also show improvements from one run to the other if viewed as
"delta from TOT" instead of as absolute values.  (It's a little confusing to
read the chart because the "delta" column shows how much better or worse TOT is
compared to the patch instead of the reverse, so if that column gets "worse",
that actually means the patch did better at improving memory use.)  The only one
that doesn't is the overall commit charge, and I'm suspicious of the TOT number
there because it dropped by an incredible amount compared to the 5-19 run even
though the other numbers didn't really move at all.

So to me this seems like a clear win for the "decrease memory use" patch and I'd
leave it in the codebase.  I think it's worth the 200K binary size increase.

[xyzzyz, 2016-05-31 22:11:32]

On 2016/05/25 01:17:56, Peter Kasting wrote:
>
https://codereview.chromium.org/1983203003/diff/70001/third_party/protobuf/RE...
> File third_party/protobuf/README.chromium (right):
> 
>
https://codereview.chromium.org/1983203003/diff/70001/third_party/protobuf/RE...
> third_party/protobuf/README.chromium:59: Chrome workloads don't seem to be
> benefitting from this.
> On 2016/05/25 00:25:47, xyzzyz wrote:
> > On 2016/05/25 00:19:50, Peter Kasting wrote:
> > > On 2016/05/25 00:15:28, xyzzyz wrote:
> > > > On 2016/05/24 23:45:34, Peter Kasting wrote:
> > > > > Did you determine the actual memory savings from this?
> > > > 
> > > > I tried, but the benchmarking results are really fuzzy.
> > > > 
> > > >
> > >
> >
>
http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...
> > > >
> > >
> >
>
http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...
> > > > 
> > > > One is with that patch kept, the other with reverted. Try guessing which
> is
> > > > which (be sure to also click on "memory" at the top to show all cases).
> Note
> > > > that ToT itself has bigger variance.
> > > 
> > > To me the second link (05-20) looks fairly consistently better than the
> first
> > > link (05-19).
> > > 
> > > Which is which?
> > 
> > The 05-20 is the optimize-memory-with-larger-binary-size one. Why would you
> say
> > it's better, though? 
> 
> Every relevant browser memory stat is improved compared to the other run:
> * Commit charge is 5 MB lower
> * Peak RSS is 1.25 MB lower
> * Final VM size is 8 MB lower
> * Peak VM size is 6 MB lower
> * Dirty private VM size is 1.5 MB lower
> * Final VM RSS size is 1.5 MB lower
> 
> Most of these also show improvements from one run to the other if viewed as
> "delta from TOT" instead of as absolute values.  (It's a little confusing to
> read the chart because the "delta" column shows how much better or worse TOT
is
> compared to the patch instead of the reverse, so if that column gets "worse",
> that actually means the patch did better at improving memory use.)  The only
one
> that doesn't is the overall commit charge, and I'm suspicious of the TOT
number
> there because it dropped by an incredible amount compared to the 5-19 run even
> though the other numbers didn't really move at all.
> 
> So to me this seems like a clear win for the "decrease memory use" patch and
I'd
> leave it in the codebase.  I think it's worth the 200K binary size increase.

Just for the record, I run another set of benchmark, and it seems that the
memory use reduction is indeed significant.

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...
http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...

[xyzzyz, 2016-05-31 23:24:32]

The CQ bit was checked by xyzzyz@chromium.org

[xyzzyz, 2016-05-31 23:24:35]

The CQ bit was unchecked by xyzzyz@chromium.org

[xyzzyz, 2016-05-31 23:24:36]

The patchset sent to the CQ was uploaded after l-g-t-m from
pkasting@chromium.org
Link to the patchset: https://codereview.chromium.org/1983203003/#ps90001
(title: "add save memory patch")

[xyzzyz, 2016-05-31 23:25:17]

The CQ bit was unchecked by xyzzyz@chromium.org

[commit-bot: I haz the power, 2016-05-31 23:25:47]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1983203003/90001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1983203003/90001

[xyzzyz, 2016-05-31 23:27:51]

The CQ bit was checked by xyzzyz@chromium.org

[commit-bot: I haz the power, 2016-05-31 23:28:46]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1983203003/90001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1983203003/90001

[commit-bot: I haz the power, 2016-05-31 23:52:55]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-05-31 23:53:00]

Try jobs failed on following builders:
  chromium_presubmit on tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[xyzzyz, 2016-06-01 20:50:03]

The CQ bit was checked by xyzzyz@chromium.org

[commit-bot: I haz the power, 2016-06-01 20:50:34]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1983203003/90001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1983203003/90001

[commit-bot: I haz the power, 2016-06-01 21:17:03]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-06-01 21:17:09]

Try jobs failed on following builders:
  chromium_presubmit on tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[Peter Kasting, 2016-06-01 22:11:16]

LGTM again.

If it fails again, I would commit with no presubmit.

[xyzzyz, 2016-06-01 22:17:36]

The CQ bit was checked by xyzzyz@chromium.org

[commit-bot: I haz the power, 2016-06-01 22:18:43]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1983203003/90001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1983203003/90001

[commit-bot: I haz the power, 2016-06-01 23:14:47]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-06-01 23:14:54]

Try jobs failed on following builders:
  chromium_presubmit on tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[xyzzyz, 2016-06-01 23:29:06]

Description was changed from

==========
Update third_party/protobuf to protobuf-v3.0.0-beta-3

This is a minor update that incorporates some of our local patches. It also
introduces lite support for proto3 language.

BUG=607751
==========

to

==========
Update third_party/protobuf to protobuf-v3.0.0-beta-3

This is a minor update that incorporates some of our local patches. It also
introduces lite support for proto3 language.

It also incorporates a patch that reduces amount of memory 
used by proto messages, at a cost of increased binary size. 

For mobile, the binary size grows by ~20 KB with significant memory usage
reduction:

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...

For desktop, the binary size grows by ~200 KB with similar reduction.

BUG=607751
==========

[xyzzyz, 2016-06-01 23:33:21]

On 2016/06/01 22:11:16, Peter Kasting wrote:
> LGTM again.
> 
> If it fails again, I would commit with no presubmit.

It looks like presubmit worked properly -- the OWNERS file got MIA.

[xyzzyz, 2016-06-01 23:35:11]

The CQ bit was checked by xyzzyz@chromium.org

[xyzzyz, 2016-06-01 23:35:15]

The patchset sent to the CQ was uploaded after l-g-t-m from
pkasting@chromium.org
Link to the patchset: https://codereview.chromium.org/1983203003/#ps100001
(title: "owners")

[commit-bot: I haz the power, 2016-06-01 23:36:10]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1983203003/100001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1983203003/100001

[commit-bot: I haz the power, 2016-06-02 01:26:49]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-06-02 01:26:54]

Try jobs failed on following builders:
  mac_chromium_rel_ng on tryserver.chromium.mac (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_...)

[Peter Kasting, 2016-06-02 04:47:16]

On 2016/06/01 23:33:21, xyzzyz wrote:
> On 2016/06/01 22:11:16, Peter Kasting wrote:
> > LGTM again.
> > 
> > If it fails again, I would commit with no presubmit.
> 
> It looks like presubmit worked properly -- the OWNERS file got MIA.

Well, that's embarrassing.  A good developer wouldn't have told you "submit
without presubmit" without having checked that first...

[xyzzyz, 2016-06-02 17:18:28]

The CQ bit was checked by xyzzyz@chromium.org

[commit-bot: I haz the power, 2016-06-02 17:19:24]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1983203003/100001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1983203003/100001

[commit-bot: I haz the power, 2016-06-02 19:58:20]

Description was changed from

==========
Update third_party/protobuf to protobuf-v3.0.0-beta-3

This is a minor update that incorporates some of our local patches. It also
introduces lite support for proto3 language.

It also incorporates a patch that reduces amount of memory 
used by proto messages, at a cost of increased binary size. 

For mobile, the binary size grows by ~20 KB with significant memory usage
reduction:

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...

For desktop, the binary size grows by ~200 KB with similar reduction.

BUG=607751
==========

to

==========
Update third_party/protobuf to protobuf-v3.0.0-beta-3

This is a minor update that incorporates some of our local patches. It also
introduces lite support for proto3 language.

It also incorporates a patch that reduces amount of memory 
used by proto messages, at a cost of increased binary size. 

For mobile, the binary size grows by ~20 KB with significant memory usage
reduction:

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...

For desktop, the binary size grows by ~200 KB with similar reduction.

BUG=607751
==========

[commit-bot: I haz the power, 2016-06-02 19:58:25]

Committed patchset #11 (id:100001)

[commit-bot: I haz the power, 2016-06-02 20:00:01]

Description was changed from

==========
Update third_party/protobuf to protobuf-v3.0.0-beta-3

This is a minor update that incorporates some of our local patches. It also
introduces lite support for proto3 language.

It also incorporates a patch that reduces amount of memory 
used by proto messages, at a cost of increased binary size. 

For mobile, the binary size grows by ~20 KB with significant memory usage
reduction:

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...

For desktop, the binary size grows by ~200 KB with similar reduction.

BUG=607751
==========

to

==========
Update third_party/protobuf to protobuf-v3.0.0-beta-3

This is a minor update that incorporates some of our local patches. It also
introduces lite support for proto3 language.

It also incorporates a patch that reduces amount of memory
used by proto messages, at a cost of increased binary size.

For mobile, the binary size grows by ~20 KB with significant memory usage
reduction:

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05...

For desktop, the binary size grows by ~200 KB with similar reduction.

BUG=607751

Committed: https://crrev.com/a5b3d51ac5c84564b9671fbdb6afb54a527ebd90
Cr-Commit-Position: refs/heads/master@{#397491}
==========

[commit-bot: I haz the power, 2016-06-02 20:00:07]

Patchset 11 (id:??) landed as
https://crrev.com/a5b3d51ac5c84564b9671fbdb6afb54a527ebd90
Cr-Commit-Position: refs/heads/master@{#397491}