|
|
DescriptionPreparing components_perftests for adding more benchmarks.
This patch creates a complete main for components perftests from
components unittests main and switches components perftests to using
testing/perf instead of base/perf.
BUG=648992
Committed: https://crrev.com/bd75dc348f2e52e1fc3e1b26b6c7528329657d65
Cr-Commit-Position: refs/heads/master@{#427043}
Patch Set 1 : Extracting new main for components_perftests #Patch Set 2 : Attempt to guess how generate_perf_json.py works #
Total comments: 1
Patch Set 3 : minimal chages #
Total comments: 6
Patch Set 4 : Review, round 1 #
Total comments: 2
Patch Set 5 : Review, round 2 #Patch Set 6 : Fixing dependencies #Patch Set 7 : Fixing android dependencies #Patch Set 8 : Fixing ios dependencies #
Total comments: 2
Messages
Total messages: 57 (27 generated)
Description was changed from ========== Creating a blank components perf test: 1 - creating main that does all the necessary initialization for components. 2 - deciding on perf frame work base/test/perf* vs testing/perf/* 3 - enabling a perf test in regular runs BUG=648992 ========== to ========== Creating a blank components perf test: 1 - creating main that does all the necessary initialization for components. 2 - deciding on perf frame work base/test/perf* vs testing/perf/* 3 - enabling a perf test in regular runs BUG=648992 ==========
dyaroshev@yandex-team.ru changed reviewers: + cpu@chromium.org, jochen@chromium.org, nduca@chromium.org, pkasting@google.com
Description was changed from ========== Creating a blank components perf test: 1 - creating main that does all the necessary initialization for components. 2 - deciding on perf frame work base/test/perf* vs testing/perf/* 3 - enabling a perf test in regular runs BUG=648992 ========== to ========== Creating a blank components perf test: 1 - creating main that does all the necessary initialization for components (taken from components_unittests) 2 - deciding on perf frame work base/test/perf* vs testing/perf/* 3 - enabling a perf test in regular runs BUG=648992 ==========
cpu@chromium.org changed reviewers: - cpu@chromium.org
dyaroshev@yandex-team.ru changed reviewers: + eyaich@google.com
dyaroshev@yandex-team.ru changed reviewers: + dtu@chromium.org
so, your'e going to hate me, but i have a lot of questions here, and a general comment. 1. Your CL is still doing too much at once, so youre going to get a lot of confused people who know one aspect of your CL who look at the other parts and either (a) say I can't review this, or (b) give you confusing feedback because they don't know the code. Myself included. You should split this more. The general princple I try to apply is that every patch is a complete sentence, not a paragraph, and definitely not an essay. Here, as far as i can tell, you have a paragraph: 1a. add components/test/components_test_suite.cc 1b. add components/test/run_all_perf_tests.cc and gn targets 1c. modify base/ for some specific reason that i don't understand but clearly has a reason. 1d. create the skeleton for the omnibox test 1e. make buildbot aware of the new componets perf unittests 1f. make perf_json_generator aware fo the new componetns perf unittests These are all patches in their own right. Each has a slightly different reviewer. My skillset is on 1d, I can somewhat review 1e and 1f. I am not the expert to review the other bits, nor am I owner in any of these places so my l-g-t is useless anyway. 3. You have a *test_suit.h and *test_suit.cc where I believe the correct filename is *test_suite.h and *test_suite.cc. 4. your ComponetsTestSuite::initialize seems very very long and its unclear to me which of those are standard boilerplate for components unit tests, and which are unique to your situation. Someone in components should review your suite instead of me and check that it is okay to have all this large amount of text there. Hope this helps. https://codereview.chromium.org/2358063002/diff/20001/components/test/compone... File components/test/components_test_suit.h (right): https://codereview.chromium.org/2358063002/diff/20001/components/test/compone... components/test/components_test_suit.h:1: // Copyright (c) 2016 The Chromium Authors. All rights reserved. At least from my perspective, this file should be called components_test_suite.h not components_test_suit.h.
On 2016/09/28 19:46:42, nduca wrote: > 4. your ComponetsTestSuite::initialize seems very very long and its unclear to > me which of those are standard boilerplate for components unit tests, and which > are unique to your situation. Someone in components should review your suite > instead of me and check that it is okay to have all this large amount of text > there. Nat asked me to glance at this one; I'm not an expert here either but my gut sense is that you don't need most of this and I would try to rip out everything, or at least as much as possible until something breaks.
On 2016/09/28 19:46:42, nduca wrote: > so, your'e going to hate me, but i have a lot of questions here, and a general > comment. > > 1. Your CL is still doing too much at once, so youre going to get a lot of > confused people who know one aspect of your CL who look at the other parts and > either (a) say I can't review this, or (b) give you confusing feedback because > they don't know the code. Myself included. You should split this more. The > general princple I try to apply is that every patch is a complete sentence, not > a paragraph, and definitely not an essay. Here, as far as i can tell, you have a > paragraph: > 1a. add components/test/components_test_suite.cc > 1b. add components/test/run_all_perf_tests.cc and gn targets > 1c. modify base/ for some specific reason that i don't understand but clearly > has a reason. > 1d. create the skeleton for the omnibox test > 1e. make buildbot aware of the new componets perf unittests > 1f. make perf_json_generator aware fo the new componetns perf unittests > > These are all patches in their own right. Each has a slightly different > reviewer. Do you mean that I have to split this up in different CLs, or different pathces would be sufficient? Most of these changes are just moving staff around so that components_perftests would work. > 3. You have a *test_suit.h and *test_suit.cc where I believe the correct > filename is *test_suite.h and *test_suite.cc. > My bad, spelling is not my strongest side. > 4. your ComponetsTestSuite::initialize seems very very long and its unclear to > me which of those are standard boilerplate for components unit tests, and which > are unique to your situation. Someone in components should review your suite > instead of me and check that it is okay to have all this large amount of text > there. > I just stole it from run_all_unittests, this is not what I know and it should not be me who refactors it.
On 2016/09/28 20:15:50, dyaroshev wrote: > > 4. your ComponetsTestSuite::initialize seems very very long and its unclear to > > me which of those are standard boilerplate for components unit tests, and > which > > are unique to your situation. Someone in components should review your suite > > instead of me and check that it is okay to have all this large amount of text > > there. > > > > I just stole it from run_all_unittests, this is not what I know and it should > not be me who refactors it. You don't need to refactor the existing code in any way, but you do need to avoid copying a bunch of functionality into brand-new code unless your test needs it.
On 2016/09/28 20:20:59, Peter Kasting wrote: > You don't need to refactor the existing code in any way, but you do need to > avoid copying a bunch of functionality into brand-new code unless your test > needs it. I didn't copy it, I extracted existing code in a new file. > Nat asked me to glance at this one; I'm not an expert here either but my gut > sense is that you don't need most of this and I would try to rip out everything, > or at least as much as possible until something breaks. 1 - I've tried to model after cc/ - all setup is moved to one place. This is actually a nice design feature because we won't have to think about two different places when modifing test suite. 2 - This would probably lead to bugs on different oses on which I don't even work (like IOS). Can I modify one existing component_perftest to use testing/perf instead of base/perf? This would make my patch cosiderably smaller: no changes in base, much less changes in cut and paste from run_components_unittests. And I won't have to create a new perf test file here - I can just use the existing one to test changes.
On 2016/09/28 21:09:20, dyaroshev wrote: > On 2016/09/28 20:20:59, Peter Kasting wrote: > > You don't need to refactor the existing code in any way, but you do need to > > avoid copying a bunch of functionality into brand-new code unless your test > > needs it. > > I didn't copy it, I extracted existing code in a new file. Oh, argh. I looked for that, and missed it. > > Nat asked me to glance at this one; I'm not an expert here either but my gut > > sense is that you don't need most of this and I would try to rip out > everything, > > or at least as much as possible until something breaks. > > 1 - I've tried to model after cc/ - all setup is moved to one place. This is > actually a nice design feature because we won't have to think about two > different places when modifing test suite. Well, it depends. If you have a unittest system that runs tens of thousands of tests with tons of dependencies and you're adding a perf test system that runs a very small number, then it seems less than ideal to hoist the former's dependencies so they're pulled in by the latter. > 2 - This would probably lead to bugs on different oses on which I don't even > work (like IOS). That's why we have trybots; if a change passes the tryjobs, it should be OK. If a platform isn't part of the try infrastructure and gets broken by a change, that's basically on them, not you. > Can I modify one existing component_perftest to use testing/perf instead of > base/perf? This would make my patch cosiderably smaller: no changes in base, > much less changes in cut and paste from run_components_unittests. And I won't > have to create a new perf test file here - I can just use the existing one to > test changes. I have no opinion on this.
pkasting@chromium.org changed reviewers: + pkasting@chromium.org - pkasting@google.com
Description was changed from ========== Creating a blank components perf test: 1 - creating main that does all the necessary initialization for components (taken from components_unittests) 2 - deciding on perf frame work base/test/perf* vs testing/perf/* 3 - enabling a perf test in regular runs BUG=648992 ========== to ========== This patch 1 - creates a complete main for components perftests 2 - decides on using testing/perf for components_perftests BUG=648992 ==========
On 2016/09/28 23:49:01, Peter Kasting (OOO Oct. 6-9) wrote: > > Can I modify one existing component_perftest to use testing/perf instead of > > base/perf? This would make my patch cosiderably smaller: no changes in base, > > much less changes in cut and paste from run_components_unittests. And I won't > > have to create a new perf test file here - I can just use the existing one to > > test changes. > > I have no opinion on this. I changed. It gives a few things: 1 - much smaller patch: no weird initialization for base/perf is required 2 - We've decided to use testing/perf for History perf tests. Now I don't have to come up with some blank test to see if this api is set up properly - we can use the existing one. I still have no idea how to properly enable it in regular runs. I suggest, we create a separate task on people who do that. We would still have a benchmark to test locally at the very least. Off-top: because of the amount of time I've spent on this benchmark, my superiors won't give me more until the actual flat maps. I'll do what I can in my spare time, but it seems like the progress would go very slow.
ping @pkasting, @nduca
I don't see major concerns, the .gn changes are the critical ones. https://codereview.chromium.org/2358063002/diff/40001/components/BUILD.gn File components/BUILD.gn (right): https://codereview.chromium.org/2358063002/diff/40001/components/BUILD.gn#new... components/BUILD.gn:41: "test/components_test_suite.h", I don't think you want these sources (and their dependencies) to appear in both this target and the one below. You probably want these, and their dependencies, in a source_set that both targets depend on. https://codereview.chromium.org/2358063002/diff/40001/components/test/compone... File components/test/components_test_suite.h (right): https://codereview.chromium.org/2358063002/diff/40001/components/test/compone... components/test/components_test_suite.h:1: // Copyright (c) 2016 The Chromium Authors. All rights reserved. Nit: No (c) (see https://chromium.googlesource.com/chromium/src/+/master/styleguide/c++/c++.md... ) (All added files) https://codereview.chromium.org/2358063002/diff/40001/components/visitedlink/... File components/visitedlink/test/visitedlink_perftest.cc (right): https://codereview.chromium.org/2358063002/diff/40001/components/visitedlink/... components/visitedlink/test/visitedlink_perftest.cc:27: // designed like base/test/perf_time_logger, but uses different output strategy. Nit: Initial capital You might want to describe what the output strategy is and/or why it wants to be different. https://codereview.chromium.org/2358063002/diff/40001/components/visitedlink/... components/visitedlink/test/visitedlink_perftest.cc:53: // integer milliseconds Nit: Add trailing period https://codereview.chromium.org/2358063002/diff/40001/components/visitedlink/... components/visitedlink/test/visitedlink_perftest.cc:54: perf_test::PrintResult(test_name_, "", "", timer_.Elapsed().InMillisecondsF(), Nit: "" -> std::string() https://codereview.chromium.org/2358063002/diff/40001/components/visitedlink/... components/visitedlink/test/visitedlink_perftest.cc:231: perf_test::PrintResult("Visited_link_cold_load_time", "", "", Nit: "" -> std::string (2 places)
On 2016/10/17 18:44:47, Peter Kasting wrote: > I don't see major concerns, the .gn changes are the critical ones. > > https://codereview.chromium.org/2358063002/diff/40001/components/BUILD.gn > File components/BUILD.gn (right): > > https://codereview.chromium.org/2358063002/diff/40001/components/BUILD.gn#new... > components/BUILD.gn:41: "test/components_test_suite.h", > I don't think you want these sources (and their dependencies) to appear in both > this target and the one below. > > You probably want these, and their dependencies, in a source_set that both > targets depend on. > > https://codereview.chromium.org/2358063002/diff/40001/components/test/compone... > File components/test/components_test_suite.h (right): > > https://codereview.chromium.org/2358063002/diff/40001/components/test/compone... > components/test/components_test_suite.h:1: // Copyright (c) 2016 The Chromium > Authors. All rights reserved. > Nit: No (c) (see > https://chromium.googlesource.com/chromium/src/+/master/styleguide/c++/c++.md... > ) > > (All added files) > > https://codereview.chromium.org/2358063002/diff/40001/components/visitedlink/... > File components/visitedlink/test/visitedlink_perftest.cc (right): > > https://codereview.chromium.org/2358063002/diff/40001/components/visitedlink/... > components/visitedlink/test/visitedlink_perftest.cc:27: // designed like > base/test/perf_time_logger, but uses different output strategy. > Nit: Initial capital > > You might want to describe what the output strategy is and/or why it wants to be > different. > > https://codereview.chromium.org/2358063002/diff/40001/components/visitedlink/... > components/visitedlink/test/visitedlink_perftest.cc:53: // integer milliseconds > Nit: Add trailing period > > https://codereview.chromium.org/2358063002/diff/40001/components/visitedlink/... > components/visitedlink/test/visitedlink_perftest.cc:54: > perf_test::PrintResult(test_name_, "", "", timer_.Elapsed().InMillisecondsF(), > Nit: "" -> std::string() > > https://codereview.chromium.org/2358063002/diff/40001/components/visitedlink/... > components/visitedlink/test/visitedlink_perftest.cc:231: > perf_test::PrintResult("Visited_link_cold_load_time", "", "", > Nit: "" -> std::string (2 places) I think I've fixed everything.
https://codereview.chromium.org/2358063002/diff/60001/components/BUILD.gn File components/BUILD.gn (right): https://codereview.chromium.org/2358063002/diff/60001/components/BUILD.gn#new... components/BUILD.gn:41: "test/components_test_suite.h", You still list these sources here and below, though they're now in your test_support library as well. Remove. https://codereview.chromium.org/2358063002/diff/60001/components/test/BUILD.gn File components/test/BUILD.gn (right): https://codereview.chromium.org/2358063002/diff/60001/components/test/BUILD.g... components/test/BUILD.gn:5: static_library("test_support") { I think you want a source_set here and not a static_library. If I'm wrong, please explain why :)
On 2016/10/18 17:31:13, Peter Kasting wrote: > https://codereview.chromium.org/2358063002/diff/60001/components/BUILD.gn > File components/BUILD.gn (right): > > https://codereview.chromium.org/2358063002/diff/60001/components/BUILD.gn#new... > components/BUILD.gn:41: "test/components_test_suite.h", > You still list these sources here and below, though they're now in your > test_support library as well. Remove. > > https://codereview.chromium.org/2358063002/diff/60001/components/test/BUILD.gn > File components/test/BUILD.gn (right): > > https://codereview.chromium.org/2358063002/diff/60001/components/test/BUILD.g... > components/test/BUILD.gn:5: static_library("test_support") { > I think you want a source_set here and not a static_library. If I'm wrong, > please explain why :) Done) Can we merge?
LGTM
The CQ bit was checked by dyaroshev@yandex-team.ru
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: ios-simulator on master.tryserver.chromium.mac (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.mac/builders/ios-simulator/bui...) mac_chromium_compile_dbg_ng on master.tryserver.chromium.mac (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_comp...)
The CQ bit was checked by dyaroshev@yandex-team.ru
The patchset sent to the CQ was uploaded after l-g-t-m from pkasting@chromium.org Link to the patchset: https://codereview.chromium.org/2358063002/#ps100001 (title: "Fixing dependencies")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: chromium_presubmit on master.tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)
@jochen - I really need your review for this cl.
The CQ bit was checked by dyaroshev@yandex-team.ru
The patchset sent to the CQ was uploaded after l-g-t-m from pkasting@chromium.org Link to the patchset: https://codereview.chromium.org/2358063002/#ps120001 (title: "Fixing android dependencies")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: ios-device on master.tryserver.chromium.mac (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.mac/builders/ios-device/builds...) ios-simulator on master.tryserver.chromium.mac (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.mac/builders/ios-simulator/bui...)
The CQ bit was checked by dyaroshev@yandex-team.ru
The patchset sent to the CQ was uploaded after l-g-t-m from pkasting@chromium.org Link to the patchset: https://codereview.chromium.org/2358063002/#ps140001 (title: "Fixing ios dependencies")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: chromium_presubmit on master.tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)
can you please format the CL description according to chromium.org/developers/contributing-code#Writing_change_list_descriptions right now, the commit would be titled "This patch".
Description was changed from ========== This patch 1 - creates a complete main for components perftests 2 - decides on using testing/perf for components_perftests BUG=648992 ========== to ========== Preparing components_perftests for adding more tests. 1 - creating a complete main for components perftests 2 - deciding on using testing/perf for components_perftests BUG=648992 ==========
Description was changed from ========== Preparing components_perftests for adding more tests. 1 - creating a complete main for components perftests 2 - deciding on using testing/perf for components_perftests BUG=648992 ========== to ========== Preparing components_perftests for adding more benchmarks. 1 - creating a complete main for components perftests 2 - deciding on using testing/perf for components_perftests BUG=648992 ==========
On 2016/10/21 11:02:20, jochen wrote: > can you please format the CL description according to > chromium.org/developers/contributing-code#Writing_change_list_descriptions > > right now, the commit would be titled "This patch". Done
Description was changed from ========== Preparing components_perftests for adding more benchmarks. 1 - creating a complete main for components perftests 2 - deciding on using testing/perf for components_perftests BUG=648992 ========== to ========== Preparing components_perftests for adding more benchmarks. This patch creates a complete main for components perftests from components unittests main and switches components perftests to using testing/perf instead of base/perf. BUG=648992 ==========
Description was changed from ========== Preparing components_perftests for adding more benchmarks. This patch creates a complete main for components perftests from components unittests main and switches components perftests to using testing/perf instead of base/perf. BUG=648992 ========== to ========== Preparing components_perftests for adding more benchmarks. This patch creates a complete main for components perftests from components unittests main and switches components perftests to using testing/perf instead of base/perf. BUG=648992 ==========
Description was changed from ========== Preparing components_perftests for adding more benchmarks. This patch creates a complete main for components perftests from components unittests main and switches components perftests to using testing/perf instead of base/perf. BUG=648992 ========== to ========== Preparing components_perftests for adding more benchmarks. This patch creates a complete main for components perftests from components unittests main and switches components perftests to using testing/perf instead of base/perf. BUG=648992 ==========
Drive-by nit https://codereview.chromium.org/2358063002/diff/140001/components/visitedlink... File components/visitedlink/test/visitedlink_perftest.cc (right): https://codereview.chromium.org/2358063002/diff/140001/components/visitedlink... components/visitedlink/test/visitedlink_perftest.cc:31: explicit TimeLogger(std::string test_name); Nit: const&
https://codereview.chromium.org/2358063002/diff/140001/components/visitedlink... File components/visitedlink/test/visitedlink_perftest.cc (right): https://codereview.chromium.org/2358063002/diff/140001/components/visitedlink... components/visitedlink/test/visitedlink_perftest.cc:31: explicit TimeLogger(std::string test_name); On 2016/10/21 19:01:46, Alexei Svitkine (slow) wrote: > Nit: const& Can't, he's doing std::move() below to move into the member.
lgtm
The CQ bit was checked by dyaroshev@yandex-team.ru
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
Message was sent while issue was closed.
Description was changed from ========== Preparing components_perftests for adding more benchmarks. This patch creates a complete main for components perftests from components unittests main and switches components perftests to using testing/perf instead of base/perf. BUG=648992 ========== to ========== Preparing components_perftests for adding more benchmarks. This patch creates a complete main for components perftests from components unittests main and switches components perftests to using testing/perf instead of base/perf. BUG=648992 ==========
Message was sent while issue was closed.
Committed patchset #8 (id:140001)
Message was sent while issue was closed.
Description was changed from ========== Preparing components_perftests for adding more benchmarks. This patch creates a complete main for components perftests from components unittests main and switches components perftests to using testing/perf instead of base/perf. BUG=648992 ========== to ========== Preparing components_perftests for adding more benchmarks. This patch creates a complete main for components perftests from components unittests main and switches components perftests to using testing/perf instead of base/perf. BUG=648992 Committed: https://crrev.com/bd75dc348f2e52e1fc3e1b26b6c7528329657d65 Cr-Commit-Position: refs/heads/master@{#427043} ==========
Message was sent while issue was closed.
Patchset 8 (id:??) landed as https://crrev.com/bd75dc348f2e52e1fc3e1b26b6c7528329657d65 Cr-Commit-Position: refs/heads/master@{#427043} |