Proof of concept for running extension API stack through dev tools.

Proof of concept for running extension API stack through dev tools.

To play with it, start test scripts as:

./tools/telemetry/examples/telemetry_extension_test.py --browser=release --load-component-extension=<path_to>/tools/telemetry_auto/data/auto_provider

./tools/telemetry_auto/examples/login_test.py --browser=release --load-component-extension=<path_to>/tools/telemetry_auto/data/auto_provider --extra-browser-args="--login-manager --stub-cros --login-profile=user --user-data-dir=/tmp/test_profile --no-first-run"

BUG=162730
TEST=none

[nduca, 2012-11-29 05:09:27]

Super cool. Left you a ton of random observations, happy to chat tomorrow if you
like.

https://codereview.chromium.org/11412238/diff/2001/tools/telemetry/examples/t...
File tools/telemetry/examples/telemetry_extension_test.py (right):

https://codereview.chromium.org/11412238/diff/2001/tools/telemetry/examples/t...
tools/telemetry/examples/telemetry_extension_test.py:9:
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
you can even move this file to the telemetry_auto folder and then copy the stuff
from tools/perf/perf_tools/__init__.py to pull in telemetry from its usual
location

also, we've found that a standard python module directory structure works well,
e.g.

project_name/
   project_module_name
      __init__.py
      module1

that way you can append $PATH_TO/project_name/ to sys.path, and then import
project_module_name as if it was local.

https://codereview.chromium.org/11412238/diff/2001/tools/telemetry/examples/t...
tools/telemetry/examples/telemetry_extension_test.py:18: browser_to_create =
telemetry.FindBrowser(options)
seems like we should add a feature on BrowserOptions for "please load this
extension when you boot the browser." Then telemetry can take care of pushing
the extension to the device if needed. I can walk you through the rough approach
if you feel like this id doing the right thing....

https://codereview.chromium.org/11412238/diff/2001/tools/telemetry/examples/t...
tools/telemetry/examples/telemetry_extension_test.py:22:
ext.runtime.Evaluate("chrome.tabs.create({url:'http://www.google.com'},function(){})")
so to do this synchronously you'd do this:

ext.runtime.Execute("window.__done = false");
def IsDone():
  return ext.runtime.Evaluate("window.__done");
ext.runtime.Evaluate(""""
chrome.tabs.create({url:'http://www.google.com'},function() {
   window.__done = true;
})
"""
WaitFor(IsDone);

https://codereview.chromium.org/11412238/diff/2001/tools/telemetry/telemetry/...
File tools/telemetry/telemetry/browser_backend.py (right):

https://codereview.chromium.org/11412238/diff/2001/tools/telemetry/telemetry/...
tools/telemetry/telemetry/browser_backend.py:116: return tab.Tab(browser, ib)
raise exception if nothing found

https://codereview.chromium.org/11412238/diff/2001/tools/telemetry/telemetry/...
tools/telemetry/telemetry/browser_backend.py:116: return tab.Tab(browser, ib)
this change looks totally landable. we'll want a unit test for
browser_backend_unittest, and we'll need the "load extension" feature on
browser_options and the various backends, but <3 this.

[zel, 2012-11-29 22:58:44]

https://codereview.chromium.org/11412238/diff/2001/tools/telemetry/examples/t...
File tools/telemetry/examples/telemetry_extension_test.py (right):

https://codereview.chromium.org/11412238/diff/2001/tools/telemetry/examples/t...
tools/telemetry/examples/telemetry_extension_test.py:18: browser_to_create =
telemetry.FindBrowser(options)
On 2012/11/29 05:09:27, nduca wrote:
> seems like we should add a feature on BrowserOptions for "please load this
> extension when you boot the browser." Then telemetry can take care of pushing
> the extension to the device if needed. I can walk you through the rough
approach
> if you feel like this id doing the right thing....

Sure, will sync up with you about this one.

https://codereview.chromium.org/11412238/diff/2001/tools/telemetry/examples/t...
tools/telemetry/examples/telemetry_extension_test.py:22:
ext.runtime.Evaluate("chrome.tabs.create({url:'http://www.google.com'},function(){})")
On 2012/11/29 05:09:27, nduca wrote:
> so to do this synchronously you'd do this:
> 
> ext.runtime.Execute("window.__done = false");
> def IsDone():
>   return ext.runtime.Evaluate("window.__done");
> ext.runtime.Evaluate(""""
> chrome.tabs.create({url:'http://www.google.com'},function() {
>    window.__done = true;
> })
> """
> WaitFor(IsDone);

Done. I've created a new class ExtensionPage that derives from Tab and extends
it with wrapper method for doing this trickery + event interception.

[nduca, 2012-11-30 07:31:43]

Haven't had a chance to look at your latest patchset lately, but it'd be good to
sync with @dtu. He, tonyg and I had an impromptu 3min chat about a plan earlier
that integrated extensions into telemetry.

[nduca, 2012-11-30 08:16:25]

https://codereview.chromium.org/11412238/diff/9002/tools/telemetry/telemetry/...
File tools/telemetry/telemetry/extension_page.py (right):

https://codereview.chromium.org/11412238/diff/9002/tools/telemetry/telemetry/...
tools/telemetry/telemetry/extension_page.py:71: def RunMethod(self, method_name,
*parameters):
Might try callling this ChromeExtensionsAPI, making it a member on chrome
extension objects, and have it expose directly the chrome extension methods in a
strongly typed way. Then, have the RunMethod stuff be completely private.

[zel, 2012-12-04 03:12:48]

also moved all files around per nduca's instructions

https://codereview.chromium.org/11412238/diff/2001/tools/telemetry/telemetry/...
File tools/telemetry/telemetry/browser_backend.py (right):

https://codereview.chromium.org/11412238/diff/2001/tools/telemetry/telemetry/...
tools/telemetry/telemetry/browser_backend.py:116: return tab.Tab(browser, ib)
On 2012/11/29 05:09:27, nduca wrote:
> this change looks totally landable. we'll want a unit test for
> browser_backend_unittest, and we'll need the "load extension" feature on
> browser_options and the various backends, but <3 this.

I don't see browser_backend_unittest.py - what am I missing here?

https://codereview.chromium.org/11412238/diff/9002/tools/telemetry/telemetry/...
File tools/telemetry/telemetry/extension_page.py (right):

https://codereview.chromium.org/11412238/diff/9002/tools/telemetry/telemetry/...
tools/telemetry/telemetry/extension_page.py:71: def RunMethod(self, method_name,
*parameters):
On 2012/11/30 08:16:25, nduca wrote:
> Might try callling this ChromeExtensionsAPI, making it a member on chrome
> extension objects, and have it expose directly the chrome extension methods in
a
> strongly typed way. Then, have the RunMethod stuff be completely private.

I've been thinking about this as the next step - with exception that strongly
typed interface will be automatically created during build time from extension
API definition files (idl, json). The plan was to push this in another CL after
we land this basic stuff.

[zel, 2012-12-06 19:21:55]

* added "--load-automation-extension" switch to browser_options
* added file push for automation component extension on ChromeOS
* aggregated Tab into ExtensionPage (instead of inherited from it)

[zel, 2012-12-06 22:47:59]

* added new unit test - testAutomationExtension - to browser_unittest.py

[nduca, 2012-12-07 00:59:18]

so firefox doesn't have an automation extension. again, telemetry api is not
supposed to let these things leak in so broadly.

if you go back in some of my previous review notes, i tried to describe a way to
do this:

--load-extension=<path-to-extension>  turns into
browser_options.extensions_to_load=[ExtensionToLoad(<path_to_extension>)
ExtensionToLoad can have a property that is "type" that you could set to
"normal" or "content". The browser backend can look at the extension file and
first verify that its compatible, and then also look at the type and do the
right thing.

In the case of telemetry auto, you wouldn't really use the --load-extension
part. Rather, you'd just do
   ext = ExtensionToLoad(<path to automation extension>)
   ext.type = 'content' // it'd be even better if this could be a field we set
on manifest.json tbh
   options.extensions_to_load(ext)

[zel, 2012-12-07 03:23:42]

Replaced "--load-automation-extension" switch with more generic
"--load-extension" and "--load-component-extension" per Nat's suggestion.

[nduca, 2012-12-07 19:23:47]

You probably need to make android backend explode in its constructor if it has
any extensions to load.

While we're at it, PossibleBrowser should have a method on it saying
"SupportsExtensions"

And cros and desktop shoudl explode if self.is_content_shell

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/examples/...
File tools/telemetry/examples/telemetry_extension_test.py (right):

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/examples/...
tools/telemetry/examples/telemetry_extension_test.py:14: parser =
options.CreateParser('telemetry_extension_test.py')
should there be an extension as part of this? just a simple one checked in
parallel to this would be good --- otherwise, this wont work on an average
computer.

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
File tools/telemetry/telemetry/browser_unittest.py (right):

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
tools/telemetry/telemetry/browser_unittest.py:46:
browser_options.ExtensionToLoad(
you have to have the extension that this references checked in somewhere into
telemetry, for the unit test. it can be a copy, but telemetry needs to self
verify without pointing to other directories in the tree.

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
File tools/telemetry/telemetry/cros_browser_backend.py (right):

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
tools/telemetry/telemetry/cros_browser_backend.py:40: # Push the automation
provider extension to the device.
looks like you forgot to make this file do the iteration. You got the others.

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
File tools/telemetry/telemetry/desktop_browser_backend.py (right):

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
tools/telemetry/telemetry/desktop_browser_backend.py:61: component_extensions +=
','
how about compoent_extensions and extensions being []'s and using
','.join(compoent_extensions) to comma separate them

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
File tools/telemetry/telemetry/extension_page.py (right):

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
tools/telemetry/telemetry/extension_page.py:16: def __init__(self, tab):
When we get close to landing this, this shouldn't have a tab as much as have the
copy & pasted parts of the tab init code, so this has an inpsector backend,
timeline, runtime but not page probably

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
tools/telemetry/telemetry/extension_page.py:27: for param in parameters:
doesn't our code do this for us?

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
tools/telemetry/telemetry/extension_page.py:52: def RunMethodWithEvents(self,
event_names, method_name, *parameters):
This stuff makes me uneasy that the last argument is the callback. Did you think
about something like

RunJavascriptAndWaitForCallback("myOwnJavaScriptThat.Calls.A.Few.Functions("with",
__TELEMETRY_COMPLETION_CALLBACK, "as some middle argument",
callback_id="__TELEMETRY_COMPLETION_CALLBACK");

Then we could move this to the runtime object

And we could always return the arguments that were passed into the completion
callback, maybe? The distinction between ExecuteJS vs EvaluateJS in the runtime
class might give you a possible design route.

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
tools/telemetry/telemetry/extension_page.py:91: def RunSyncMethod(self,
method_name, *parameters):
I think I'm fine with this on runtime object

[zel, 2012-12-12 21:03:41]

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/examples/...
File tools/telemetry/examples/telemetry_extension_test.py (right):

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/examples/...
tools/telemetry/examples/telemetry_extension_test.py:14: parser =
options.CreateParser('telemetry_extension_test.py')
On 2012/12/07 19:23:47, nduca wrote:
> should there be an extension as part of this? just a simple one checked in
> parallel to this would be good --- otherwise, this wont work on an average
> computer.

will ping you about this in person

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
File tools/telemetry/telemetry/browser_unittest.py (right):

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
tools/telemetry/telemetry/browser_unittest.py:46:
browser_options.ExtensionToLoad(
On 2012/12/07 19:23:47, nduca wrote:
> you have to have the extension that this references checked in somewhere into
> telemetry, for the unit test. it can be a copy, but telemetry needs to self
> verify without pointing to other directories in the tree.

Done.

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
File tools/telemetry/telemetry/cros_browser_backend.py (right):

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
tools/telemetry/telemetry/cros_browser_backend.py:40: # Push the automation
provider extension to the device.
On 2012/12/07 19:23:47, nduca wrote:
> looks like you forgot to make this file do the iteration. You got the others.

iteration? what iteration?

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
File tools/telemetry/telemetry/desktop_browser_backend.py (right):

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
tools/telemetry/telemetry/desktop_browser_backend.py:61: component_extensions +=
','
On 2012/12/07 19:23:47, nduca wrote:
> how about compoent_extensions and extensions being []'s and using
> ','.join(compoent_extensions) to comma separate them

done, made it even more pythony

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
File tools/telemetry/telemetry/extension_page.py (right):

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
tools/telemetry/telemetry/extension_page.py:27: for param in parameters:
On 2012/12/07 19:23:47, nduca wrote:
> doesn't our code do this for us?

not sure. what do you think I should be calling instead?

https://codereview.chromium.org/11412238/diff/23001/tools/telemetry/telemetry...
tools/telemetry/telemetry/extension_page.py:52: def RunMethodWithEvents(self,
event_names, method_name, *parameters):
On 2012/12/07 19:23:47, nduca wrote:
> This stuff makes me uneasy that the last argument is the callback. Did you
think
> about something like
> 
>
RunJavascriptAndWaitForCallback("myOwnJavaScriptThat.Calls.A.Few.Functions("with",
> __TELEMETRY_COMPLETION_CALLBACK, "as some middle argument",
> callback_id="__TELEMETRY_COMPLETION_CALLBACK");
> 
> Then we could move this to the runtime object
> 
> And we could always return the arguments that were passed into the completion
> callback, maybe? The distinction between ExecuteJS vs EvaluateJS in the
runtime
> class might give you a possible design route.

we need to chat in person about this one too

[zel, 2012-12-13 02:44:45]

few more things:
- made android and content_shell backend explode in its constructor if it has
any extensions to load.
- added supports_extensions property to PossibleBrowser

[nduca, 2013-01-05 06:29:11]

Hey Zel, just getting back from holidays. This is looking really solid. In
particular, extensions support, and the basic creation of telemetry_auto are
starting to look almost landable. I think there's more we want to iterate on,
around tabs etc.

What do you think about breaking out at least
- patch to create telemetry_auto folder
- pure patch to add support for extensions to telemetry [with nobody but tests
that use them]

I'm thinking that'll give us some smaller patches that I can review more quickly
etc....

[zel, 2013-01-11 23:03:21]

On 2013/01/05 06:29:11, nduca wrote:
> Hey Zel, just getting back from holidays. This is looking really solid. In
> particular, extensions support, and the basic creation of telemetry_auto are
> starting to look almost landable. I think there's more we want to iterate on,
> around tabs etc.
> 
> What do you think about breaking out at least
> - patch to create telemetry_auto folder
> - pure patch to add support for extensions to telemetry [with nobody but tests
> that use them]
> 
> I'm thinking that'll give us some smaller patches that I can review more
quickly
> etc....

achuith@ is taking over this work since I am swamped with other work items. He
will do the split as you suggested.