[telemetry] Read MSRs from a separate privileged process.

[telemetry] Read MSRs from a separate privileged process.

This allows us to read MSRs on Windows Vista+.
Also move ReadMsr into PlatformBackend.

BUG=336558
TEST=tools/telemetry/run_tests msr

Committed: https://crrev.com/2156b0323c83d6f11e68344ecf7e7e0c833cfa14
Cr-Commit-Position: refs/heads/master@{#295634}

Committed: https://crrev.com/e8f709b238dad796f7babb928d482963a5421794
Cr-Commit-Position: refs/heads/master@{#295737}

[dtu, 2014-09-17 23:54:59]

dtu@chromium.org changed reviewers:
+ tonyg@chromium.org

[dtu, 2014-09-18 00:10:33]

Please review, but I think I'm leaving a Python process around, and need to
debug that one thing. Unfortunately, there's no scope for platform_backend or
power_monitor, so there's not a great place to clean up.

Explanation for the choice of TCP server is in msr_server.py

[tonyg, 2014-09-18 00:18:06]

tonyg@chromium.org changed reviewers:
+ thestig@chromium.org

[tonyg, 2014-09-18 00:21:46]

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
File tools/telemetry/telemetry/core/platform/msr_server_win.py (right):

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/platform/msr_server_win.py:91:
SocketServer.TCPServer.allow_reuse_address = True
Does setting daemon_threads help w/ the leaving processes around problem? If
not, what about an atexit handler?

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
File tools/telemetry/telemetry/core/platform/platform_backend.py (right):

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/platform/platform_backend.py:211: def
ReadMsr(self, msr_number):
I think we should add a comment that defines MSR.

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
File tools/telemetry/telemetry/core/platform/win_platform_backend.py (right):

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/platform/win_platform_backend.py:321:
_InstallWinRing0()
Since the server must be privileged, should we do any of this stuff there?

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/platform/win_platform_backend.py:331:
socket.create_connection(('127.0.0.1', self._msr_server_port)).close()
Server starting/stopping seems like it should be shared across platforms.
Thoughts?

[dtu, 2014-09-18 05:22:29]

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
File tools/telemetry/telemetry/core/platform/msr_server_win.py (right):

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/platform/msr_server_win.py:91:
SocketServer.TCPServer.allow_reuse_address = True
On 2014/09/18 00:21:45, tonyg wrote:
> Does setting daemon_threads help w/ the leaving processes around problem? If
> not, what about an atexit handler?

Done. Used an atexit handler. I don't think daemon processes exist in this
context.

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
File tools/telemetry/telemetry/core/platform/platform_backend.py (right):

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/platform/platform_backend.py:211: def
ReadMsr(self, msr_number):
On 2014/09/18 00:21:46, tonyg wrote:
> I think we should add a comment that defines MSR.

Done.

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
File tools/telemetry/telemetry/core/platform/win_platform_backend.py (right):

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/platform/win_platform_backend.py:321:
_InstallWinRing0()
On 2014/09/18 00:21:46, tonyg wrote:
> Since the server must be privileged, should we do any of this stuff there?

Yes, but I felt it to be cleaner for the server to not have any Telemetry
dependencies. That's also why I moved WinRing0.dll from telemetry/bin/win/ to
the Python directory.

https://codereview.chromium.org/578123002/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/platform/win_platform_backend.py:331:
socket.create_connection(('127.0.0.1', self._msr_server_port)).close()
On 2014/09/18 00:21:46, tonyg wrote:
> Server starting/stopping seems like it should be shared across platforms.
> Thoughts?

I'm not sure if all platforms will use a server like this, or if they'll
communicate with the server in the same way. Linux might use a
multiprocessing.Pipe or some other mechanism.

[tonyg, 2014-09-19 00:10:45]

lgtm

[dtu, 2014-09-19 00:14:21]

The CQ bit was checked by dtu@chromium.org

[commit-bot: I haz the power, 2014-09-19 00:15:59]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/578123002/40001

[commit-bot: I haz the power, 2014-09-19 01:13:23]

Committed patchset #3 (id:40001) as 67d8f9774e6902a769f273fb300ec8067e58ceed

[commit-bot: I haz the power, 2014-09-19 01:14:02]

Patchset 3 (id:??) landed as
https://crrev.com/2156b0323c83d6f11e68344ecf7e7e0c833cfa14
Cr-Commit-Position: refs/heads/master@{#295634}

[vasilii, 2014-09-19 08:30:42]

A revert of this CL (patchset #3 id:40001) has been created in
https://codereview.chromium.org/582123003/ by vasilii@chromium.org.

The reason for reverting is: Fails telemetry_unittests and
telemetry_perf_unittests on XP Tests (2)
http://build.chromium.org/p/chromium.win/builders/XP%20Tests%20%282%29/builds...

Traceback (most recent call last):
  run at E:\b\depot_tools\python276_bin\lib\unittest\case.py:331
    testMethod()
  testPowerMonitoringSync at
tools\telemetry\telemetry\core\platform\platform_backend_unittest.py:16
    if not platform.CanMonitorPower():
  CanMonitorPower at tools\telemetry\telemetry\core\platform\__init__.py:248
    return self._platform_backend.CanMonitorPower()
  CanMonitorPower at
tools\telemetry\telemetry\core\platform\win_platform_backend.py:313
    return self._power_monitor.CanMonitorPower()
  CanMonitorPower at
tools\telemetry\telemetry\core\platform\power_monitor\msr_power_monitor.py:51
    if self._PackageEnergyJoules() <= 0:
  _PackageEnergyJoules at
tools\telemetry\telemetry\core\platform\power_monitor\msr_power_monitor.py:97
    return (self._backend.ReadMsr(MSR_PKG_ENERGY_STATUS) *
  ReadMsr at tools\telemetry\telemetry\core\platform\win_platform_backend.py:347
    self._StartMsrServerIfNeeded()
  _StartMsrServerIfNeeded at
tools\telemetry\telemetry\core\platform\win_platform_backend.py:338
    sys.executable, parameters, elevate_privilege=True)
  LaunchApplication at
tools\telemetry\telemetry\core\platform\win_platform_backend.py:293
    if elevate_privilege and not IsCurrentProcessElevated():
  IsCurrentProcessElevated at
tools\telemetry\telemetry\core\platform\win_platform_backend.py:80
    win32security.GetTokenInformation(token, win32security.TokenElevation))
error: (87, 'GetTokenInformation', 'The parameter is incorrect.')

Locals:
  handle : -1
  token  : <PyHANDLE:0>

[  FAILED  ]
platform.platform_backend_unittest.PlatformBackendTest.testPowerMonitoringSync
(3422 ms)

Traceback (most recent call last):
  run at E:\b\depot_tools\python276_bin\lib\unittest\case.py:331
    testMethod()
  wrapper at tools\telemetry\telemetry\decorators.py:84
    func(*args, **kwargs)
  testMsrRuns at
tools\telemetry\telemetry\core\platform\power_monitor\msr_power_monitor_unittest.py:19
    if not power_monitor.CanMonitorPower():
  CanMonitorPower at
tools\telemetry\telemetry\core\platform\power_monitor\msr_power_monitor.py:51
    if self._PackageEnergyJoules() <= 0:
  _PackageEnergyJoules at
tools\telemetry\telemetry\core\platform\power_monitor\msr_power_monitor.py:97
    return (self._backend.ReadMsr(MSR_PKG_ENERGY_STATUS) *
  ReadMsr at tools\telemetry\telemetry\core\platform\win_platform_backend.py:347
    self._StartMsrServerIfNeeded()
  _StartMsrServerIfNeeded at
tools\telemetry\telemetry\core\platform\win_platform_backend.py:338
    sys.executable, parameters, elevate_privilege=True)
  LaunchApplication at
tools\telemetry\telemetry\core\platform\win_platform_backend.py:293
    if elevate_privilege and not IsCurrentProcessElevated():
  IsCurrentProcessElevated at
tools\telemetry\telemetry\core\platform\win_platform_backend.py:80
    win32security.GetTokenInformation(token, win32security.TokenElevation))
error: (87, 'GetTokenInformation', 'The parameter is incorrect.')

Locals:
  handle : -1
  token  : <PyHANDLE:0>

[  FAILED  ]
platform.power_monitor.msr_power_monitor_unittest.MsrPowerMonitorTest.testMsrRuns
(32 ms)
[----------] 1 test (32 ms total)

Traceback (most recent call last):
  run at E:\b\depot_tools\python276_bin\lib\unittest\case.py:331
    testMethod()
  wrapper at tools\telemetry\telemetry\decorators.py:55
    func(*args, **kwargs)
  BenchmarkSmokeTest at tools\perf\benchmarks\benchmark_unittest.py:59
    self.assertEqual(0, SinglePageBenchmark().Run(options),
  Run at tools\telemetry\telemetry\benchmark.py:95
    page_runner.Run(pt, ps, expectations, finder_options, results)
  Run at tools\telemetry\telemetry\page\page_runner.py:429
    page, credentials_path, possible_browser, results, state)
  _PrepareAndRunPage at tools\telemetry\telemetry\page\page_runner.py:259
    finder_options)
  StartBrowserIfNeeded at tools\telemetry\telemetry\page\page_runner.py:54
    test.WillStartBrowser(possible_browser.platform)
  WillStartBrowser at tools\perf\measurements\webrtc.py:21
    self._power_metric = power.PowerMetric(platform)
  __init__ at tools\perf\metrics\power.py:31
    self._MeasureQuiescentPower(quiescent_measurement_time_s)
  _MeasureQuiescentPower at tools\perf\metrics\power.py:54
    if not self._platform.CanMonitorPower() or \
  CanMonitorPower at tools\telemetry\telemetry\core\platform\__init__.py:248
    return self._platform_backend.CanMonitorPower()
  CanMonitorPower at
tools\telemetry\telemetry\core\platform\win_platform_backend.py:313
    return self._power_monitor.CanMonitorPower()
  CanMonitorPower at
tools\telemetry\telemetry\core\platform\power_monitor\msr_power_monitor.py:51
    if self._PackageEnergyJoules() <= 0:
  _PackageEnergyJoules at
tools\telemetry\telemetry\core\platform\power_monitor\msr_power_monitor.py:97
    return (self._backend.ReadMsr(MSR_PKG_ENERGY_STATUS) *
  ReadMsr at tools\telemetry\telemetry\core\platform\win_platform_backend.py:347
    self._StartMsrServerIfNeeded()
  _StartMsrServerIfNeeded at
tools\telemetry\telemetry\core\platform\win_platform_backend.py:338
    sys.executable, parameters, elevate_privilege=True)
  LaunchApplication at
tools\telemetry\telemetry\core\platform\win_platform_backend.py:293
    if elevate_privilege and not IsCurrentProcessElevated():
  IsCurrentProcessElevated at
tools\telemetry\telemetry\core\platform\win_platform_backend.py:80
    win32security.GetTokenInformation(token, win32security.TokenElevation))
error: (87, 'GetTokenInformation', 'The parameter is incorrect.')

Locals:
  handle : -1
  token  : <PyHANDLE:0>

[  FAILED  ] BenchmarkSmokeTest.webrtc.webrtc_cases (766 ms).

[dtu, 2014-09-19 17:10:38]

XP: Added check to skip security token check.
Vista+: Increased server launch timeout. I don't know why it's working locally
but failing on the perf bots, and this has been the problem in the past.

[dtu, 2014-09-19 17:10:54]

The CQ bit was checked by dtu@chromium.org

[commit-bot: I haz the power, 2014-09-19 17:11:49]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/578123002/60001

[commit-bot: I haz the power, 2014-09-19 18:09:16]

Committed patchset #4 (id:60001) as 5937b083416943623582ede6ffa5caa6c8d1b34a

[commit-bot: I haz the power, 2014-09-19 18:10:08]

Patchset 4 (id:??) landed as
https://crrev.com/e8f709b238dad796f7babb928d482963a5421794
Cr-Commit-Position: refs/heads/master@{#295737}