Backporting ilevy@'s ParsePythonCfg so it can be used in Annotator.

Backporting ilevy@'s ParsePythonCfg so it can be used in Annotator.

R=ilevy@chromium.org,xusydoc@chromium.org


Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=185296

[agable, 2013-02-27 23:13:15]

Isaac, Mike -- pulling this in on its own so we can use it before Isaac's full
CL lands.

Interestingly, running into presubmit errors. From old code. No clue why. Pasted
below:

Running presubmit upload checks ...
Running pylint on 302 files.
************* Module slave.chromium_commands
W0231:654,2:ApplyIssue.__init__: __init__ method from base class 'Command' is
not called

** Presubmit Warnings **
Fix pylint errors first.

test/slaves_cfg_test.py failed!
Command test/slaves_cfg_test.py returned non-zero exit status 1 in
/work/chri/build
cbuildbot_chromite not found!  Returning empty config dictionary.
build15-m2 duplicated in masters: master.chromiumos.unused master.chromeos
build47-m2 duplicated in masters: master.chromiumos.unused master.chromeos
build22-m2 duplicated in masters: master.chromiumos.unused master.chromeos

[Mike Stip (use stip instead), 2013-02-28 18:49:08]

> Interestingly, running into presubmit errors. From old code. No clue why.

Fix 'em when you see 'em! https://codereview.chromium.org/12388023/

[Mike Stip (use stip instead), 2013-02-28 18:52:29]

I'm fine with this. ilevy's change is large and we need this functionality
sooner than later. lgtm

[commit-bot: I haz the power, 2013-02-28 18:59:16]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/agable@chromium.org/12351003/3001

[commit-bot: I haz the power, 2013-02-28 19:04:20]

Presubmit check for 12351003-3001 failed and returned exit status 1.

INFO:root:Found 1 file(s).
INFO:PRESUBMIT:Running pylint on 302 files
cbuildbot_chromite not found!  Returning empty config dictionary.
1 masters failed:
master.chromium.webkit
47 masters succeeded, 1 failed, 12 skipped in 237.2s.
Trying master.chromium
Trying master.chromium.chrome
Trying master.chromium.chromebot
Trying master.chromium.chromiumos
Trying master.chromium.endure
Trying master.chromium.flaky
Trying master.chromium.fyi
Trying master.chromium.git
Trying master.chromium.gpu
Trying master.chromium.gpu.fyi
Trying master.chromium.linux
Trying master.chromium.lkgr
Trying master.chromium.mac
Trying master.chromium.memory
Trying master.chromium.memory.fyi
Trying master.chromium.perf
Trying master.chromium.perf_av
Trying master.chromium.pyauto
Trying master.chromium.swarm
Trying master.chromium.webkit
2013-02-28 11:01:10-0800 [-] Starting factory
<buildbot.status.web.baseweb.RotateLogSite instance at 0x2af09c0c5050>
2013-02-28 11:01:10-0800 [-] Setting up http.log rotating 10 files of 1000000
bytes each
2013-02-28 11:01:10-0800 [-] WebStatus using
(/b/commit-queue/workdir/tools/build/masters/master.chromium/public_html)
2013-02-28 11:01:10-0800 [-] adding IStatusReceiver <WebStatus on port tcp:9055
at 0x45c97a0>
2013-02-28 11:01:10-0800 [-] buildbot.status.web.baseweb.RotateLogSite starting
on 9055
2013-02-28 11:01:10-0800 [-] Starting factory
<buildbot.status.web.baseweb.RotateLogSite instance at 0x2af09c0d1908>
2013-02-28 11:01:10-0800 [-] Setting up http.log rotating 10 files of 1000000
bytes each
2013-02-28 11:01:10-0800 [-] WebStatus using
(/b/commit-queue/workdir/tools/build/masters/master.chromium/public_html)
2013-02-28 11:01:10-0800 [-] removing 0 old schedulers, updating 0, and adding
38
2013-02-28 11:01:15-0800 [-] Unhandled Error
	Traceback (most recent call last):
	Failure: twisted.internet.defer.FirstError: FirstError[#4, [Failure instance:
Traceback: <class 'sqlalchemy.exc.OperationalError'>: (OperationalError)
database is locked u'INSERT INTO schedulers (name, state, class_name) VALUES (?,
?, ?)' ('s4_webkit_rel_trigger', '{}',
'buildbot.schedulers.triggerable.Triggerable')
	/usr/lib/python2.6/threading.py:504:__bootstrap
	/usr/lib/python2.6/threading.py:532:__bootstrap_inner
	/usr/lib/python2.6/threading.py:484:run
	--- <exception caught here> ---
	/b/commit-queue/workdir/tools/build/third_party/twisted_10_2/twisted/python/threadpool.py:207:_worker
	/b/commit-queue/workdir/tools/build/third_party/twisted_10_2/twisted/python/context.py:59:callWithContext
	/b/commit-queue/workdir/tools/build/third_party/twisted_10_2/twisted/python/context.py:37:callWithContext
	/b/commit-queue/workdir/tools/build/third_party/buildbot_8_4p1/buildbot/db/pool.py:112:thd
	/b/commit-queue/workdir/tools/build/third_party/buildbot_8_4p1/buildbot/db/schedulers.py:165:thd
	/b/commit-queue/workdir/tools/build/third_party/sqlalchemy_0_7_1/sqlalchemy/engine/base.py:1358:execute
	/b/commit-queue/workdir/tools/build/third_party/sqlalchemy_0_7_1/sqlalchemy/engine/base.py:1491:_execute_clauseelement
	/b/commit-queue/workdir/tools/build/third_party/sqlalchemy_0_7_1/sqlalchemy/engine/base.py:1599:_execute_context
	/b/commit-queue/workdir/tools/build/third_party/sqlalchemy_0_7_1/sqlalchemy/engine/base.py:1592:_execute_context
	/b/commit-queue/workdir/tools/build/third_party/sqlalchemy_0_7_1/sqlalchemy/engine/default.py:325:do_execute
	]]

Trying master.chromium.webrtc
Trying master.chromium.webrtc.fyi
Trying master.chromium.win
Trying master.chromiumos
Trying master.client.dart
Trying master.client.dart.fyi
Trying master.client.drmemory
Trying master.client.dynamorio
Trying master.client.libjingle
Trying master.client.libyuv
Trying master.client.nacl
Trying master.client.nacl.chrome
Trying master.client.nacl.llvm
Trying master.client.nacl.ports
Trying master.client.nacl.ragel
Trying master.client.nacl.sdk
Trying master.client.nacl.sdk.addin
Trying master.client.nacl.sdk.mono
Trying master.client.nacl.toolchain
Trying master.client.omaha
Trying master.client.pagespeed
Trying master.client.v8
Trying master.client.webrtc
Trying master.devtools
Trying master.tryserver.chromium
Trying master.tryserver.chromium.linux
Trying master.tryserver.nacl
Trying master.tryserver.webrtc
/b/commit-queue/workdir/tools/build/third_party/zope/__init__.py:19:
UserWarning: Module mock was already imported from
/b/commit-queue/workdir/tools/build/third_party/mock-1.0.1/mock.py, but
/usr/local/lib/python2.6/dist-packages/mock-1.0b1-py2.6.egg is being added to
sys.path
  import pkg_resources
test_android (__main__.TestMailNotifier) ... ok
test_cq_succeed (__main__.TestMailNotifier) ... ok
test_simple (__main__.TestMailNotifier) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.250s

OK
Parsing client.v8
Parsing chromium.endure
Parsing chromium.memory
Skipping chromium.swarm, fix and enable in masters_cfg_test.py!
Skipping client.nacl.chrome, fix and enable in masters_cfg_test.py!
Parsing experimental
Parsing chromium.win
Parsing client.skia
Parsing chromium
Parsing tryserver.webrtc
Parsing client.sfntly
Parsing chromium.webrtc
Parsing client.dart.fyi
Parsing chromium.webrtc.fyi
Parsing client.pagespeed
Parsing chromium.pyauto
Parsing client.libjingle
Parsing tryserver.chromium
perf_id: chromium-dbg-linux-try
perf_subdir: linux-try-debug
url: http://build.chromium.org/buildbot/coverage/linux-try-debug/total_coverage
Skipping chromiumos.tryserver, fix and enable in masters_cfg_test.py!
Parsing client.dynamorio
Parsing client.syzygy
Parsing client.omaha
Parsing client.nacl.llvm
Parsing chromium.perf
Parsing chromium.memory.fyi
Parsing tryserver.nacl
Parsing chromium.webkit
Parsing client.nacl.sdk
Parsing chromium.linux
Parsing chromium.perf_av
Parsing chromium.gpu.fyi
Parsing chromium.chrome
Parsing chromiumos
Parsing client.nacl
Parsing client.nacl.toolchain
Parsing tryserver.chromium.linux
Parsing chromium.lkgr
Parsing client.nacl.sdk.mono
Parsing chromium.flaky
Parsing client.webrtc
Parsing client.nacl.ragel
Parsing chromium.fyi
perf_id: chromium-dbg-linux
perf_subdir: linux-debug
url: http://build.chromium.org/buildbot/coverage/linux-debug/total_coverage
Parsing devtools
Parsing client.nacl.sdk.addin
Parsing client.drmemory
Parsing chromium.mac
Parsing client.libyuv
Skipping chromium.chromebot, fix and enable in masters_cfg_test.py!
Parsing chromium.gpu
Parsing client.dart
Parsing chromium.chromiumos
Parsing client.nacl.ports
Parsing chromium.git
All master.cfg files parsed successfully!
testFailuresNotifierCreation (__main__.FailuresNotifierTest) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
test_Creation (__main__.GateKeeperTest) ... ok
test_buildMessage (__main__.GateKeeperTest) ... ok
test_no_url (__main__.GateKeeperTest) ... ok
test_tree_closed (__main__.GateKeeperTest) ... ok
test_tree_open (__main__.GateKeeperTest) ... ok
test_url_fetch_fail (__main__.GateKeeperTest) ... ok

----------------------------------------------------------------------
Ran 6 tests in 0.029s

OK
test_maxsize (__main__.FailuresHistoryTest) ... ok
test_stress_maxsize (__main__.FailuresHistoryTest) ... ok
test_timer (__main__.FailuresHistoryTest) ... ok
test_trivial (__main__.FailuresHistoryTest) ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.255s

OK
testFilteredMeanAndStandardDeviation
(__main__.FilteredMeanAndStandardDeviationTest) ... ok
testFilteredMeanAndStandardDeviationOne
(__main__.FilteredMeanAndStandardDeviationTest) ... ok
testConstructorCurring (__main__.PartiallyInitializeTest) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK
testParseText (__main__.PaseOptionsTest) ... ok
test_dict_comma_life_like (__main__.PaseOptionsTest) ... ok
test_dict_comma_merge (__main__.PaseOptionsTest) ... ok
test_dict_comma_not_key (__main__.PaseOptionsTest) ... ok
test_dict_comma_state_1 (__main__.PaseOptionsTest) ... ok
test_dict_comma_state_1_1 (__main__.PaseOptionsTest) ... ok
test_dict_comma_state_1_1_drop (__main__.PaseOptionsTest) ... ok
test_dict_comma_state_1_2 (__main__.PaseOptionsTest) ... ok
test_dict_comma_state_1_2_3 (__main__.PaseOptionsTest) ... ok
test_dict_comma_state_1_2_3_4_3 (__main__.PaseOptionsTest) ... ok
test_dict_comma_state_1_2_3_4_value1 (__main__.PaseOptionsTest) ... ok
test_dict_comma_state_1_2_3_4_value2 (__main__.PaseOptionsTest) ... ok
test_dict_comma_state_1_2_3_bad (__main__.PaseOptionsTest) ... ok
test_dict_comma_state_1_2_4_3 (__main__.PaseOptionsTest) ... ok
test_dict_comma_trailing_comma (__main__.PaseOptionsTest) ... ok
test_parse_options_clobber_false (__main__.PaseOptionsTest) ... ok
test_parse_options_clobber_true (__main__.PaseOptionsTest) ... ok
test_parse_options_defaults (__main__.PaseOptionsTest) ... ok

----------------------------------------------------------------------
Ran 18 tests in 0.002s

OK
testChromiumNotifierCreation (__main__.ChromiumNotifierTest) ... ok
testChromiumNotifierDecoration (__main__.ChromiumNotifierTest) ... ok
testChromiumNotifierSingleMatch (__main__.ChromiumNotifierTest) ... ok
testChromiumNotifierWildcard (__main__.ChromiumNotifierTest) ... ok
testChromiumNotifierWildcardNoBuilderMatch (__main__.ChromiumNotifierTest) ...
ok

----------------------------------------------------------------------
Ran 5 tests in 0.002s

OK
/b/commit-queue/workdir/tools/build/third_party/zope/__init__.py:19:
UserWarning: Module mock was already imported from
/b/commit-queue/workdir/tools/build/third_party/mock-1.0.1/mock.pyc, but
/usr/local/lib/python2.6/dist-packages/mock-1.0b1-py2.6.egg is being added to
sys.path
  import pkg_resources
Test count of multiple IMPROVE only is correct. ... ok
Test count of multiple REGRESS and IMPROVE is correct. ... ok
Test count of multiple REGRESS only is correct. ... ok
Tests email context c…
(message too large)

[commit-bot: I haz the power, 2013-02-28 19:24:20]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/agable@chromium.org/12351003/3001

[commit-bot: I haz the power, 2013-02-28 19:28:45]

Change committed as 185296

[Isaac (away), 2013-03-04 04:52:13]

This lgtm; are you sure '__builtins__' is always in returned dict?

[agable, 2013-03-04 18:21:19]

On 2013/03/04 04:52:13, Isaac wrote:
> This lgtm; are you sure '__builtins__' is always in returned dict?

Yes. It's meaning changes slightly depending on if you're in __main__ or not,
but __builtins__ always exists in local scope, so it will always be in the
returned dict.

[Isaac (away), 2013-03-04 19:26:07]

OK, SGTM