Add support for wildcards in svn remote configuration.

Add support for wildcards in svn remote configuration.

BUG=none
TEST=manual

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=78741

[Bernhard Bauer, 2011-03-01 16:50:56]

Please review.

My last CL broke things for some v8 people who are working on an svn branch
instead of trunk. This should fix it.

Thanks!

[evanm, 2011-03-01 17:29:37]

On 2011/03/01 16:50:56, Bernhard Bauer wrote:
> Please review.
> 
> My last CL broke things for some v8 people who are working on an svn branch
> instead of trunk. This should fix it.

Can you add a test?

[M-A Ruel, 2011-03-02 14:24:59]

http://codereview.chromium.org/6598068/diff/1/git_cl/git_cl.py
File git_cl/git_cl.py (right):

http://codereview.chromium.org/6598068/diff/1/git_cl/git_cl.py#newcode99
git_cl/git_cl.py:99: 
Two empty lines between file level symbols.

http://codereview.chromium.org/6598068/diff/1/git_cl/git_cl.py#newcode108
git_cl/git_cl.py:108: glob_match = re.match('(.+/)?(\*|{[^/]*})(/.+)?',
glob_spec[0])
Otherwise you could use fnmatch.fnmatch(). It won't work with '?' or '*' in
branch name but I don't think we care.

[Bernhard Bauer, 2011-03-02 14:36:05]

http://codereview.chromium.org/6598068/diff/1/git_cl/git_cl.py
File git_cl/git_cl.py (right):

http://codereview.chromium.org/6598068/diff/1/git_cl/git_cl.py#newcode99
git_cl/git_cl.py:99: 
On 2011/03/02 14:24:59, Marc-Antoine Ruel wrote:
> Two empty lines between file level symbols.

Done.

http://codereview.chromium.org/6598068/diff/1/git_cl/git_cl.py#newcode108
git_cl/git_cl.py:108: glob_match = re.match('(.+/)?(\*|{[^/]*})(/.+)?',
glob_spec[0])
On 2011/03/02 14:24:59, Marc-Antoine Ruel wrote:
> Otherwise you could use fnmatch.fnmatch(). It won't work with '?' or '*' in
> branch name but I don't think we care.

The thing is I have to support whatever git-svn is configured with, which
includes * and {foo,bar} wildcards. I thought I could get by without them, but
the recent v8 breakage proved me wrong, so I'm trying to be as correct as
possible this time.

[M-A Ruel, 2011-03-03 18:52:04]

I'd like to get these pending git-cl changes in and move git_cl.py one directory
up to start code deduplication.

[Bernhard Bauer, 2011-03-08 18:34:26]

On 2011/03/01 17:29:37, evanm wrote:
> On 2011/03/01 16:50:56, Bernhard Bauer wrote:
> > Please review.
> > 
> > My last CL broke things for some v8 people who are working on an svn branch
> > instead of trunk. This should fix it.
> 
> Can you add a test?

Done! PTAL.

[evanm, 2011-03-08 18:37:58]

Lgtm

brevity due to phone
On Mar 8, 2011 10:34 AM, <bauerb@chromium.org> wrote:
> On 2011/03/01 17:29:37, evanm wrote:
>> On 2011/03/01 16:50:56, Bernhard Bauer wrote:
>> > Please review.
>> >
>> > My last CL broke things for some v8 people who are working on an svn
>> branch
>> > instead of trunk. This should fix it.
>
>> Can you add a test?
>
> Done! PTAL.
>
> http://codereview.chromium.org/6598068/

[M-A Ruel, 2011-03-08 18:42:51]

small nit, lgtm

http://codereview.chromium.org/6598068/diff/8001/git_cl/git_cl.py
File git_cl/git_cl.py (right):

http://codereview.chromium.org/6598068/diff/8001/git_cl/git_cl.py#newcode107
git_cl/git_cl.py:107: glob_spec = glob_spec.split(':')
Do you mind putting something like:
fetch_suburl, as_ref = glob_spec.split(':', 1)

I think it would make the code easier to read than using glob_spec[N] later on.

[Bernhard Bauer, 2011-03-08 18:47:45]

http://codereview.chromium.org/6598068/diff/8001/git_cl/git_cl.py
File git_cl/git_cl.py (right):

http://codereview.chromium.org/6598068/diff/8001/git_cl/git_cl.py#newcode107
git_cl/git_cl.py:107: glob_spec = glob_spec.split(':')
On 2011/03/08 18:42:51, Marc-Antoine Ruel wrote:
> Do you mind putting something like:
> fetch_suburl, as_ref = glob_spec.split(':', 1)
> 
> I think it would make the code easier to read than using glob_spec[N] later
on.

Nice! Done.

[commit-bot: I haz the power, 2011-03-08 18:50:28]

Running presubmit hooks...
** Presubmit Messages **
3 unit tests failed.

***************
Test 'tests.gclient_scm_test' failed with code 1

...........EE...................
======================================================================
ERROR: testUpdateUnstagedConflict (__main__.GitWrapperTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_scm_test.py",
line 756, in testUpdateUnstagedConflict
    scm.update(options, (), [])
  File "/mnt/data/b/commit-queue/workdir/depot_tools/gclient_scm.py", line 363,
in update
    raise gclient_utils.Error(e.stderr)
Error: error: Your local changes to 'b' would be overwritten by merge. 
Aborting.
Please, commit your changes or stash them before you can merge.


======================================================================
ERROR: testUpdateUnstagedConflict (__main__.GitWrapperTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_scm_test.py",
line 539, in tearDown
    StdoutCheck.tearDown(self)
  File "super_mox.py", line 101, in tearDown
    self.assertEquals('', sys.stdout.getvalue())
AssertionError: '' != '\n_____ . at refs/heads/master\n'

----------------------------------------------------------------------
Ran 31 tests in 1.973s

FAILED (errors=2)


Test 'tests.gclient_smoketest' failed with code 1

......EEEEE.....EEEEEEE...............
======================================================================
ERROR: testMultiSolutions (__main__.GClientSmokeBoth)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
910, in setUp
    self.enabled = self.FAKE_REPOS.set_up_svn() and self.FAKE_REPOS.set_up_git()
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
406, in set_up_git
    self.check_port_is_free(self.git_port)
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
438, in check_port_is_free
    assert False, '%d shouldn\'t be bound' % port
AssertionError: 9418 shouldn't be bound

======================================================================
ERROR: testMultiSolutionsJobs (__main__.GClientSmokeBoth)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
910, in setUp
    self.enabled = self.FAKE_REPOS.set_up_svn() and self.FAKE_REPOS.set_up_git()
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
391, in set_up_git
    assert self.git_pid_file == None
AssertionError

======================================================================
ERROR: testMultiSolutionsMultiRev (__main__.GClientSmokeBoth)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
910, in setUp
    self.enabled = self.FAKE_REPOS.set_up_svn() and self.FAKE_REPOS.set_up_git()
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
391, in set_up_git
    assert self.git_pid_file == None
AssertionError

======================================================================
ERROR: testRecurse (__main__.GClientSmokeBoth)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
910, in setUp
    self.enabled = self.FAKE_REPOS.set_up_svn() and self.FAKE_REPOS.set_up_git()
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
391, in set_up_git
    assert self.git_pid_file == None
AssertionError

======================================================================
ERROR: testRevInfo (__main__.GClientSmokeBoth)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
910, in setUp
    self.enabled = self.FAKE_REPOS.set_up_svn() and self.FAKE_REPOS.set_up_git()
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
391, in set_up_git
    assert self.git_pid_file == None
AssertionError

======================================================================
ERROR: testRevInfo (__main__.GClientSmokeGIT)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
693, in setUp
    self.enabled = self.FAKE_REPOS.set_up_git()
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
391, in set_up_git
    assert self.git_pid_file == None
AssertionError

======================================================================
ERROR: TODO(maruel): Remove this line once this test is fixed.
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
693, in setUp
    self.enabled = self.FAKE_REPOS.set_up_git()
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
391, in set_up_git
    assert self.git_pid_file == None
AssertionError

======================================================================
ERROR: testRunHooks (__main__.GClientSmokeGIT)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
693, in setUp
    self.enabled = self.FAKE_REPOS.set_up_git()
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
391, in set_up_git
    assert self.git_pid_file == None
AssertionError

======================================================================
ERROR: testSync (__main__.GClientSmokeGIT)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
693, in setUp
    self.enabled = self.FAKE_REPOS.set_up_git()
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
391, in set_up_git
    assert self.git_pid_file == None
AssertionError

======================================================================
ERROR: TODO(maruel): This will become an error soon.
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
693, in setUp
    self.enabled = self.FAKE_REPOS.set_up_git()
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
391, in set_up_git
    assert self.git_pid_file == None
AssertionError

======================================================================
ERROR: testSyncJobs (__main__.GClientSmokeGIT)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
693, in setUp
    self.enabled = self.FAKE_REPOS.set_up_git()
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
391, in set_up_git
    assert self.git_pid_file == None
AssertionError

======================================================================
ERROR: testSyncNoSolutionName (__main__.GClientSmokeGIT)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
693, in setUp
    self.enabled = self.FAKE_REPOS.set_up_git()
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
391, in set_up_git
    assert self.git_pid_file == None
AssertionError

----------------------------------------------------------------------
Ran 38 tests in 119.673s

FAILED (errors=12)


Test 'tests.scm_unittest' failed with code 1

....['MatchSvnGlob']
F......
======================================================================
FAIL: testMembersChanged (__main__.RootTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/data/b/commit-queue/workdir/depot_tools/third_party/pymox/mox.py",
line 1618, in new_method
    func(self, *args, **kwargs)
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/scm_unittest.py",
line 46, in testMembersChanged
    self.compareMembers(scm, members)
  File "super_mox.py", line 79, in compareMembers
    self.assertEqual(actual_members, expected_members)
AssertionError: ['GIT', 'GenFakeDiff', 'GetCasedPath', 'MatchSvnGlob', 'SVN',
'ValidateEmail', 'cStringIO', 'gclient_utils', 'glob', 'logging', 'os', 're',
'shutil', 'subprocess', 'sys', 'tempfile', 'time', 'xml'] != ['GIT',
'GenFakeDiff', 'GetCasedPath', 'SVN', 'ValidateEmail', 'cStringIO',
'gclient_utils', 'glob', 'logging', 'os', 're', 'shutil', 'subprocess', 'sys',
'tempfile', 'time', 'xml']

----------------------------------------------------------------------
Ran 11 tests in 13.316s

FAILED (failures=1)
***************

** Presubmit Warnings **
Fix pylint errors first.

** Presubmit ERRORS **
Install google_appengine sdk in
/mnt/data/b/commit-queue/workdir/google_appengine

Presubmit checks took 149.7s to calculate.

[M-A Ruel, 2011-03-08 19:05:16]

It had a stray git-daemon server living, trying again.

[commit-bot: I haz the power, 2011-03-08 19:08:25]

Running presubmit hooks...
** Presubmit Messages **
3 unit tests failed.

***************
Test 'tests.gclient_scm_test' failed with code 1

...........EE...................
======================================================================
ERROR: testUpdateUnstagedConflict (__main__.GitWrapperTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_scm_test.py",
line 756, in testUpdateUnstagedConflict
    scm.update(options, (), [])
  File "/mnt/data/b/commit-queue/workdir/depot_tools/gclient_scm.py", line 363,
in update
    raise gclient_utils.Error(e.stderr)
Error: error: Your local changes to 'b' would be overwritten by merge. 
Aborting.
Please, commit your changes or stash them before you can merge.


======================================================================
ERROR: testUpdateUnstagedConflict (__main__.GitWrapperTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_scm_test.py",
line 539, in tearDown
    StdoutCheck.tearDown(self)
  File "super_mox.py", line 101, in tearDown
    self.assertEquals('', sys.stdout.getvalue())
AssertionError: '' != '\n_____ . at refs/heads/master\n'

----------------------------------------------------------------------
Ran 31 tests in 1.772s

FAILED (errors=2)


Test 'tests.gclient_smoketest' failed with code 1

.......F..............................
======================================================================
FAIL: testMultiSolutionsJobs (__main__.GClientSmokeBoth)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
958, in testMultiSolutionsJobs
    untangle=True)
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
76, in parseGclient
    return self.checkBlock(stdout, items)
  File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py", line
129, in checkBlock
    self.checkString(results[i][0][2], path, (i, results[i][0][2], path))
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py", line
733, in checkString
    self.assertEquals(expected, result, msg)
AssertionError: (3,
'/tmp/triald9M1b3/__main__.GClientSmokeBoth.testMultiSolutionsJobs',
'/tmp/triald9M1b3/__main__.GClientSmokeBoth.testMultiSolutionsJobs/src/file/other')

----------------------------------------------------------------------
Ran 38 tests in 153.573s

FAILED (failures=1)


Test 'tests.scm_unittest' failed with code 1

....['MatchSvnGlob']
F......
======================================================================
FAIL: testMembersChanged (__main__.RootTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/data/b/commit-queue/workdir/depot_tools/third_party/pymox/mox.py",
line 1618, in new_method
    func(self, *args, **kwargs)
  File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/scm_unittest.py",
line 46, in testMembersChanged
    self.compareMembers(scm, members)
  File "super_mox.py", line 79, in compareMembers
    self.assertEqual(actual_members, expected_members)
AssertionError: ['GIT', 'GenFakeDiff', 'GetCasedPath', 'MatchSvnGlob', 'SVN',
'ValidateEmail', 'cStringIO', 'gclient_utils', 'glob', 'logging', 'os', 're',
'shutil', 'subprocess', 'sys', 'tempfile', 'time', 'xml'] != ['GIT',
'GenFakeDiff', 'GetCasedPath', 'SVN', 'ValidateEmail', 'cStringIO',
'gclient_utils', 'glob', 'logging', 'os', 're', 'shutil', 'subprocess', 'sys',
'tempfile', 'time', 'xml']

----------------------------------------------------------------------
Ran 11 tests in 13.330s

FAILED (failures=1)
***************

** Presubmit Warnings **
Fix pylint errors first.

** Presubmit ERRORS **
Install google_appengine sdk in
/mnt/data/b/commit-queue/workdir/google_appengine

Presubmit checks took 184.0s to calculate.

[M-A Ruel, 2011-03-08 19:11:34]

> Test 'tests.gclient_scm_test' failed with code 1
> 
> ...........EE...................
> ======================================================================
> ERROR: testUpdateUnstagedConflict (__main__.GitWrapperTestCase)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_scm_test.py",
> line 756, in testUpdateUnstagedConflict
>     scm.update(options, (), [])
>   File "/mnt/data/b/commit-queue/workdir/depot_tools/gclient_scm.py", line
363,
> in update
>     raise gclient_utils.Error(e.stderr)
> Error: error: Your local changes to 'b' would be overwritten by merge. 
> Aborting.
> Please, commit your changes or stash them before you can merge.
> 
> 
> ======================================================================
> ERROR: testUpdateUnstagedConflict (__main__.GitWrapperTestCase)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
"/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_scm_test.py",
> line 539, in tearDown
>     StdoutCheck.tearDown(self)
>   File "super_mox.py", line 101, in tearDown
>     self.assertEquals('', sys.stdout.getvalue())
> AssertionError: '' != '\n_____ . at refs/heads/master\n'
> 
> ----------------------------------------------------------------------
> Ran 31 tests in 1.772s
> 
> FAILED (errors=2)

No idea why this one failed.



> Test 'tests.gclient_smoketest' failed with code 1
> 
> .......F..............................
> ======================================================================
> FAIL: testMultiSolutionsJobs (__main__.GClientSmokeBoth)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py",
line
> 958, in testMultiSolutionsJobs
>     untangle=True)
>   File
> "/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py",
line
> 76, in parseGclient
>     return self.checkBlock(stdout, items)
>   File
> "/mnt/data/b/commit-queue/workdir/depot_tools/tests/gclient_smoketest.py",
line
> 129, in checkBlock
>     self.checkString(results[i][0][2], path, (i, results[i][0][2], path))
>   File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/fake_repos.py",
line
> 733, in checkString
>     self.assertEquals(expected, result, msg)
> AssertionError: (3,
> '/tmp/triald9M1b3/__main__.GClientSmokeBoth.testMultiSolutionsJobs',
>
'/tmp/triald9M1b3/__main__.GClientSmokeBoth.testMultiSolutionsJobs/src/file/other')
> 
> ----------------------------------------------------------------------
> Ran 38 tests in 153.573s
> 
> FAILED (failures=1)

No idea why this one failed.



> Test 'tests.scm_unittest' failed with code 1
> 
> ....['MatchSvnGlob']
> F......
> ======================================================================
> FAIL: testMembersChanged (__main__.RootTestCase)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
"/mnt/data/b/commit-queue/workdir/depot_tools/third_party/pymox/mox.py",
> line 1618, in new_method
>     func(self, *args, **kwargs)
>   File "/mnt/data/b/commit-queue/workdir/depot_tools/tests/scm_unittest.py",
> line 46, in testMembersChanged
>     self.compareMembers(scm, members)
>   File "super_mox.py", line 79, in compareMembers
>     self.assertEqual(actual_members, expected_members)
> AssertionError: ['GIT', 'GenFakeDiff', 'GetCasedPath', 'MatchSvnGlob', 'SVN',
> 'ValidateEmail', 'cStringIO', 'gclient_utils', 'glob', 'logging', 'os', 're',
> 'shutil', 'subprocess', 'sys', 'tempfile', 'time', 'xml'] != ['GIT',
> 'GenFakeDiff', 'GetCasedPath', 'SVN', 'ValidateEmail', 'cStringIO',
> 'gclient_utils', 'glob', 'logging', 'os', 're', 'shutil', 'subprocess', 'sys',
> 'tempfile', 'time', 'xml']

This one is genuine.


> ** Presubmit ERRORS **
> Install google_appengine sdk in
> /mnt/data/b/commit-queue/workdir/google_appengine

Argh, we need to have it auto-download... I'll download it manually in the
meantime.

[Bernhard Bauer, 2011-03-15 09:32:48]

I tried locally, and the presubmit checks passed, except for
GClientSmokeBoth.testMultiSolutionsJobs, which failed even on trunk (but not
when run in isolation, interestingly).

[commit-bot: I haz the power, 2011-03-15 17:14:24]

Running presubmit hooks...
Checking out rietveld...
Checking out django...
'LocalRietveld' object has no attribute 'out'

[M-A Ruel, 2011-03-15 17:16:56]

Will fix real soon (tm).

Le 15 mars 2011 13:14, <commit-bot@chromium.org> a écrit :

> Running presubmit hooks...
> Checking out rietveld...
> Checking out django...
> 'LocalRietveld' object has no attribute 'out'
>
>
> http://codereview.chromium.org/6598068/
>

[commit-bot: I haz the power, 2011-03-18 20:09:33]

Change committed as 78741

[M-A Ruel, 2011-03-18 20:10:21]

On 2011/03/15 17:16:56, Marc-Antoine Ruel wrote:
> Will fix real soon (tm).

For a certain quantity of 'soon'... :)