Teach webkit-patch update-flaky-tests how to upload a change

Teach webkit-patch update-flaky-tests how to upload a change

I accidentally already landed part of this support
in an earlier change, but this makes --upload actually
work.  You can see an example change here:
https://codereview.chromium.org/308743011/
(which again accidentally includes this change).

This also adds a hack-and-slash version of sheriff-calendar
code which I and Eric Boren originally wrote for the AutoRollBot
but that lives in a different repository:
https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/tools/blink_roller/auto_roll.py

This combined with a cron job should make
it easy for gardeners to keep FlakyTests up to date.

Looking at 308743011, we clearly need to address some
of this marauding flakiness as FlakyTests should be
a very stable file.  Right now it is changing by the hour.

BUG=242366
Committed: https://src.chromium.org/viewvc/blink?view=rev&revision=175228

[ojan, 2014-06-01 02:59:09]

I did a random sample of the changing lines in
https://codereview.chromium.org/308743011/ and they all looked right. I think
the frequent changes is a byproduct of us not restricting to only very flaky
tests. Right now a test can only needs to fail 2x every 100 runs (not including
retry) to be considered flaky. We should make only_ignore_very_flaky more
aggressive about how flaky a test needs to be.

Alternately, we could put a concerted effort into making tests less flaky.
Although, we know that we have a systemic flakiness problem on windows http
tests.

https://codereview.chromium.org/307183002/diff/1/Tools/Scripts/webkitpy/commo...
File Tools/Scripts/webkitpy/common/net/sheriff_calendar.py (right):

https://codereview.chromium.org/307183002/diff/1/Tools/Scripts/webkitpy/commo...
Tools/Scripts/webkitpy/common/net/sheriff_calendar.py:15: '[A-Za-z0-9\.-]+$')
Nit: Why the linewrap?

https://codereview.chromium.org/307183002/diff/1/Tools/Scripts/webkitpy/commo...
Tools/Scripts/webkitpy/common/net/sheriff_calendar.py:18: def
_complete_email(name):
All these functions should be super easy to unittest. I don't see any reason not
to. It makes for more maintainable code.

https://codereview.chromium.org/307183002/diff/1/Tools/Scripts/webkitpy/tool/...
File Tools/Scripts/webkitpy/tool/commands/flakytests.py (right):

https://codereview.chromium.org/307183002/diff/1/Tools/Scripts/webkitpy/tool/...
Tools/Scripts/webkitpy/tool/commands/flakytests.py:57: def execute(self,
options, args, tool):
I'd like to see a unittest that executes this function at all. Just verifying
that it doesn't throw an error is valuable IMO.

With a mock BotTestExpectationsFactory(), it shouldn't be too hard to verify
that this command does the right thing either.

[ojan, 2014-06-01 03:02:54]

Incidentally, Win7 is >3/7 of the file and Mavericks is >2/7. Clearly there are
some systemic sources of flakiness on Win7 and Mavericks.

[Dirk Pranke, 2014-06-01 21:32:34]

On 2014/06/01 03:02:54, ojan wrote:
> Incidentally, Win7 is >3/7 of the file and Mavericks is >2/7. Clearly there
are
> some systemic sources of flakiness on Win7 and Mavericks.

lgtm.

I didn't think there were that many Mavericks failures when I glanced through
it, but i didn't look carefully. I think I have seen Mavericks flakiness before
in the past, but I don't recall noticing it recently. I wonder if something has
actually regressed.

[eseidel, 2014-06-01 21:50:46]

The CQ bit was checked by eseidel@chromium.org

[commit-bot: I haz the power, 2014-06-01 21:50:54]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/eseidel@chromium.org/307183002/80001

[commit-bot: I haz the power, 2014-06-01 23:09:30]

Change committed as 175228

[ojan, 2014-06-01 23:30:35]

> I didn't think there were that many Mavericks failures when I glanced through
> it, but i didn't look carefully. I think I have seen Mavericks flakiness
before
> in the past, but I don't recall noticing it recently. I wonder if something
has
> actually regressed.

I just grepped the file. ~200 of the ~650 lines were Mavericks.