[Findit] Make Findit more robust.

[Findit] Make Findit more robust.

1. Add retry to http_client.
2. Make Findit more robust for unrecognized chromium revision.
3. Workaround a harmless bug in python 2.7
4. Give a better error message for check failure.

NOTRY=true

Committed: https://crrev.com/482979416f81edf748be33194125c926f66c2f97
Cr-Commit-Position: refs/heads/master@{#293528}

[stgao, 2014-09-05 06:39:49]

Patchset #1 (id:1) has been deleted

[stgao, 2014-09-05 08:33:17]

stgao@chromium.org changed reviewers:
+ aarya@google.com, mbarbella@chromium.org

[stgao, 2014-09-05 08:33:18]

https://codereview.chromium.org/538383002/diff/80001/tools/findit/crash_utils.py
File tools/findit/crash_utils.py (left):

https://codereview.chromium.org/538383002/diff/80001/tools/findit/crash_utils...
tools/findit/crash_utils.py:26: function, args, kwargs, result_semaphore =
TASK_QUEUE.get()
The exception is as below:

Exception in thread worker_9 (most likely raised during interpreter shutdown):  

Traceback (most recent call last):                                              

  File "/usr/lib/python2.7/threading.py", line 808, in __bootstrap_inner        

  File "/usr/lib/python2.7/threading.py", line 761, in run                      

  File "/mnt/scratch0/clusterfuzz/src/tools/findit/crash_utils.py", line 26, in
Worker
  File "/usr/lib/python2.7/Queue.py", line 168, in get                          

  File "/usr/lib/python2.7/threading.py", line 332, in wait                     

<type 'exceptions.TypeError'>: 'NoneType' object is not callable

https://codereview.chromium.org/538383002/diff/80001/tools/findit/crash_utils.py
File tools/findit/crash_utils.py (right):

https://codereview.chromium.org/538383002/diff/80001/tools/findit/crash_utils...
tools/findit/crash_utils.py:47: print '%s is quitting' %
threading.currentThread().name
I will revert the print before committing.
But I'd like to show this fix will work, because the order of quitting is as
below:

worker_3 is quitting
 worker_2 is quitting
 worker_7 is quitting
worker_1 is quitting
worker_6 is quitting
 worker_8 is quitting
worker_5 is quitting
worker_9 is quitting
 worker_0 is quitting
worker_4 is quitting
Dummy-1 is quitting

[aarya, 2014-09-05 15:19:06]

lgtm

[aarya, 2014-09-05 15:19:29]

lgtm.

please make sure to remove the two prints before committing. Also please update
cf copy.

[stgao, 2014-09-05 15:41:41]

The CQ bit was checked by stgao@chromium.org

[stgao, 2014-09-05 15:42:27]

On 2014/09/05 15:19:29, aarya wrote:
> lgtm.
> 
> please make sure to remove the two prints before committing. Also please
update
> cf copy.

Thanks.
Will update cf later after this CL is in.

[commit-bot: I haz the power, 2014-09-05 15:43:20]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/stgao@chromium.org/538383002/100001

[commit-bot: I haz the power, 2014-09-05 16:00:02]

Committed patchset #5 (id:100001) as 128f2eaff507f7abfc01c0b213c492f45b93f084

[commit-bot: I haz the power, 2014-09-10 03:39:49]

Patchset 5 (id:??) landed as
https://crrev.com/482979416f81edf748be33194125c926f66c2f97
Cr-Commit-Position: refs/heads/master@{#293528}