CQ: catch IOError thrown on socket timeouts

commit-queue/verification/try_job_on_rietveld.py

 # coding=utf8
 # Copyright (c) 2012 The Chromium Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 """Sends patches to the Try server and reads back results.
 
 - RietveldTryJobs contains RietveldTryJob, one per try job on a builder.
 - TryRunnerRietveld uses Rietveld to signal and poll job results.
 """
 
@@ skipping 729 matching lines @@
           status,
           self.context.checkout,
           now)
     except urllib2.HTTPError as e:
       if e.code in (500, 502, 503):
         # Temporary AppEngine hiccup. Just log it and return failure.
         logging.warning(str(e))
         return False
       else:
         raise
     except socket.error as e:

[Vadim Sh., 2013/11/08 19:33:57]

When making HTTPS requests with client-side timeout urllib2 raises ssl.SSLError
(that's completely unrelated to IOError or socket.error). I think you need to
catch it too.

       # Temporary AppEngine hiccup. Just log it and return failure.
       if e.errno == errno.ECONNRESET:

[Vadim Sh., 2013/11/08 19:33:57]

It'll only catch timeouts while opening a connection.

If connection was established, but socket.read() later times out, it's a
different exception (socket.timeout I think, that's inherited from
socket.error).

         logging.warning(str(e))
         return False
       else:
         raise
+    except IOError as e:
+      # Temporary AppEngine hiccup. Just log it and return failure.
+      if e.errno == 'socket error':

[Vadim Sh., 2013/11/08 19:33:57]

errno is a string?.. How can this happen?

[Paweł Hajdan Jr., 2013/11/08 20:14:19]

I believe it can. All of this is based on what I've seen happening.

When I see the other errors I'll add appropriate error handling. I prefer not to
be too speculative about this.

+        logging.warning(str(e))
+        return False
+      raise
     if handle:
       for updated_key in keys:
         job = jobs.try_jobs[updated_key]
         self._update_dashboard(pending, job, status)
         jobs.signal_as_failed_if_needed(job, self._build_status_url(job), now)
 
     return True