rebaseline_server: periodically log tasks_queue size

rebaseline_server: periodically log tasks_queue size

Without this, it's hard to tell whether the server is stuck (or why it's taking so long).

NOTRY=true

Committed: https://skia.googlesource.com/skia/+/9c7695b0b59b97da933cd11014c922344a8d7654

[epoger, 2014-08-08 18:35:19]

Stephan, PTAL (please take a look) at patchset 1.

Here's sample output:

08/08/2014 14:27:57 INFO Reading JSON image summaries from dirs
/tmp/tmpv4mQTL/setA and /tmp/tmpv4mQTL/setB...
08/08/2014 14:27:57 INFO Comparing summary dicts...
08/08/2014 14:27:57 INFO Starting to add imagepairs to queue.
08/08/2014 14:27:57 INFO Asynchronously requesting pixel diffs for dict #1 of 4,
"./renderskp-defaults.json"...
08/08/2014 14:27:58 INFO tasks_queue size is 1000
08/08/2014 14:27:59 INFO tasks_queue size is 2000
08/08/2014 14:28:00 INFO tasks_queue size is 3000
08/08/2014 14:28:00 INFO tasks_queue size is 4000
08/08/2014 14:28:02 INFO tasks_queue size is 5000
08/08/2014 14:28:03 INFO tasks_queue size is 6000
08/08/2014 14:28:04 INFO Asynchronously requesting pixel diffs for dict #2 of 4,
"./renderskp-deferImageDecoding.json"...
08/08/2014 14:28:05 INFO tasks_queue size is 7000
08/08/2014 14:28:06 INFO tasks_queue size is 8000
08/08/2014 14:28:08 INFO tasks_queue size is 9000
08/08/2014 14:28:09 INFO tasks_queue size is 10000
08/08/2014 14:28:10 INFO tasks_queue size is 11000
08/08/2014 14:28:12 INFO tasks_queue size is 12000
08/08/2014 14:28:14 INFO tasks_queue size is 13000
08/08/2014 14:28:15 INFO Asynchronously requesting pixel diffs for dict #3 of 4,
"./renderskp-grid.json"...
08/08/2014 14:28:20 INFO Asynchronously requesting pixel diffs for dict #4 of 4,
"./renderskp-rtree.json"...
08/08/2014 14:28:21 INFO Finished adding imagepairs to queue.
08/08/2014 14:28:21 INFO tasks_queue size is 12950
08/08/2014 14:29:05 INFO tasks_queue size is 12000
08/08/2014 14:29:48 INFO tasks_queue size is 11000
08/08/2014 14:30:24 INFO tasks_queue size is 10000
08/08/2014 14:31:06 INFO tasks_queue size is 9000
08/08/2014 14:31:51 INFO tasks_queue size is 8000
08/08/2014 14:32:35 INFO tasks_queue size is 7000
08/08/2014 14:32:45 INFO tasks_queue size is 6000
08/08/2014 14:32:45 INFO tasks_queue size is 5000
08/08/2014 14:32:45 INFO tasks_queue size is 4000
08/08/2014 14:32:45 INFO tasks_queue size is 3000
08/08/2014 14:32:45 INFO tasks_queue size is 2000
08/08/2014 14:32:46 INFO tasks_queue size is 1000
08/08/2014 14:32:46 INFO tasks_queue size is 0
08/08/2014 14:32:46 INFO Number of download file collisions: 0
08/08/2014 14:32:46 INFO Results complete; took 289 seconds.

[stephana, 2014-08-11 13:35:18]

https://codereview.chromium.org/454953002/diff/1/gm/rebaseline_server/imagedi...
File gm/rebaseline_server/imagediffdb.py (right):

https://codereview.chromium.org/454953002/diff/1/gm/rebaseline_server/imagedi...
gm/rebaseline_server/imagediffdb.py:301: 
This is a nit, but why do you need a lock for reading the queue size ?

[epoger, 2014-08-11 18:49:09]

Thanks, Stephan...

https://codereview.chromium.org/454953002/diff/1/gm/rebaseline_server/imagedi...
File gm/rebaseline_server/imagediffdb.py (right):

https://codereview.chromium.org/454953002/diff/1/gm/rebaseline_server/imagedi...
gm/rebaseline_server/imagediffdb.py:301: 
On 2014/08/11 13:35:18, stephana wrote:
> This is a nit, but why do you need a lock for reading the queue size ?

Actually, we need it for self._last_queue_size_reported; without it, several
threads can report at the same queue size.  Added a comment line to hopefully
make that clearer.

[stephana, 2014-08-11 19:27:03]

On 2014/08/11 18:49:09, epoger wrote:
> Thanks, Stephan...
> 
>
https://codereview.chromium.org/454953002/diff/1/gm/rebaseline_server/imagedi...
> File gm/rebaseline_server/imagediffdb.py (right):
> 
>
https://codereview.chromium.org/454953002/diff/1/gm/rebaseline_server/imagedi...
> gm/rebaseline_server/imagediffdb.py:301: 
> On 2014/08/11 13:35:18, stephana wrote:
> > This is a nit, but why do you need a lock for reading the queue size ?
> 
> Actually, we need it for self._last_queue_size_reported; without it, several
> threads can report at the same queue size.  Added a comment line to hopefully
> make that clearer.

Thanks. 
LGTM

[epoger, 2014-08-12 15:28:06]

The CQ bit was checked by epoger@google.com

[commit-bot: I haz the power, 2014-08-12 15:28:58]

CQ is trying da patch. Follow status at
 https://skia-tree-status.appspot.com/cq/epoger@google.com/454953002/20001

[commit-bot: I haz the power, 2014-08-12 15:28:59]

Note for Reviewers:
The CQ is waiting for an approval.
If you believe that the CL is not ready yet, or if you would like to L-G-T-M
with comments then please uncheck the CQ checkbox.

Waiting for LGTM from a valid reviewer till 2014-08-12 21:28 UTC

[rmistry, 2014-08-12 15:29:46]

LGTM

I need to add Stephen to the Skia committers list.

[epoger, 2014-08-12 15:29:58]

The CQ bit was unchecked by epoger@google.com

[epoger, 2014-08-12 15:29:58]

The CQ bit was checked by epoger@google.com

[commit-bot: I haz the power, 2014-08-12 15:30:03]

CQ is trying da patch. Follow status at
 https://skia-tree-status.appspot.com/cq/epoger@google.com/454953002/20001

[commit-bot: I haz the power, 2014-08-12 15:30:15]

Change committed as 9c7695b0b59b97da933cd11014c922344a8d7654