Add flock and timeout to infra/libs.

Add flock and timeout to infra/libs.

These currently fully work for linux only (flock works for osx as well).

BUG=479059
Committed: https://chromium.googlesource.com/infra/infra/+/4be68e66fc5997711d58af8a8b77bdc6f35a16a3

[ghost stip (do not use), 2015-04-23 06:08:49]

stip@chromium.org changed reviewers:
+ agable@chromium.org, iannucci@chromium.org

[ghost stip (do not use), 2015-04-23 06:08:54]

ptal

[tandrii(chromium), 2015-04-23 13:02:47]

tandrii@chromium.org changed reviewers:
+ tandrii@chromium.org

[tandrii(chromium), 2015-04-23 13:02:47]

lgtm

I've written something very similar before, and had problems with it in 3 cases:
a) weird filesystem mount options combined with bugs in kernel. Given the
context, we are likely ext3/4 on Linux, so should be OK.
b) forking processes inheriting file descriptors.
c) others misusing this flock in threads, usually indirectly by calling flocking
routines in many threads for increasing speed.

https://codereview.chromium.org/1096683003/diff/1/infra/libs/service_utils/da...
File infra/libs/service_utils/daemon.py (right):

https://codereview.chromium.org/1096683003/diff/1/infra/libs/service_utils/da...
infra/libs/service_utils/daemon.py:25: """Keeps a critical section from
executing concurrently using a file lock.
You should document that this lock only gives exclusivity to processes, not
threads. That is, two threads in the same processes will both happily acquire
lock at the same time.

https://codereview.chromium.org/1096683003/diff/1/infra/libs/service_utils/da...
infra/libs/service_utils/daemon.py:36: with daemon.flock('toaster') as acquired:
IMO, it's more Pythonic to raise exception instead of yielding False.

[agable, 2015-04-27 18:48:33]

LGTM with some comments.

https://codereview.chromium.org/1096683003/diff/1/infra/libs/service_utils/da...
File infra/libs/service_utils/daemon.py (right):

https://codereview.chromium.org/1096683003/diff/1/infra/libs/service_utils/da...
infra/libs/service_utils/daemon.py:36: with daemon.flock('toaster') as acquired:
On 2015/04/23 at 13:02:47, tandrii(chromium) wrote:
> IMO, it's more Pythonic to raise exception instead of yielding False.

So then the construct would be

try:
  with daemon.flock('toaster') as acq:
    put_bread_in_toaster()
except LockAlreadyLocked:
  print 'toaster is occupied!'

? That seems fine to me. I agree that it is more pythonic.

https://codereview.chromium.org/1096683003/diff/1/infra/libs/service_utils/da...
infra/libs/service_utils/daemon.py:95: """Adds a timeout to a command."""
Note that it does so using the linux /bin/timeout command.

[tandrii(chromium), 2015-04-27 21:17:09]

https://codereview.chromium.org/1096683003/diff/1/infra/libs/service_utils/da...
File infra/libs/service_utils/daemon.py (right):

https://codereview.chromium.org/1096683003/diff/1/infra/libs/service_utils/da...
infra/libs/service_utils/daemon.py:36: with daemon.flock('toaster') as acquired:
On 2015/04/27 18:48:33, agable wrote:
> On 2015/04/23 at 13:02:47, tandrii(chromium) wrote:
> > IMO, it's more Pythonic to raise exception instead of yielding False.
> 
> So then the construct would be
> 
> try:
>   with daemon.flock('toaster') as acq:
>     put_bread_in_toaster()
> except LockAlreadyLocked:
>   print 'toaster is occupied!'
> 
> ? That seems fine to me. I agree that it is more pythonic.

Yep, and you don't even need "as acq" part :)

[ghost stip (do not use), 2015-04-27 22:03:47]

https://chromiumcodereview.appspot.com/1096683003/diff/1/infra/libs/service_u...
File infra/libs/service_utils/daemon.py (right):

https://chromiumcodereview.appspot.com/1096683003/diff/1/infra/libs/service_u...
infra/libs/service_utils/daemon.py:36: with daemon.flock('toaster') as acquired:
On 2015/04/27 21:17:09, tandrii(chromium) wrote:
> On 2015/04/27 18:48:33, agable wrote:
> > On 2015/04/23 at 13:02:47, tandrii(chromium) wrote:
> > > IMO, it's more Pythonic to raise exception instead of yielding False.
> > 
> > So then the construct would be
> > 
> > try:
> >   with daemon.flock('toaster') as acq:
> >     put_bread_in_toaster()
> > except LockAlreadyLocked:
> >   print 'toaster is occupied!'
> > 
> > ? That seems fine to me. I agree that it is more pythonic.
> 
> Yep, and you don't even need "as acq" part :)

Done.

https://chromiumcodereview.appspot.com/1096683003/diff/1/infra/libs/service_u...
infra/libs/service_utils/daemon.py:95: """Adds a timeout to a command."""
On 2015/04/27 18:48:33, agable wrote:
> Note that it does so using the linux /bin/timeout command.

Done.

[ghost stip (do not use), 2015-04-27 22:03:51]

The CQ bit was checked by stip@chromium.org

[ghost stip (do not use), 2015-04-27 22:03:51]

The patchset sent to the CQ was uploaded after l-g-t-m from agable@chromium.org,
tandrii@chromium.org
Link to the patchset: https://chromiumcodereview.appspot.com/1096683003/#ps60001
(title: "Correct typos.")

[commit-bot: I haz the power, 2015-04-27 22:04:01]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1096683003/60001

[commit-bot: I haz the power, 2015-04-27 22:10:23]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-04-27 22:10:24]

Try jobs failed on following builders:
  infra_tester on tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/infra_tester/bu...)

[ghost stip (do not use), 2015-04-27 22:17:50]

The CQ bit was checked by stip@chromium.org

[ghost stip (do not use), 2015-04-27 22:17:50]

The patchset sent to the CQ was uploaded after l-g-t-m from agable@chromium.org,
tandrii@chromium.org
Link to the patchset: https://chromiumcodereview.appspot.com/1096683003/#ps80001
(title: "Improve tests and coverage.")

[commit-bot: I haz the power, 2015-04-27 22:18:05]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1096683003/80001

[commit-bot: I haz the power, 2015-04-27 22:21:48]

Committed patchset #5 (id:80001) as
https://chromium.googlesource.com/infra/infra/+/4be68e66fc5997711d58af8a8b77b...