Retry delbr on linux_router deconfig

Retry delbr on linux_router deconfig

With small but non-zero probability test bomb out doing an "addif"
in tests where there's a "config" step followed by another "config"
step.  This is because in order to do the "config", we quickly do
a "deconfig" and another "config".  In that short period of time
after hostapd is started, it might take a while for hostapd to come
to a halt, holding the bridge device open.  If you look through the
logs, you'll find that the previous "delbr" of the bridge interface
did not succeed, but its error was ignored.

More troublingly, it seems that the automated tests seem to run into
this issue much more regularly than developers do with "run_remote_tests".

This change retries the "delbr" between sleeps if the first one
fails, and then exit with an error if the last attempt is not
successful.

BUG=none (chat from dalecurtis)
TEST=Reran SecMat 00*

Revert "Move deftxkey into first config block"

This reverts commit ad52b3d64a0ee579caa77123969288f3b9fe81d2.

[DaleCurtis, 2010-09-01 20:49:55]

LGTM

[Sam Leffler, 2010-09-01 21:15:07]

http://codereview.chromium.org/3334002/diff/1/2
File server/site_linux_router.py (right):

http://codereview.chromium.org/3334002/diff/1/2#newcode328
server/site_linux_router.py:328: (self.bridgeif, result.stderr))
would it be better to wait on the hostapd pid and/or do the delbr in a subshell
where hostapd is launched? (hostapd ...; trap 0 delbr ...)&

[Paul Stewart, 2010-09-01 23:00:47]

Neither of those techniques are easily implementable and solve the base problem,
alas.

http://codereview.chromium.org/3334002/diff/1/2
File server/site_linux_router.py (right):

http://codereview.chromium.org/3334002/diff/1/2#newcode328
server/site_linux_router.py:328: (self.bridgeif, result.stderr))
On 2010/09/01 21:15:07, Sam Leffler wrote:
> would it be better to wait on the hostapd pid and/or do the delbr in a
subshell
> where hostapd is launched? (hostapd ...; trap 0 delbr ...)&

The "trap" technique just kicks the can down the road.  At some point we really
need to wait for the old hostapd to die so even if the bridge cleans up
automatically.  The bridge needs to be cleared before we fire up another
hostapd.  The "wait" idea is a nice one, but at least in ash, you can't wait for
a pid that you're not a parent of.  It's unlikely we need to do this more than
once, but we need to cover this likelihood.

[Sam Leffler, 2010-09-01 23:35:37]

LGTM then