Reduce load on autotest by cutting number of forks and parsers.

Reduce load on autotest by cutting number of forks and parsers.

Forking is still required to keep logging and parsing working.  By using a multiprocessing queue instead of a normal queue ServerJob can fork once per test host instead of once per test.

Also make continuous parsing an opt in request instead of a default feature.  Continuous parsing adds a parser per host, and with many concurrent hosts in concurrent tests for concurrent builds this load adds up.  BVT runs take a few minutes and don’t need an instantaneous update in the front end.  But longer running suites will still benefit from being able to give status updates as they run.

Change-Id: I03c898d7aff4eaae44d55c5106f789fc16326783

BUG=None.
TEST=Multi-Machine BVT with and without continuous parsing.

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=e4afc77

[pauldean_chromium, 2011-04-26 15:31:18]

Hi Eric,

Here's a CL to reduce the server load while running distribute jobs.

Thanks,
paul

[ericli, 2011-04-26 16:52:45]

LGTM.