Enable autotest client package fetching based on updated image.

Enable autotest client package fetching based on updated image.

The new customization will enable autotest client to fetch a test package
from dev server based on its update url. This could avoid to pre-load all
client packages on stateful partition and shrinked its space from 450M ~ 90M
approximately.

It will only be activated after we add a fetch_location entry in global_config.ini.

Change-Id: I0f57a856b9d78c3245f1ff2a841acd98d6d3717e

BUG=None
TEST=server/autoserv -m IP {client,server}/tests/sleeptest/control --image update_url

carefully read autoserv and client DEBUG log to see packages were downloaded from the dev server.

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=2eb800f

[DaleCurtis, 2011-04-21 19:16:55]

http://codereview.chromium.org/6883099/diff/1/client/common_lib/site_packages.py
File client/common_lib/site_packages.py (right):

http://codereview.chromium.org/6883099/diff/1/client/common_lib/site_packages...
client/common_lib/site_packages.py:2: from autotest_lib.client.common_lib import
global_config
You don't use any of these imports except for base_packages

http://codereview.chromium.org/6883099/diff/1/server/site_autotest.py
File server/site_autotest.py (right):

http://codereview.chromium.org/6883099/diff/1/server/site_autotest.py#newcode20
server/site_autotest.py:20: c = global_config.global_config
You only use this once, might as well combine with repos =.

http://codereview.chromium.org/6883099/diff/1/server/site_autotest.py#newcode21
server/site_autotest.py:21: repos = c.get_config_value("PACKAGES",
'fetch_location', type=list,
Single quotes.

http://codereview.chromium.org/6883099/diff/1/server/site_autotest.py#newcode26
server/site_autotest.py:26: build = '/'.join(urlparse.urlparse(
No real reason for the urlparse. [-2:] will return the same thing with the full
URL. If you really want to be safe you should add a rstrip('/') to the path.
Otherwise if a image url comes in w/ a trailing slash this will fail.

http://codereview.chromium.org/6883099/diff/1/server/site_autotest.py#newcode28
server/site_autotest.py:28: repo += '/' + build + '/autotest'
'/%s/autotest' % build

http://codereview.chromium.org/6883099/diff/1/server/site_autotest.py#newcode30
server/site_autotest.py:30: repos = new_repos
Just "return new_repos"

[ericli, 2011-04-21 20:26:37]

Thanks for your suggestion.

PTAL.

[DaleCurtis, 2011-04-21 20:35:14]

http://codereview.chromium.org/6883099/diff/5/client/common_lib/site_packages.py
File client/common_lib/site_packages.py (right):

http://codereview.chromium.org/6883099/diff/5/client/common_lib/site_packages...
client/common_lib/site_packages.py:1: import logging, os
logging,os are still unused.

http://codereview.chromium.org/6883099/diff/5/server/site_autotest.py
File server/site_autotest.py (right):

http://codereview.chromium.org/6883099/diff/5/server/site_autotest.py#newcode7
server/site_autotest.py:7: c = global_config.global_config
Seems strange to have a global variable with a single letter name. Up to you,
but I'd use config or something else.

[ericli, 2011-04-21 20:54:53]

change the logic of image parameter handling. PTAL.

[DaleCurtis, 2011-04-21 21:51:02]

LGTM % nits.

http://codereview.chromium.org/6883099/diff/6002/server/site_autotest.py
File server/site_autotest.py (right):

http://codereview.chromium.org/6883099/diff/6002/server/site_autotest.py#newc...
server/site_autotest.py:6: parser = autoserv_parser.autoserv_parser
Alphabetize config/parser.

http://codereview.chromium.org/6883099/diff/6002/server/site_autotest.py#newc...
server/site_autotest.py:8: 
Two lines between top level.

http://codereview.chromium.org/6883099/diff/6002/server/site_autotest.py#newc...
server/site_autotest.py:28: path = parser.options.image.rstrip('/')
No need to compute path or build in loop. Move them out.

[ericli, 2011-04-21 23:48:12]

http://codereview.chromium.org/6883099/diff/6002/server/site_autotest.py
File server/site_autotest.py (right):

http://codereview.chromium.org/6883099/diff/6002/server/site_autotest.py#newc...
server/site_autotest.py:6: parser = autoserv_parser.autoserv_parser
On 2011/04/21 21:51:02, dalec wrote:
> Alphabetize config/parser.

Done.

http://codereview.chromium.org/6883099/diff/6002/server/site_autotest.py#newc...
server/site_autotest.py:8: 
On 2011/04/21 21:51:02, dalec wrote:
> Two lines between top level.

Done.

http://codereview.chromium.org/6883099/diff/6002/server/site_autotest.py#newc...
server/site_autotest.py:28: path = parser.options.image.rstrip('/')
No, I would say only compute inside loop.

We will only compute the path when certain pattern appears in the repo list, and
we know for sure it will only appear once.

On 2011/04/21 21:51:02, dalec wrote:
> No need to compute path or build in loop. Move them out.