Use the new nacl_core_sdk mode for package_version within Chromium.

Use the new nacl_core_sdk mode for package_version within Chromium.

A new "nacl_core_sdk" mode has been introduced for NaCl toolchain
packages which allows the NaCl tree to specify what external facing
users of the toolchain should download. Currently it only filters out
the arm_trusted package which is not useful for Chromium. Eventually we
will also use this to filter out a set of "raw" toolchains which do not
contain the core_sdk.

R=bradnelson@chromium.org, dschuff@chromium.org
BUG=None
TEST=trybots

Committed: https://crrev.com/24988063e1b1233b450eb3756dd40b09780872b5
Cr-Commit-Position: refs/heads/master@{#305102}

[David Yen, 2014-11-18 22:10:57]

Prepared CL which uses the new "nacl_core_sdk" mode for package_version within
the chromium tree. I will wait for the next DEPS Roll before testing/landing
this.

Roping in Brad because he is the owner of the following files:
  chrome/test/nacl_test_injection/buildbot_chrome_nacl_stage.py
  native_client_sdk/src/build_tools/build_sdk.py

Let me know if the CL description does not give enough context of what the
change does.

[Derek Schuff, 2014-11-18 22:14:53]

lgtm

[bradn, 2014-11-18 23:22:49]

bradnelson@google.com changed reviewers:
+ bradnelson@google.com

[bradn, 2014-11-18 23:22:50]

lgtm

[bradnelson, 2014-11-19 18:50:47]

lgtm

[David Yen, 2014-11-20 22:01:50]

The CQ bit was checked by dyen@chromium.org

[commit-bot: I haz the power, 2014-11-20 22:04:06]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/735163002/20001

[commit-bot: I haz the power, 2014-11-20 23:02:29]

Committed patchset #2 (id:20001)

[commit-bot: I haz the power, 2014-11-20 23:04:03]

Patchset 2 (id:??) landed as
https://crrev.com/24988063e1b1233b450eb3756dd40b09780872b5
Cr-Commit-Position: refs/heads/master@{#305102}

[Alexander Potapenko, 2014-11-25 09:48:29]

glider@chromium.org changed reviewers:
+ glider@chromium.org

[Alexander Potapenko, 2014-11-25 09:48:29]

I'm seeing the following error message during runhooks:

________ running '/usr/bin/python src/build/download_nacl_toolchains.py
--exclude arm_trusted' in '/usr/local/google/ssd/chrome-git'
usage: download_nacl_toolchains.py [-h] [-v] [-q] [--platform HOST_PLATFORM]
                                   [--arch HOST_ARCH]
                                   [--package-targets PACKAGE_TARGETS]
                                   [--packages PACKAGES] [--append PACKAGE]
                                   [--exclude PACKAGE]
                                   [--packages-json PACKAGES_JSON]
                                   [--revisions-dir REVISIONS_DIR]
                                   [--dest-dir DEST_DIR] [--tar-dir TAR_DIR]
                                   [--annotate] [--cloud-bucket CLOUD_BUCKET]
                                  
{sync,getrevision,setrevision,extract,fillemptytars,list,archive,upload}
                                   ...
download_nacl_toolchains.py: error: argument command: invalid choice:
'nacl_core_sdk' (choose from 'sync', 'getrevision', 'setrevision', 'extract',
'fillemptytars', 'list', 'archive', 'upload')

Have the corresponding changes been pulled into native_client/ ?

[David Yen, 2014-11-25 18:36:08]

On 2014/11/25 09:48:29, Alexander Potapenko wrote:
> I'm seeing the following error message during runhooks:
> 
> ________ running '/usr/bin/python src/build/download_nacl_toolchains.py
> --exclude arm_trusted' in '/usr/local/google/ssd/chrome-git'
> usage: download_nacl_toolchains.py [-h] [-v] [-q] [--platform HOST_PLATFORM]
>                                    [--arch HOST_ARCH]
>                                    [--package-targets PACKAGE_TARGETS]
>                                    [--packages PACKAGES] [--append PACKAGE]
>                                    [--exclude PACKAGE]
>                                    [--packages-json PACKAGES_JSON]
>                                    [--revisions-dir REVISIONS_DIR]
>                                    [--dest-dir DEST_DIR] [--tar-dir TAR_DIR]
>                                    [--annotate] [--cloud-bucket CLOUD_BUCKET]
>                                   
> {sync,getrevision,setrevision,extract,fillemptytars,list,archive,upload}
>                                    ...
> download_nacl_toolchains.py: error: argument command: invalid choice:
> 'nacl_core_sdk' (choose from 'sync', 'getrevision', 'setrevision', 'extract',
> 'fillemptytars', 'list', 'archive', 'upload')
> 
> Have the corresponding changes been pulled into native_client/ ?

Yes, that had landed some time last week. Have you run gclient sync?

[Alexander Potapenko, 2014-11-26 09:09:21]

On 2014/11/25 18:36:08, David Yen wrote:
> On 2014/11/25 09:48:29, Alexander Potapenko wrote:
> > I'm seeing the following error message during runhooks:
> > 
> > ________ running '/usr/bin/python src/build/download_nacl_toolchains.py
> > --exclude arm_trusted' in '/usr/local/google/ssd/chrome-git'
> > usage: download_nacl_toolchains.py [-h] [-v] [-q] [--platform HOST_PLATFORM]
> >                                    [--arch HOST_ARCH]
> >                                    [--package-targets PACKAGE_TARGETS]
> >                                    [--packages PACKAGES] [--append PACKAGE]
> >                                    [--exclude PACKAGE]
> >                                    [--packages-json PACKAGES_JSON]
> >                                    [--revisions-dir REVISIONS_DIR]
> >                                    [--dest-dir DEST_DIR] [--tar-dir TAR_DIR]
> >                                    [--annotate] [--cloud-bucket
CLOUD_BUCKET]
> >                                   
> > {sync,getrevision,setrevision,extract,fillemptytars,list,archive,upload}
> >                                    ...
> > download_nacl_toolchains.py: error: argument command: invalid choice:
> > 'nacl_core_sdk' (choose from 'sync', 'getrevision', 'setrevision',
'extract',
> > 'fillemptytars', 'list', 'archive', 'upload')
> > 
> > Have the corresponding changes been pulled into native_client/ ?
> 
> Yes, that had landed some time last week. Have you run gclient sync?

The problem actually happens during gclient sync.

[David Yen, 2014-11-26 17:31:35]

On 2014/11/26 09:09:21, Alexander Potapenko wrote:
> On 2014/11/25 18:36:08, David Yen wrote:
> > On 2014/11/25 09:48:29, Alexander Potapenko wrote:
> > > I'm seeing the following error message during runhooks:
> > > 
> > > ________ running '/usr/bin/python src/build/download_nacl_toolchains.py
> > > --exclude arm_trusted' in '/usr/local/google/ssd/chrome-git'
> > > usage: download_nacl_toolchains.py [-h] [-v] [-q] [--platform
HOST_PLATFORM]
> > >                                    [--arch HOST_ARCH]
> > >                                    [--package-targets PACKAGE_TARGETS]
> > >                                    [--packages PACKAGES] [--append
PACKAGE]
> > >                                    [--exclude PACKAGE]
> > >                                    [--packages-json PACKAGES_JSON]
> > >                                    [--revisions-dir REVISIONS_DIR]
> > >                                    [--dest-dir DEST_DIR] [--tar-dir
TAR_DIR]
> > >                                    [--annotate] [--cloud-bucket
> CLOUD_BUCKET]
> > >                                   
> > > {sync,getrevision,setrevision,extract,fillemptytars,list,archive,upload}
> > >                                    ...
> > > download_nacl_toolchains.py: error: argument command: invalid choice:
> > > 'nacl_core_sdk' (choose from 'sync', 'getrevision', 'setrevision',
> 'extract',
> > > 'fillemptytars', 'list', 'archive', 'upload')
> > > 
> > > Have the corresponding changes been pulled into native_client/ ?
> > 
> > Yes, that had landed some time last week. Have you run gclient sync?
> 
> The problem actually happens during gclient sync.

Is this only on your machine? or do you see it on a trybot? It looks like the
chromium build bots look like they are all syncing correctly. Do you have a
gclient configuration that does not sync native_client? Check if your
native_client directory is synced to the pinned revision. Currently the DEPS
file has it set to:

'nacl_revision': '4b87bff882895fc945954ba07f5f80bdfc7b55e7', # from svn revision
r14137

[Alexander Potapenko, 2014-11-27 07:41:50]

> Is this only on your machine? or do you see it on a trybot? 
Nope, this is only on my machine
> It looks like the
> chromium build bots look like they are all syncing correctly. Do you have a
> gclient configuration that does not sync native_client?
No. Here's the contents of my .gclient:
===========================================================
solutions = [{u'managed': False, u'name': u'src', u'url':
u'https://chromium.googlesource.com/chromium/src.git', u'custom_deps':
{"src/third_party/valgrind":
"http://src.chromium.org/chrome/trunk/deps/third_party/valgrind/binaries",},
u'deps_file': u'.DEPS.git', u'safesync_url': u''}]
===========================================================
> Check if your
> native_client directory is synced to the pinned revision. Currently the DEPS
> file has it set to:
> 
> 'nacl_revision': '4b87bff882895fc945954ba07f5f80bdfc7b55e7', # from svn
revision
> r14137
Same revision for me.
I've also tried deleting the native_client directory and syncing again, but that
didn't help.

[Alexander Potapenko, 2014-11-27 09:09:26]

Looking at native_client/build/package_version/package_version.py I see that
it's synced to 05178d8a5a2c1a7aa8170413206dca3042896f35 (Brad's commit from Jun
5),
Not sure how this could happen. I'll try to copy package_version.py from another
checkout and see if that helps.

[Alexander Potapenko, 2014-11-27 09:20:25]

I did the following:

cd native_client/build/package_version
git checkout 4b87bff882895fc945954ba07f5f80bdfc7b55e7
cd ../../../
gclient runhooks

As a result the NaCl toolchains have been downloaded. But then I ran gclient
sync, and the problem appeared again.

[David Yen, 2014-12-01 23:13:42]

On 2014/11/27 09:20:25, Alexander Potapenko wrote:
> I did the following:
> 
> cd native_client/build/package_version
> git checkout 4b87bff882895fc945954ba07f5f80bdfc7b55e7
> cd ../../../
> gclient runhooks
> 
> As a result the NaCl toolchains have been downloaded. But then I ran gclient
> sync, and the problem appeared again.

Could the problem be because you have a stale .DEPS.git file in your src
directory? I believe it is not suppose to exist and only the DEPS file exists
now.

[bradn, 2014-12-02 00:13:38]

Also please confirm you're in a git client pointed at the current server?

[Alexander Potapenko, 2014-12-02 09:04:57]

On 2014/12/02 00:13:38, bradn wrote:
> Also please confirm you're in a git client pointed at the current server?

I think I was. At least I was able to fast-forward to the recent changes in
NaCl.

I've created another Chromium checkout and that one doesn't have this problem.