Check library version and handle library load errors

Check library version and handle library load errors

This CL modifies the build to incorporate the expected C++ library version
in the Java code. This is then checked when the library is loaded, to
make sure that the C++ and Java builds match.

This CL also implements error handling when library loads fail or the
loaded version doesn't match the expected version.

BUG=311644
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=237779

[aberent, 2013-11-11 20:13:27]

yfriedman@chromium.org: Please review all changes. 

piman@chromium.org: Please review, as OWNER, changes to
android_library_loader_hooks.h and results_code_list.h

joth@chromium.org: Please review, as OWNER, changes in android_webview. 

Clank side CL follows.

[joth, 2013-11-11 20:28:56]

https://codereview.chromium.org/59533009/diff/1/android_webview/java/generate...
File
android_webview/java/generated_src/org/chromium/content/app/NativeLibraries.java
(right):

https://codereview.chromium.org/59533009/diff/1/android_webview/java/generate...
android_webview/java/generated_src/org/chromium/content/app/NativeLibraries.java:21:
static String versionName = "";
nit: sVersionName (and libraries should be too)

oh, or does the TODO cover both of them? 

(can the be static final? in which case LIBRARIES and VERSION_NAME would be
better)

https://codereview.chromium.org/59533009/diff/1/android_webview/java/src/org/...
File android_webview/java/src/org/chromium/android_webview/AwBrowserProcess.java
(right):

https://codereview.chromium.org/59533009/diff/1/android_webview/java/src/org/...
android_webview/java/src/org/chromium/android_webview/AwBrowserProcess.java:49:
if( !BrowserStartupController.get(context).startBrowserProcessesSync(
this is a bit tricky now - it can return an error boolean return code, and can
throw exception. Could we move it to just do one or other? (Probably exception)

https://codereview.chromium.org/59533009/diff/1/chrome/app/android/chrome_and...
File chrome/app/android/chrome_android_initializer.cc (right):

https://codereview.chromium.org/59533009/diff/1/chrome/app/android/chrome_and...
chrome/app/android/chrome_android_initializer.cc:24:
content::SetVersionName(vi.Version());
Version() returns the version "number" so bit confusing to pass it to set
version "name".

(maybe a comment on content::SetVersionName explaining exactly what the string
set is for, and how it is used, would make sense)

https://codereview.chromium.org/59533009/diff/1/content/app/android/library_l...
File content/app/android/library_loader_hooks.cc (right):

https://codereview.chromium.org/59533009/diff/1/content/app/android/library_l...
content/app/android/library_loader_hooks.cc:39: std::string library_version_name
= "";
g_library_version_name...

https://codereview.chromium.org/59533009/diff/1/content/app/android/library_l...
content/app/android/library_loader_hooks.cc:39: std::string library_version_name
= "";
this is a global C++ object, not allowed by style guide.

It'd be nice if you could just make this a const char*
I think that means the caller should just #include chrome_version_info_posix.h
and pass PRODUCT_VERSION as the param.


Tangent: for webview we'd really like chrome_version_info_posix.h (and VERSION
text file) moved into content/ layer so we can use the same version number in
webview UA in future. Seems like doing that would make this CL much simpler too?
(the GetVersinNumber JNI method would just return PRODUCT_VERSION converted to
java string)

https://codereview.chromium.org/59533009/diff/1/content/app/android/library_l...
content/app/android/library_loader_hooks.cc:150: void SetVersionName(const
std::string & version_name) {
not space before &

(and in header)

[Yaron, 2013-11-12 02:58:18]

https://codereview.chromium.org/59533009/diff/1/android_webview/java/src/org/...
File android_webview/java/src/org/chromium/android_webview/AwBrowserProcess.java
(right):

https://codereview.chromium.org/59533009/diff/1/android_webview/java/src/org/...
android_webview/java/src/org/chromium/android_webview/AwBrowserProcess.java:49:
if( !BrowserStartupController.get(context).startBrowserProcessesSync(
On 2013/11/11 20:28:56, joth wrote:
> this is a bit tricky now - it can return an error boolean return code, and can
> throw exception. Could we move it to just do one or other? (Probably
exception)

Ya, I tend to agree with joth

https://codereview.chromium.org/59533009/diff/1/chrome/android/testshell/java...
File
chrome/android/testshell/java/src/org/chromium/chrome/testshell/signin/AccountsChangedReceiver.java
(left):

https://codereview.chromium.org/59533009/diff/1/chrome/android/testshell/java...
chrome/android/testshell/java/src/org/chromium/chrome/testshell/signin/AccountsChangedReceiver.java:55:
BrowserStartupController.get(context).startBrowserProcessesAsync(callback);
Make this always throw would help ensure we don't swallow failure cases such as
here.

https://codereview.chromium.org/59533009/diff/1/content/shell/android/browser...
File
content/shell/android/browsertests_apk/src/org/chromium/content_browsertests_apk/ContentBrowserTestsActivity.java
(right):

https://codereview.chromium.org/59533009/diff/1/content/shell/android/browser...
content/shell/android/browsertests_apk/src/org/chromium/content_browsertests_apk/ContentBrowserTestsActivity.java:37:
System.exit(-1);
I think finish() is the cleaner way to shut down the activity. I suppose the
process won't evert start correctly but I'm a little hesitant to exit as it's
generally discouraged

[piman, 2013-11-12 19:27:19]

For content/, what joth@ said, after that LGTM.

[aberent, 2013-11-13 20:20:52]

https://codereview.chromium.org/59533009/diff/1/android_webview/java/generate...
File
android_webview/java/generated_src/org/chromium/content/app/NativeLibraries.java
(right):

https://codereview.chromium.org/59533009/diff/1/android_webview/java/generate...
android_webview/java/generated_src/org/chromium/content/app/NativeLibraries.java:21:
static String versionName = "";
On 2013/11/11 20:28:56, joth wrote:
> nit: sVersionName (and libraries should be too)
> 
> oh, or does the TODO cover both of them? 
> 
> (can the be static final? in which case LIBRARIES and VERSION_NAME would be
> better)
I have changed the names to constant names, but I can't actually make
VERSION_NAME (now VERSION_NUMBER, based on a later comment) static final here or
in the NativeLibraries.Template because that makes it a compile time constant.
This breaks with our current build system, which replaces NativeLibraries.class
based on the product being built.

I will also add a TODO to the second comment, since what is to do is slightly
different from the first case.

https://codereview.chromium.org/59533009/diff/1/android_webview/java/src/org/...
File android_webview/java/src/org/chromium/android_webview/AwBrowserProcess.java
(right):

https://codereview.chromium.org/59533009/diff/1/android_webview/java/src/org/...
android_webview/java/src/org/chromium/android_webview/AwBrowserProcess.java:49:
if( !BrowserStartupController.get(context).startBrowserProcessesSync(
On 2013/11/12 02:58:18, Yaron wrote:
> On 2013/11/11 20:28:56, joth wrote:
> > this is a bit tricky now - it can return an error boolean return code, and
can
> > throw exception. Could we move it to just do one or other? (Probably
> exception)
> 
> Ya, I tend to agree with joth

Done.

https://codereview.chromium.org/59533009/diff/1/chrome/android/testshell/java...
File
chrome/android/testshell/java/src/org/chromium/chrome/testshell/signin/AccountsChangedReceiver.java
(left):

https://codereview.chromium.org/59533009/diff/1/chrome/android/testshell/java...
chrome/android/testshell/java/src/org/chromium/chrome/testshell/signin/AccountsChangedReceiver.java:55:
BrowserStartupController.get(context).startBrowserProcessesAsync(callback);
On 2013/11/12 02:58:18, Yaron wrote:
> Make this always throw would help ensure we don't swallow failure cases such
as
> here.

Done.

https://codereview.chromium.org/59533009/diff/1/chrome/app/android/chrome_and...
File chrome/app/android/chrome_android_initializer.cc (right):

https://codereview.chromium.org/59533009/diff/1/chrome/app/android/chrome_and...
chrome/app/android/chrome_android_initializer.cc:24:
content::SetVersionName(vi.Version());
On 2013/11/11 20:28:56, joth wrote:
> Version() returns the version "number" so bit confusing to pass it to set
> version "name".

I thought originally that this was going to match the Android "VersionName", but
in the end this didn't work. I will change this to Version or VersionNumber (or
VERSION_NUMBER etc.) throughout (even if "1.2.3.4" isn't really a number).

> 
> (maybe a comment on content::SetVersionName explaining exactly what the string
> set is for, and how it is used, would make sense)
Isn't this covered by the comment in android_library_loader_hooks.h?

https://codereview.chromium.org/59533009/diff/1/content/app/android/library_l...
File content/app/android/library_loader_hooks.cc (right):

https://codereview.chromium.org/59533009/diff/1/content/app/android/library_l...
content/app/android/library_loader_hooks.cc:39: std::string library_version_name
= "";
On 2013/11/11 20:28:56, joth wrote:
> this is a global C++ object, not allowed by style guide.
> 
> It'd be nice if you could just make this a const char*
> I think that means the caller should just #include chrome_version_info_posix.h
> and pass PRODUCT_VERSION as the param.
> 
> 
> Tangent: for webview we'd really like chrome_version_info_posix.h (and VERSION
> text file) moved into content/ layer so we can use the same version number in
> webview UA in future. Seems like doing that would make this CL much simpler
too?
> (the GetVersinNumber JNI method would just return PRODUCT_VERSION converted to
> java string)

I have changed it to a const char *, but I am setting it though a strdup from
the version number in chrome_version_info.cc; since otherwise we have to put
lifetime constraints on the argument to SetVersionName, which I never like if I
can avoid it (this sort of constraint is too easy to break in future code
changes). This only happens once per Chrome run, so the performance cost is
trivial.

I can see some dangers in moving the version number down to content, at the
moment it is the Chrome  version number, will this always be in step with the
content version number? Also consider, for example, Opera. They might want to
use this to check that their C++ library is in step with their JAR, and will
probably use their own versioning scheme which will have nothing to do with
Chrome versions; yet they are, as far as I know, using content unchanged.

https://codereview.chromium.org/59533009/diff/1/content/app/android/library_l...
content/app/android/library_loader_hooks.cc:150: void SetVersionName(const
std::string & version_name) {
On 2013/11/11 20:28:56, joth wrote:
> not space before &
> 
> (and in header)

Done.

https://codereview.chromium.org/59533009/diff/1/content/shell/android/browser...
File
content/shell/android/browsertests_apk/src/org/chromium/content_browsertests_apk/ContentBrowserTestsActivity.java
(right):

https://codereview.chromium.org/59533009/diff/1/content/shell/android/browser...
content/shell/android/browsertests_apk/src/org/chromium/content_browsertests_apk/ContentBrowserTestsActivity.java:37:
System.exit(-1);
On 2013/11/12 02:58:18, Yaron wrote:
> I think finish() is the cleaner way to shut down the activity. I suppose the
> process won't evert start correctly but I'm a little hesitant to exit as it's
> generally discouraged

The issue with finish is that it only shuts down the activity, not the
application, so, if nothing else kills the application, the next activity will
find that the (wrong) library is already loaded, so will not load it again, even
if we call LoadLibrary again. The only way of unloading the library (as far as I
can find out) is to close down the application completely using exit or through
a crash.

We believe that the original bug is a race condition on installing the new C++
library, so we definitely do want to try loading the library again the next time
anything tries to start a Chrome activity, since by that time the correct
library may be in place.

I am not sure that this is an issue in practice for tests, but it is certainly
an issue for Chrome itself, and I wanted to be consistent.

[Yaron, 2013-11-16 00:10:25]

https://codereview.chromium.org/59533009/diff/110001/build/java_apk.gypi
File build/java_apk.gypi (left):

https://codereview.chromium.org/59533009/diff/110001/build/java_apk.gypi#oldc...
build/java_apk.gypi:64: 'app_manifest_version_code%':
'<(android_app_version_code)',
I think this is breaking some people. remoting/remoting.gyp sets this veariable.
Also, can we remove the default value of it from build/common.gypi?

https://codereview.chromium.org/59533009/diff/110001/build/java_apk.gypi
File build/java_apk.gypi (right):

https://codereview.chromium.org/59533009/diff/110001/build/java_apk.gypi#newc...
build/java_apk.gypi:246: #            '<(native_libraries_template_data_file)',
Remove these

https://codereview.chromium.org/59533009/diff/110001/build/java_apk.gypi#newc...
build/java_apk.gypi:253:
'--array-output=<(native_libraries_template_data_file)',
What is "array-output"? Maybe rename native-libraries-list?

https://codereview.chromium.org/59533009/diff/110001/chrome/android/testshell...
File
chrome/android/testshell/javatests/src/org/chromium/chrome/testshell/ChromiumTestShellTestBase.java
(right):

https://codereview.chromium.org/59533009/diff/110001/chrome/android/testshell...
chrome/android/testshell/javatests/src/org/chromium/chrome/testshell/ChromiumTestShellTestBase.java:45:
// TODO(aberent): Auto-generated catch block
Follow pattern from other places. Log.e and System.exit. Actually maybe you can
just call |fail| with the exception in this case.

https://codereview.chromium.org/59533009/diff/110001/content/app/android/libr...
File content/app/android/library_loader_hooks.cc (right):

https://codereview.chromium.org/59533009/diff/110001/content/app/android/libr...
content/app/android/library_loader_hooks.cc:5: #include <string.h>
Nit: should go after the header include for this file.

https://codereview.chromium.org/59533009/diff/110001/content/app/android/libr...
content/app/android/library_loader_hooks.cc:41: const char *
g_library_version_number = "";
nit: unnecessary space

https://codereview.chromium.org/59533009/diff/110001/content/public/android/j...
File
content/public/android/java/src/org/chromium/content/app/ChildProcessService.java
(right):

https://codereview.chromium.org/59533009/diff/110001/content/public/android/j...
content/public/android/java/src/org/chromium/content/app/ChildProcessService.java:144:
System.exit(-1);
What happens in this case? if somehow the browser process loaded fine but we get
this in renderer, browser just treats this as a renderer crash cause the service
went away?

https://codereview.chromium.org/59533009/diff/110001/content/public/android/j...
File content/public/android/java/src/org/chromium/content/app/LibraryLoader.java
(right):

https://codereview.chromium.org/59533009/diff/110001/content/public/android/j...
content/public/android/java/src/org/chromium/content/app/LibraryLoader.java:144:
if(! NativeLibraries.VERSION_NUMBER.equals(nativeGetVersionNumber()) ) {
So this is a no-op for aw and content shell right? It defaults to "" and they
don't set the value.

https://codereview.chromium.org/59533009/diff/110001/content/public/android/j...
File content/public/android/java/templates/native_libraries_version.h (right):

https://codereview.chromium.org/59533009/diff/110001/content/public/android/j...
content/public/android/java/templates/native_libraries_version.h:6: // is
compiled with no array defined, and then the build system creates a
s/array/native library version/ (and update the rest of the comment)

[aberent, 2013-11-18 15:11:37]

New patch follows.

https://codereview.chromium.org/59533009/diff/110001/build/java_apk.gypi
File build/java_apk.gypi (left):

https://codereview.chromium.org/59533009/diff/110001/build/java_apk.gypi#oldc...
build/java_apk.gypi:64: 'app_manifest_version_code%':
'<(android_app_version_code)',
On 2013/11/16 00:10:26, Yaron wrote:
> I think this is breaking some people. remoting/remoting.gyp sets this
veariable.
> Also, can we remove the default value of it from build/common.gypi?
This change was a mistake (left over from some local testing I was doing).
Thanks for spotting it. Reverted

https://codereview.chromium.org/59533009/diff/110001/build/java_apk.gypi
File build/java_apk.gypi (right):

https://codereview.chromium.org/59533009/diff/110001/build/java_apk.gypi#newc...
build/java_apk.gypi:246: #            '<(native_libraries_template_data_file)',
On 2013/11/16 00:10:26, Yaron wrote:
> Remove these

Done.

https://codereview.chromium.org/59533009/diff/110001/build/java_apk.gypi#newc...
build/java_apk.gypi:253:
'--array-output=<(native_libraries_template_data_file)',
On 2013/11/16 00:10:26, Yaron wrote:
> What is "array-output"? Maybe rename native-libraries-list?

Done.

https://codereview.chromium.org/59533009/diff/110001/chrome/android/testshell...
File
chrome/android/testshell/javatests/src/org/chromium/chrome/testshell/ChromiumTestShellTestBase.java
(right):

https://codereview.chromium.org/59533009/diff/110001/chrome/android/testshell...
chrome/android/testshell/javatests/src/org/chromium/chrome/testshell/ChromiumTestShellTestBase.java:45:
// TODO(aberent): Auto-generated catch block
On 2013/11/16 00:10:26, Yaron wrote:
> Follow pattern from other places. Log.e and System.exit. Actually maybe you
can
> just call |fail| with the exception in this case.

Done.

https://codereview.chromium.org/59533009/diff/110001/content/app/android/libr...
File content/app/android/library_loader_hooks.cc (right):

https://codereview.chromium.org/59533009/diff/110001/content/app/android/libr...
content/app/android/library_loader_hooks.cc:41: const char *
g_library_version_number = "";
On 2013/11/16 00:10:26, Yaron wrote:
> nit: unnecessary space

Done.

https://codereview.chromium.org/59533009/diff/110001/content/public/android/j...
File
content/public/android/java/src/org/chromium/content/app/ChildProcessService.java
(right):

https://codereview.chromium.org/59533009/diff/110001/content/public/android/j...
content/public/android/java/src/org/chromium/content/app/ChildProcessService.java:144:
System.exit(-1);
On 2013/11/16 00:10:26, Yaron wrote:
> What happens in this case? if somehow the browser process loaded fine but we
get
> this in renderer, browser just treats this as a renderer crash cause the
service
> went away?

Yes.

https://codereview.chromium.org/59533009/diff/110001/content/public/android/j...
File content/public/android/java/src/org/chromium/content/app/LibraryLoader.java
(right):

https://codereview.chromium.org/59533009/diff/110001/content/public/android/j...
content/public/android/java/src/org/chromium/content/app/LibraryLoader.java:144:
if(! NativeLibraries.VERSION_NUMBER.equals(nativeGetVersionNumber()) ) {
On 2013/11/16 00:10:26, Yaron wrote:
> So this is a no-op for aw and content shell right? It defaults to "" and they
> don't set the value.

Yes. We may want to add the check to at least Android Webview in future, but for
now it a no-op.

https://codereview.chromium.org/59533009/diff/110001/content/public/android/j...
File content/public/android/java/templates/native_libraries_version.h (right):

https://codereview.chromium.org/59533009/diff/110001/content/public/android/j...
content/public/android/java/templates/native_libraries_version.h:6: // is
compiled with no array defined, and then the build system creates a
On 2013/11/16 00:10:26, Yaron wrote:
> s/array/native library version/ (and update the rest of the comment)

Done.

[aberent, 2013-11-18 18:26:52]

https://codereview.chromium.org/59533009/diff/110001/build/java_apk.gypi
File build/java_apk.gypi (right):

https://codereview.chromium.org/59533009/diff/110001/build/java_apk.gypi#newc...
build/java_apk.gypi:246: #            '<(native_libraries_template_data_file)',
On 2013/11/18 15:11:37, aberent wrote:
> On 2013/11/16 00:10:26, Yaron wrote:
> > Remove these
> 
> Done.

Actually I have put these back in properly (not as comments), made the creation
of NativeLibraries.java dependent on these, and taken out the now unnecessary
stamp file. Without these as outputs they were sometimes failing to rebuild
(possibly only when I had done something strange to my local output directory),
and this seems more correct since the function of this step is to produce these
files.

[Yaron, 2013-11-18 19:07:54]

lgtm

https://codereview.chromium.org/59533009/diff/110001/content/app/android/libr...
File content/app/android/library_loader_hooks.cc (right):

https://codereview.chromium.org/59533009/diff/110001/content/app/android/libr...
content/app/android/library_loader_hooks.cc:5: #include <string.h>
On 2013/11/16 00:10:26, Yaron wrote:
> Nit: should go after the header include for this file.

bump

https://codereview.chromium.org/59533009/diff/250001/build/android/gyp/create...
File build/android/gyp/create_native_libraries_header.py (right):

https://codereview.chromium.org/59533009/diff/250001/build/android/gyp/create...
build/android/gyp/create_native_libraries_header.py:9: This header should
contain the list of native libraries to load in the form:
Nit: could use minor update.

[joth, 2013-11-18 19:45:22]

aw/ lgtm
I skimmed over all of content/ as well, but let me know if you want me to do a
detailed review on that.

https://codereview.chromium.org/59533009/diff/250001/content/app/android/libr...
File content/app/android/library_loader_hooks.cc (right):

https://codereview.chromium.org/59533009/diff/250001/content/app/android/libr...
content/app/android/library_loader_hooks.cc:5: #include <string.h>
nit - remove

https://codereview.chromium.org/59533009/diff/250001/content/public/android/j...
File content/public/android/java/src/org/chromium/content/app/LibraryLoader.java
(right):

https://codereview.chromium.org/59533009/diff/250001/content/public/android/j...
content/public/android/java/src/org/chromium/content/app/LibraryLoader.java:144:
if(! NativeLibraries.VERSION_NUMBER.equals(nativeGetVersionNumber()) ) {
nit: space after if, not before function name

https://codereview.chromium.org/59533009/diff/250001/content/public/android/j...
File content/public/android/java/templates/native_libraries_array.h (right):

https://codereview.chromium.org/59533009/diff/250001/content/public/android/j...
content/public/android/java/templates/native_libraries_array.h:9: ={}
wowzers, this is the strangest pair of header files I did see.

but, i guess, it's OK. The comment could explain the context this is used in?
(variable name prefix, semi-colon postfix)

Hmmmm this is only ever #included from java. Does it need to be a .h file at
all?

https://codereview.chromium.org/59533009/diff/250001/content/public/app/andro...
File content/public/app/android_library_loader_hooks.h (right):

https://codereview.chromium.org/59533009/diff/250001/content/public/app/andro...
content/public/app/android_library_loader_hooks.h:10: #include <string>
nit - remove

[aberent, 2013-11-25 18:31:41]

https://codereview.chromium.org/59533009/diff/250001/build/android/gyp/create...
File build/android/gyp/create_native_libraries_header.py (right):

https://codereview.chromium.org/59533009/diff/250001/build/android/gyp/create...
build/android/gyp/create_native_libraries_header.py:9: This header should
contain the list of native libraries to load in the form:
On 2013/11/18 19:08:00, Yaron wrote:
> Nit: could use minor update.

Done.

https://codereview.chromium.org/59533009/diff/250001/content/app/android/libr...
File content/app/android/library_loader_hooks.cc (right):

https://codereview.chromium.org/59533009/diff/250001/content/app/android/libr...
content/app/android/library_loader_hooks.cc:5: #include <string.h>
On 2013/11/18 19:45:23, joth wrote:
> nit - remove

Done.

https://codereview.chromium.org/59533009/diff/250001/content/public/android/j...
File content/public/android/java/src/org/chromium/content/app/LibraryLoader.java
(right):

https://codereview.chromium.org/59533009/diff/250001/content/public/android/j...
content/public/android/java/src/org/chromium/content/app/LibraryLoader.java:144:
if(! NativeLibraries.VERSION_NUMBER.equals(nativeGetVersionNumber()) ) {
On 2013/11/18 19:45:23, joth wrote:
> nit: space after if, not before function name

Done.

https://codereview.chromium.org/59533009/diff/250001/content/public/android/j...
File content/public/android/java/templates/native_libraries_array.h (right):

https://codereview.chromium.org/59533009/diff/250001/content/public/android/j...
content/public/android/java/templates/native_libraries_array.h:9: ={}
On 2013/11/18 19:45:23, joth wrote:
> wowzers, this is the strangest pair of header files I did see.
> 
> but, i guess, it's OK. The comment could explain the context this is used in?
> (variable name prefix, semi-colon postfix)
I have added to the comment. The change goes with making LIBRARIES final in
NativeLibraries.template, which means that it has to be
given a value when it is declared.
> 
> Hmmmm this is only ever #included from java. Does it need to be a .h file at
> all?
No, but it seems to be the convention we are following for all files included by
the C preprocessor, whatever the final language of the resulting code.

https://codereview.chromium.org/59533009/diff/250001/content/public/app/andro...
File content/public/app/android_library_loader_hooks.h (right):

https://codereview.chromium.org/59533009/diff/250001/content/public/app/andro...
content/public/app/android_library_loader_hooks.h:10: #include <string>
On 2013/11/18 19:45:23, joth wrote:
> nit - remove

Done.

[Yaron, 2013-11-25 23:22:07]

still lgtm

[commit-bot: I haz the power, 2013-11-27 12:16:48]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/aberent@chromium.org/59533009/450001

[commit-bot: I haz the power, 2013-11-27 13:13:37]

Retried try job too often on android_dbg for step(s) slave_steps
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=android_db...

[commit-bot: I haz the power, 2013-11-27 13:56:24]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/aberent@chromium.org/59533009/470001

[commit-bot: I haz the power, 2013-11-27 17:02:32]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/aberent@chromium.org/59533009/490001

[commit-bot: I haz the power, 2013-11-27 18:16:59]

Retried try job too often on android_aosp for step(s) compile
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=android_ao...

[commit-bot: I haz the power, 2013-11-28 09:20:45]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/aberent@chromium.org/59533009/510001

[commit-bot: I haz the power, 2013-11-28 12:22:47]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/aberent@chromium.org/59533009/530001

[commit-bot: I haz the power, 2013-11-28 14:28:54]

Change committed as 237779