Add Android NDK build for WebRTC

Add Android NDK build for WebRTC

BUG=none
TEST=Running builds on local master and slave.

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=195132

[kjellander_chromium, 2013-02-07 14:43:16]

I think this is starting to look like something that could be deployed. I have
now created an annotation script for building the app in WebRTC, which I'm using
here.

I figure the preferred approach would be to make the demo app to compile as a
part of the normal steps on bb_bot_run.py, but I'm not sure how to do that
properly so I hope this can do for now.
It will need to be resolved in order to use the try bot configurations since
when I tried, the findbugs step failed since it cannot find any class files (it
would be resolved if we can compile the demo app before, since it produces a
collection of class files).

[kjellander_chromium, 2013-03-15 14:27:00]

Isaac, I'd like to bring this forward. Could you take a look now?

I've been trying to create a GYP target so the custom build script is not needed
for the app, but without luck. I'm trying to convince the developers to get that
in place, looking at how other NDK apps are built.

On 2013/02/07 14:43:16, Henrik Kjellander wrote:
> I think this is starting to look like something that could be deployed. I have
> now created an annotation script for building the app in WebRTC, which I'm
using
> here.
> 
> I figure the preferred approach would be to make the demo app to compile as a
> part of the normal steps on bb_bot_run.py, but I'm not sure how to do that
> properly so I hope this can do for now.
> It will need to be resolved in order to use the try bot configurations since
> when I tried, the findbugs step failed since it cannot find any class files
(it
> would be resolved if we can compile the demo app before, since it produces a
> collection of class files).

[Isaac (away), 2013-03-18 17:54:21]

+Mike Stipicevic

Code looks good; one design question inline.

https://codereview.chromium.org/11896066/diff/5001/scripts/master/factory/web...
File scripts/master/factory/webrtc_factory.py (right):

https://codereview.chromium.org/11896066/diff/5001/scripts/master/factory/web...
scripts/master/factory/webrtc_factory.py:91: def WebRTCAndroidNDKFactory(self,
annotation_script,
I would rather us use ChromiumAnnotationFactory if possible.  What things does
this have that the standard annotation factory lacks?  Is it something we could
add?

[Mike Stip (use stip instead), 2013-03-18 18:25:43]

Henrik,

You may be interested in AnnotatorFactory in
scripts/master/factory/annotator_factory.py. All it does it invoke a single
annotation script with the intent on forcing all configuration and execution on
the slave.

Right now there is code checkout but no real compile step (that's being worked
on).

[Mike Stip (use stip instead), 2013-03-21 00:26:35]

Henrik,

It looks like this does the normal actions of a ChromiumAnnotationFactory, but
adds the ndk_apps as a final step. Is there a way to work this into
bb_run_bot.py?

[kjellander_chromium, 2013-03-21 15:44:13]

On 2013/03/21 00:26:35, Mike Stipicevic wrote:
> Henrik,
> 
> It looks like this does the normal actions of a ChromiumAnnotationFactory, but
> adds the ndk_apps as a final step. Is there a way to work this into
> bb_run_bot.py?

I think that would be possible, but since this is not very generic and that it
would be useless work when
https://code.google.com/p/webrtc/issues/detail?id=1029 is resolved, I'm not sure
it's worth getting it into bb_run_bot.py.

[kjellander_chromium, 2013-03-21 15:46:17]

I don't think I can use ChromiumAnnotationFactory right now, since until
https://code.google.com/p/webrtc/issues/detail?id=1029 is resolved, I'm going to
need the Python script to build our demo app.

https://codereview.chromium.org/11896066/diff/5001/masters/master.client.webr...
File masters/master.client.webrtc/master_linux_cfg.py (right):

https://codereview.chromium.org/11896066/diff/5001/masters/master.client.webr...
masters/master.client.webrtc/master_linux_cfg.py:127: 'android_bot_id':
'webrtc-builder',
It's not clear to me how I'm going to use the android_bot_id properly, since
when I tried to add a try-bot with the build config 'try-builder-dbg' in
src/build/android/buildbot/bb_run_bot.py, I got a failing findbugs step since it
cannot find any Java files.
I guess when https://code.google.com/p/webrtc/issues/detail?id=1029 is resolved,
this won't be a problem since Java files will be compiled during the 'build'
step.

https://codereview.chromium.org/11896066/diff/5001/scripts/master/factory/web...
File scripts/master/factory/webrtc_factory.py (right):

https://codereview.chromium.org/11896066/diff/5001/scripts/master/factory/web...
scripts/master/factory/webrtc_factory.py:91: def WebRTCAndroidNDKFactory(self,
annotation_script,
On 2013/03/18 17:54:21, Isaac wrote:
> I would rather us use ChromiumAnnotationFactory if possible.  What things does
> this have that the standard annotation factory lacks?  Is it something we
could
> add?

Due to the fact that the WebRTC NDK demo application does not compile as a part
of a normal GYP target, it will not be compiled by bb_run_bot.py. I don't see an
easy way to add that.
That's why I created this factory that adds an additional step to handle that.
As soon https://code.google.com/p/webrtc/issues/detail?id=1029 is resolved, I
can remove this and use a plain ChromiumAnnotationFactory.

[Mike Stip (use stip instead), 2013-03-22 20:23:36]

OK, I'm good with this as long as we have a cleanup bug filed (blocking on the
other issue).

Isaac, wdyt?

[Isaac (away), 2013-03-22 20:26:14]

I'm not sure I follow.  bb_run_bot can run an arbitrary sequence of commands. 
Is the issue that you don't want to duplicate the gyp logic in src and in build?

[kjellander_chromium, 2013-03-28 09:58:16]

I'll be happy to file a clean up bug after committing, or do you want it to be
created before that?

On 2013/03/22 20:23:36, Mike Stipicevic wrote:
> OK, I'm good with this as long as we have a cleanup bug filed (blocking on the
> other issue).
> 
> Isaac, wdyt?

[kjellander_chromium, 2013-03-28 10:03:30]

On 2013/03/22 20:26:14, Isaac wrote:
> I'm not sure I follow.  bb_run_bot can run an arbitrary sequence of commands. 
> Is the issue that you don't want to duplicate the gyp logic in src and in
build?

As I understand it, the steps in bb_run_bot.py are only referencing the
functions in buildbot_functions.sh, where bb_compile simply compiles the All
target. As the application we're building is not yet a part of that compilation
(pending webrtc:1029) I have to run a separate script to build it.

I assume you're not suggesting I should create a new function in
buildbot_functions.sh and a way to invoke it from bb_bot_run.py? 

If there's another way to have bb_run_bot to run an arbitrary sequence of
commands, please let me know an example (I couldn't find any).

[Isaac (away), 2013-04-01 05:52:42]

Yes, you would add a function in buildbot_functions similar to
bb_compile_experimental.  In long run this will get cleaned up, but it is far
easier to cleanup than vs a non-annotated script.  And this is more
maintainable.

Let me know if there are other issues with that that I haven't anticipated -- I
don't know your full use case so it's hard for me to gauge if that's the only
hiccup.

[kjellander_chromium, 2013-04-12 13:44:31]

Finally I'm able to get this sorted out in a nice way using
ChromiumAnnotationFactory instead.

With http://review.webrtc.org/1286004/ landed we can build everything as a part
of the "All" target, eliminating the need to invoke the custom script for the
Android Video demo app.

This has been tested locally and for the builds to pass, we need to first roll
our chromium_revision in WebRTC DEPS to something more recent (covered in
http://review.webrtc.org/1317004/) since the Zip Build step fails otherwise.

https://codereview.chromium.org/11896066/diff/27001/scripts/master/factory/we...
File scripts/master/factory/webrtc_factory.py (right):

https://codereview.chromium.org/11896066/diff/27001/scripts/master/factory/we...
scripts/master/factory/webrtc_factory.py:39:
chromium_factory.ChromiumFactory.__init__(
It seems safe to call this first and then replace the self._solutions list. Even
if this is not optimal, the only other solution would be to completely duplicate
the code in the chromium_factory.ChromiumAnnotationFactory method, which is
worse IMO.

[Isaac (away), 2013-04-19 08:52:41]

lgtm, thanks!

https://codereview.chromium.org/11896066/diff/27001/masters/master.tryserver....
File masters/master.tryserver.webrtc/master.cfg (right):

https://codereview.chromium.org/11896066/diff/27001/masters/master.tryserver....
masters/master.tryserver.webrtc/master.cfg:220: 'android_bot_id': 'builder',
please use 'webrtc-builder-dbg'.  The goal of this field is for every slave to
have a different ID.

https://codereview.chromium.org/11896066/diff/27001/masters/master.tryserver....
masters/master.tryserver.webrtc/master.cfg:221: 'buildtool': 'ninja',
this field is not needed anymore, you can delete.

[kjellander_chromium, 2013-04-19 09:58:53]

Thanks for review. 
Isaac: Please have a look at my comment about android_bot_id naming below.

https://codereview.chromium.org/11896066/diff/27001/masters/master.tryserver....
File masters/master.tryserver.webrtc/master.cfg (right):

https://codereview.chromium.org/11896066/diff/27001/masters/master.tryserver....
masters/master.tryserver.webrtc/master.cfg:220: 'android_bot_id': 'builder',
OK I used 'webrtc-builder-dbg' for the buildbot and 'webrtc-trybot-builder-dbg'
for the trybot.

When I tried 'webrtc-try-builder-dbg' it picked up the 'try-builder-dbg'
configuration, which fails on the 'Check licenses for WebView' step for us. I
figured just changing the string would make it default back to the 'builder'
step instead. Doesn't feel very safe for future changes of bb_run_bot.py though,
what do you think?


On 2013/04/19 08:52:41, Isaac wrote:
> please use 'webrtc-builder-dbg'.  The goal of this field is for every slave to
> have a different ID.

https://codereview.chromium.org/11896066/diff/27001/masters/master.tryserver....
masters/master.tryserver.webrtc/master.cfg:221: 'buildtool': 'ninja',
On 2013/04/19 08:52:41, Isaac wrote:
> this field is not needed anymore, you can delete.

Done.

[commit-bot: I haz the power, 2013-04-19 11:13:49]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/kjellander@chromium.org/11896066/36001

[commit-bot: I haz the power, 2013-04-19 11:14:48]

Change committed as 195132