Chromium Code Reviews
|
|
Chromium Code Reviews
Issue
226033002:
Ensure seccomp-bpf cannot be silently disabled for non-SFI NaCl (Closed)
|
Created:
6 years, 8 months ago by hamaji Modified:
6 years, 8 months ago CC:
chromium-reviews Base URL:
svn://svn.chromium.org/chrome/trunk/src Visibility:
Public. |
DescriptionEnsure seccomp-bpf cannot be silently disabled for non-SFI NaCl
Also introduce --nacl-dangerous-no-sandbox-nonsfi, which
allows us to skip sandbox entirely for development.
TEST=out/Release/browser_tests --gtest_filter='NaCl*'
TEST=trybots
BUG=359230
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=263824
Patch Set 1 #
Total comments: 7
Patch Set 2 : #
Total comments: 8
Patch Set 3 : #
Total comments: 4
Patch Set 4 : #Patch Set 5 : #
Total comments: 3
Messages
Total messages: 29 (0 generated)
https://codereview.chromium.org/226033002/diff/1/components/nacl/loader/nacl_... File components/nacl/loader/nacl_helper_linux.cc (right): https://codereview.chromium.org/226033002/diff/1/components/nacl/loader/nacl_... components/nacl/loader/nacl_helper_linux.cc:58: if (getenv("NACL_DANGEROUS_DISABLE_NONSFI_SANDBOX")) { We need to whitelist this (and other NACL-prefixed variables such as NACLVERBOSITY?) when we fix http://crbug.com/358413. This is extremely useful when we debug our app.
FYI, you should manually run the "linux_rel_precise32" trybot for all Non-SFI-Mode-related changes, now that Hidehiko has enabled tests that only run on 32-bit Linux. https://codereview.chromium.org/226033002/diff/1/components/nacl/loader/nacl_... File components/nacl/loader/nacl_helper_linux.cc (right): https://codereview.chromium.org/226033002/diff/1/components/nacl/loader/nacl_... components/nacl/loader/nacl_helper_linux.cc:58: if (getenv("NACL_DANGEROUS_DISABLE_NONSFI_SANDBOX")) { On 2014/04/04 14:36:58, hamaji wrote: > We need to whitelist this (and other NACL-prefixed variables such as > NACLVERBOSITY?) when we fix http://crbug.com/358413. > > This is extremely useful when we debug our app. Is there anything specific you want it for? I'll let Julien comment on whether this is OK. :-) Having a command line option would be preferred over an env var. (The reason native_client has tended to use env vars is the difficulty of plumbing command line options over from the Chromium side.) I'd prefer leaving this out for now, but we can still discuss the use cases for it. https://codereview.chromium.org/226033002/diff/1/components/nacl/loader/nacl_... components/nacl/loader/nacl_helper_linux.cc:65: // TODO(hamaji): Add a strict seccomp sandbox for non-SFI NaCl. Nit: I suspect you wouldn't be changing this part of the code when addressing the TODO, so a TODO doesn't belong here, because it would be easy to forget to remove it. There's an issue tracking this, which is enough. https://codereview.chromium.org/226033002/diff/1/components/nacl/loader/nacl_... components/nacl/loader/nacl_helper_linux.cc:67: bool bpf_sandbox_initialized = InitializeBPFSandbox(); When I talked with Julien yesterday, he pointed out that "--disable-seccomp-filter-sandbox" would stop this enabling the sandbox, which we don't want to allow to happen. That option is currently safe(ish), but wouldn't be safe for Non-SFI NaCl.
https://codereview.chromium.org/226033002/diff/1/components/nacl/loader/nacl_... File components/nacl/loader/nacl_helper_linux.cc (right): https://codereview.chromium.org/226033002/diff/1/components/nacl/loader/nacl_... components/nacl/loader/nacl_helper_linux.cc:58: if (getenv("NACL_DANGEROUS_DISABLE_NONSFI_SANDBOX")) { On 2014/04/04 15:45:37, Mark Seaborn wrote: > On 2014/04/04 14:36:58, hamaji wrote: > > We need to whitelist this (and other NACL-prefixed variables such as > > NACLVERBOSITY?) when we fix http://crbug.com/358413. > > > > This is extremely useful when we debug our app. > > Is there anything specific you want it for? Yes. Our app has a hack to attach GDB appropriately. The hack is like: 1. If /tmp/bare-metal-gdb.lock exists, our app waits until this file is removed at the beginning. 2. Our launch script attaches GDB to nacl_helper and set up Python extension. 3. The python extension tells GDB the load address of the loader by Python extension. It also sets a breakpoint to the loader so newly loaded objects will be also handled by the Python extension. 4. Our launch script removes /tmp/bare-metal-gdb.lock. The program starts. This is an ugly hack, but I don't think we will think about how non-SFI NaCl supports GDB in M36. So, to keep this hack in our side works, this is mandatory for us. I think we also want to use this for: https://code.google.com/p/chromium/issues/detail?id=341378 Though this is not a high priority right now, we want to abandon our trusted plugin mode. Of course, changing this to a flag is OK for us. Let me see. > I'll let Julien comment on whether this is OK. :-) I hope he is fine with this :) I think we can easily make SFI NaCl unsafe if you can add env variables, so I'm guessing this is acceptable. https://codereview.chromium.org/226033002/diff/1/components/nacl/loader/nacl_... components/nacl/loader/nacl_helper_linux.cc:65: // TODO(hamaji): Add a strict seccomp sandbox for non-SFI NaCl. On 2014/04/04 15:45:37, Mark Seaborn wrote: > Nit: I suspect you wouldn't be changing this part of the code when addressing > the TODO, so a TODO doesn't belong here, because it would be easy to forget to > remove it. There's an issue tracking this, which is enough. I was planning to change this to nacl::nonsfi::InitializeSandbox() and stop checking the return value. I guess InitializeSandbox() should just crash when it fails, instead of returning false. But yeah, I agree this TODO does not make much sense. I'll remove this. https://codereview.chromium.org/226033002/diff/1/components/nacl/loader/nacl_... components/nacl/loader/nacl_helper_linux.cc:67: bool bpf_sandbox_initialized = InitializeBPFSandbox(); On 2014/04/04 15:45:37, Mark Seaborn wrote: > When I talked with Julien yesterday, he pointed out that > "--disable-seccomp-filter-sandbox" would stop this enabling the sandbox, which > we don't want to allow to happen. That option is currently safe(ish), but > wouldn't be safe for Non-SFI NaCl. It seems the sandbox is enabled even with --disable-seccomp-filter-sandbox. I didn't check this well, but nacl_helper does not have full args when we set up seccomp sandbox in nacl_helper_linux.cc?
On 4 April 2014 11:57, <hamaji@chromium.org> wrote: > > https://codereview.chromium.org/226033002/diff/1/ > components/nacl/loader/nacl_helper_linux.cc > File components/nacl/loader/nacl_helper_linux.cc (right): > > https://codereview.chromium.org/226033002/diff/1/ > components/nacl/loader/nacl_helper_linux.cc#newcode58 > components/nacl/loader/nacl_helper_linux.cc:58: if > (getenv("NACL_DANGEROUS_DISABLE_NONSFI_SANDBOX")) { > On 2014/04/04 15:45:37, Mark Seaborn wrote: > >> On 2014/04/04 14:36:58, hamaji wrote: >> > We need to whitelist this (and other NACL-prefixed variables such as >> > NACLVERBOSITY?) when we fix http://crbug.com/358413. >> > >> > This is extremely useful when we debug our app. >> > > Is there anything specific you want it for? >> > > Yes. Our app has a hack to attach GDB appropriately. The hack is like: > > 1. If /tmp/bare-metal-gdb.lock exists, our app waits until this file is > removed at the beginning. > 2. Our launch script attaches GDB to nacl_helper and set up Python > extension. > 3. The python extension tells GDB the load address of the loader by > Python extension. It also sets a breakpoint to the loader so newly > loaded objects will be also handled by the Python extension. > 4. Our launch script removes /tmp/bare-metal-gdb.lock. The program > starts. > Does this hack involve changing any Chromium code? It sounds like it doesn't. I'm not sure how you would find the load address of the user nexe in step (3) without changing Chromium. Do you look at /proc/FD/maps for that? Rather than waiting on a file in /tmp, you could have the nexe wait on a global variable, and use GDB to set the variable's value to 1 to make the program start. > > https://codereview.chromium.org/226033002/diff/1/ > components/nacl/loader/nacl_helper_linux.cc#newcode67 > components/nacl/loader/nacl_helper_linux.cc:67: bool > bpf_sandbox_initialized = InitializeBPFSandbox(); > On 2014/04/04 15:45:37, Mark Seaborn wrote: > >> When I talked with Julien yesterday, he pointed out that >> "--disable-seccomp-filter-sandbox" would stop this enabling the sandbox, >> which > > we don't want to allow to happen. That option is currently safe(ish), but > > wouldn't be safe for Non-SFI NaCl. >> > > It seems the sandbox is enabled even with > --disable-seccomp-filter-sandbox. I didn't check this well, but > nacl_helper does not have full args when we set up seccomp sandbox in > nacl_helper_linux.cc? Julien and I talked about this yesterday. nacl_helper doesn't inherit all arguments by default, but it does inherit "--disable-seccomp-filter-sandbox". This is done in components/nacl/zygote/nacl_fork_delegate_linux.cc -- see kForwardSwitches. Cheers, Mark To unsubscribe from this group and stop receiving emails from it, send an email to chromium-reviews+unsubscribe@chromium.org.
On 2014/04/04 19:49:52, Mark Seaborn wrote: > On 4 April 2014 11:57, <mailto:hamaji@chromium.org> wrote: > > > > > https://codereview.chromium.org/226033002/diff/1/ > > components/nacl/loader/nacl_helper_linux.cc > > File components/nacl/loader/nacl_helper_linux.cc (right): > > > > https://codereview.chromium.org/226033002/diff/1/ > > components/nacl/loader/nacl_helper_linux.cc#newcode58 > > components/nacl/loader/nacl_helper_linux.cc:58: if > > (getenv("NACL_DANGEROUS_DISABLE_NONSFI_SANDBOX")) { > > On 2014/04/04 15:45:37, Mark Seaborn wrote: > > > >> On 2014/04/04 14:36:58, hamaji wrote: > >> > We need to whitelist this (and other NACL-prefixed variables such as > >> > NACLVERBOSITY?) when we fix http://crbug.com/358413. > >> > > >> > This is extremely useful when we debug our app. > >> > > > > Is there anything specific you want it for? > >> > > > > Yes. Our app has a hack to attach GDB appropriately. The hack is like: > > > > 1. If /tmp/bare-metal-gdb.lock exists, our app waits until this file is > > removed at the beginning. > > 2. Our launch script attaches GDB to nacl_helper and set up Python > > extension. > > 3. The python extension tells GDB the load address of the loader by > > Python extension. It also sets a breakpoint to the loader so newly > > loaded objects will be also handled by the Python extension. > > 4. Our launch script removes /tmp/bare-metal-gdb.lock. The program > > starts. > > > > Does this hack involve changing any Chromium code? It sounds like it > doesn't. I'm not sure how you would find the load address of the user nexe > in step (3) without changing Chromium. Do you look at /proc/FD/maps for > that? We don't change any Chromium code. We are using "info proc mapping" of GDB as reading /proc/<PID>/maps in remote machine (ARM device) is a bit trickier. > > Rather than waiting on a file in /tmp, you could have the nexe wait on a > global variable, and use GDB to set the variable's value to 1 to make the > program start. The default value of such global should be 1. Otherwise, we always need to attach GDB to run our app. So, this means we re-compile the program when we attach GDB? Other downsides: 1. We need to know the address of the global variable, so we cannot attach GDB to stripped nexe. 2. We need to set the first breakpoint in nacl_helper before it transfers the control to runnable-ld.so. This means we need the symbol table of nacl_helper as well. 3. The breakpoint in nacl_helper can be changed in future. This will not be a problem once non-SFI mode has become stable, though. We are using pinned Chrome for trusted plugin and SFI NaCl, so we can modify the script when we roll Chrome. However, we are currently working on Chrome HEAD. For now, my updated patch uses a command line flag instead of a environment variable. > > > > > > https://codereview.chromium.org/226033002/diff/1/ > > components/nacl/loader/nacl_helper_linux.cc#newcode67 > > components/nacl/loader/nacl_helper_linux.cc:67: bool > > bpf_sandbox_initialized = InitializeBPFSandbox(); > > On 2014/04/04 15:45:37, Mark Seaborn wrote: > > > >> When I talked with Julien yesterday, he pointed out that > >> "--disable-seccomp-filter-sandbox" would stop this enabling the sandbox, > >> which > > > > we don't want to allow to happen. That option is currently safe(ish), but > > > > wouldn't be safe for Non-SFI NaCl. > >> > > > > It seems the sandbox is enabled even with > > --disable-seccomp-filter-sandbox. I didn't check this well, but > > nacl_helper does not have full args when we set up seccomp sandbox in > > nacl_helper_linux.cc? > > > Julien and I talked about this yesterday. nacl_helper doesn't inherit all > arguments by default, but it does inherit > "--disable-seccomp-filter-sandbox". This is done > in components/nacl/zygote/nacl_fork_delegate_linux.cc -- > see kForwardSwitches. Sorry, I was testing with a wrong branch so I didn't actually add --disable-seccomp-filter-sandbox. Yes, we need to disregard this flag. I guessed we want to ignore all flags except a few whitelisted ones (e.g., --nacl-dangerous-no-sandbox-nonsfi, I've just added) for non-SFI, so now we reset all command line flags. > > Cheers, > Mark > > To unsubscribe from this group and stop receiving emails from it, send an email > to mailto:chromium-reviews+unsubscribe@chromium.org.
> > Julien and I talked about this yesterday. nacl_helper doesn't inherit all > > arguments by default, but it does inherit > > "--disable-seccomp-filter-sandbox". This is done > > in components/nacl/zygote/nacl_fork_delegate_linux.cc -- > > see kForwardSwitches. > > Sorry, I was testing with a wrong branch so I didn't actually add > --disable-seccomp-filter-sandbox. Yes, we need to disregard this flag. I guessed > we want to ignore all flags except a few whitelisted ones (e.g., > --nacl-dangerous-no-sandbox-nonsfi, I've just added) for non-SFI, so now we > reset all command line flags. (Note: I didn't read the rest of the thread carefully). If possible, I would like to keep the logic simple. --disable-seccomp-filter-sandbox should continue to do what it's supposed to do. However, in BMM, we should CHECK() that the seccomp-bpf sandbox is engage before pursuing. It means that someone trying to use BMM with --disable-seccomp-filter-sandbox will get a crash, and I think that's ok. Also I updated https://code.google.com/p/chromium/issues/detail?id=359230 about some of the other stuff we need to do. I can take care of the threading issue.
https://codereview.chromium.org/226033002/diff/20001/components/nacl/loader/n... File components/nacl/loader/nacl_helper_linux.cc (right): https://codereview.chromium.org/226033002/diff/20001/components/nacl/loader/n... components/nacl/loader/nacl_helper_linux.cc:110: no_sandbox = cmd_line->HasSwitch(switches::kNaClDangerousNoSandboxNonSfi); Does this really work? How does this get forwarded to nacl_loader? Typically you'd need to whitelist it in the Zygote. https://codereview.chromium.org/226033002/diff/20001/components/nacl/loader/n... components/nacl/loader/nacl_helper_linux.cc:122: CHECK(cmd_line->Init(0, NULL)); Instead of doing this, simply relax the CHECK() line 65 for this specific condition.
https://codereview.chromium.org/226033002/diff/20001/components/nacl/common/n... File components/nacl/common/nacl_switches.cc (right): https://codereview.chromium.org/226033002/diff/20001/components/nacl/common/n... components/nacl/common/nacl_switches.cc:52: const char kNaClDangerousNoSandboxNonSfi[] = Nit: please sort these https://codereview.chromium.org/226033002/diff/20001/components/nacl/common/n... File components/nacl/common/nacl_switches.h (right): https://codereview.chromium.org/226033002/diff/20001/components/nacl/common/n... components/nacl/common/nacl_switches.h:24: extern const char kNaClDangerousNoSandboxNonSfi[]; Nit: sort these as per comment in the code above
https://codereview.chromium.org/226033002/diff/20001/components/nacl/common/n... File components/nacl/common/nacl_switches.cc (right): https://codereview.chromium.org/226033002/diff/20001/components/nacl/common/n... components/nacl/common/nacl_switches.cc:52: const char kNaClDangerousNoSandboxNonSfi[] = On 2014/04/07 23:39:48, Mark Seaborn wrote: > Nit: please sort these Done. https://codereview.chromium.org/226033002/diff/20001/components/nacl/common/n... File components/nacl/common/nacl_switches.h (right): https://codereview.chromium.org/226033002/diff/20001/components/nacl/common/n... components/nacl/common/nacl_switches.h:24: extern const char kNaClDangerousNoSandboxNonSfi[]; On 2014/04/07 23:39:48, Mark Seaborn wrote: > Nit: sort these as per comment in the code above Done. https://codereview.chromium.org/226033002/diff/20001/components/nacl/loader/n... File components/nacl/loader/nacl_helper_linux.cc (right): https://codereview.chromium.org/226033002/diff/20001/components/nacl/loader/n... components/nacl/loader/nacl_helper_linux.cc:110: no_sandbox = cmd_line->HasSwitch(switches::kNaClDangerousNoSandboxNonSfi); On 2014/04/07 23:38:10, jln wrote: > Does this really work? How does this get forwarded to nacl_loader? Typically > you'd need to whitelist it in the Zygote. No, I forgot to merge two commits. Now the flag should be transferred appropriately (browser => zygote => nacl_helper). https://codereview.chromium.org/226033002/diff/20001/components/nacl/loader/n... components/nacl/loader/nacl_helper_linux.cc:122: CHECK(cmd_line->Init(0, NULL)); On 2014/04/07 23:38:10, jln wrote: > Instead of doing this, simply relax the CHECK() line 65 for this specific > condition. Done.
https://chromiumcodereview.appspot.com/226033002/diff/30001/components/nacl/c... File components/nacl/common/nacl_switches.h (right): https://chromiumcodereview.appspot.com/226033002/diff/30001/components/nacl/c... components/nacl/common/nacl_switches.h:19: extern const char kNaClDangerousNoSandboxNonSfi[]; Please, add this to chrome/browser/ui/startup/bad_flags_prompt.cc as well. https://chromiumcodereview.appspot.com/226033002/diff/30001/components/nacl/l... File components/nacl/loader/nacl_helper_linux.cc (right): https://chromiumcodereview.appspot.com/226033002/diff/30001/components/nacl/l... components/nacl/loader/nacl_helper_linux.cc:107: bool no_sandbox = false; This makes for a logic that is way too complicated. What I meant is: 1. Keep all the logic in InitializeSandbox. Always call InitializeSandbox(). 2. In Initialize sandbox, in BMM mode, CHECK that the BPF sandbox is enabled, *except* if nacl_dangerous_no_sandbox_ok is true. This means that we keep the normal flag logic (--no-sandbox, or --disable-setuid-sandbox + --disable-seccomp-filters-sandbox), but someone using these flags will crash in BMM mode, except if using the --dangerous-XXX flag. The reason for doing it this way is that we can keep the flag logic as-is and not introduce any extra complexity. We simply add a new assertion that flags are using as appropriate for BMM.
https://codereview.chromium.org/226033002/diff/30001/components/nacl/common/n... File components/nacl/common/nacl_switches.h (right): https://codereview.chromium.org/226033002/diff/30001/components/nacl/common/n... components/nacl/common/nacl_switches.h:19: extern const char kNaClDangerousNoSandboxNonSfi[]; On 2014/04/08 22:25:16, jln wrote: > Please, add this to chrome/browser/ui/startup/bad_flags_prompt.cc as well. Done. https://codereview.chromium.org/226033002/diff/30001/components/nacl/loader/n... File components/nacl/loader/nacl_helper_linux.cc (right): https://codereview.chromium.org/226033002/diff/30001/components/nacl/loader/n... components/nacl/loader/nacl_helper_linux.cc:107: bool no_sandbox = false; On 2014/04/08 22:25:16, jln wrote: > This makes for a logic that is way too complicated. > > What I meant is: > > 1. Keep all the logic in InitializeSandbox. Always call InitializeSandbox(). > > 2. In Initialize sandbox, in BMM mode, CHECK that the BPF sandbox is enabled, > *except* if nacl_dangerous_no_sandbox_ok is true. > > This means that we keep the normal flag logic (--no-sandbox, or > --disable-setuid-sandbox + --disable-seccomp-filters-sandbox), but someone using > these flags will crash in BMM mode, except if using the --dangerous-XXX flag. > > The reason for doing it this way is that we can keep the flag logic as-is and > not introduce any extra complexity. We simply add a new assertion that flags are > using as appropriate for BMM. Done.
This looks ok in general, let's see what Mark says! https://chromiumcodereview.appspot.com/226033002/diff/70001/chrome/browser/ch... File chrome/browser/chrome_content_browser_client.cc (right): https://chromiumcodereview.appspot.com/226033002/diff/70001/chrome/browser/ch... chrome/browser/chrome_content_browser_client.cc:1641: switches::kNaClDangerousNoSandboxNonSfi, I don't know what this is, but it shouldn't be needed. Flags such as switches::kDisableSeccompFilterSandbox don't seem to require it.
Note: I'm planning to change a few things anyways: - Enforce the sandbox status (as in about:sandbox) in NaCl - Check for threads prior to engaging seccomp-bpf (as explained in https://crbug.com/359230, this is a major pitfall). Meanwhile, please make sure (manually) that everything works with --disable-setuid-sandbox (check in about:sandbox that the flag worked). By disabling the setuid sandbox, seccomp-bpf will be able to check for threads properly.
https://codereview.chromium.org/226033002/diff/70001/chrome/browser/chrome_co... File chrome/browser/chrome_content_browser_client.cc (right): https://codereview.chromium.org/226033002/diff/70001/chrome/browser/chrome_co... chrome/browser/chrome_content_browser_client.cc:1641: switches::kNaClDangerousNoSandboxNonSfi, On 2014/04/10 19:08:24, jln wrote: > I don't know what this is, but it shouldn't be needed. Flags such as > switches::kDisableSeccompFilterSandbox don't seem to require it. This is necessary to transfer the flag from browser process to zygote process. switches::kDisableSeccompFilterSandbox is handled in content/browser/zygote_host/zygote_host_impl_linux.cc. As zygote_host_impl_linux.cc is in content/, we cannot refer switches in component/. OTOH, kDisableSeccompFilterSandbox is a content_switch, so it can be directly referred in zygote_host_impl_linux.cc. It seems some other NaCl related flags are also using this file. You can see kEnableNaCl is listed in line 1592 to pass it from browser to renderer, for example.
lgtm (but let's wait for Mark) https://codereview.chromium.org/226033002/diff/70001/chrome/browser/chrome_co... File chrome/browser/chrome_content_browser_client.cc (right): https://codereview.chromium.org/226033002/diff/70001/chrome/browser/chrome_co... chrome/browser/chrome_content_browser_client.cc:1641: switches::kNaClDangerousNoSandboxNonSfi, On 2014/04/10 19:17:55, hamaji wrote: > On 2014/04/10 19:08:24, jln wrote: > > I don't know what this is, but it shouldn't be needed. Flags such as > > switches::kDisableSeccompFilterSandbox don't seem to require it. > > This is necessary to transfer the flag from browser process to zygote process. > > switches::kDisableSeccompFilterSandbox is handled in > content/browser/zygote_host/zygote_host_impl_linux.cc. As > zygote_host_impl_linux.cc is in content/, we cannot refer switches in > component/. OTOH, kDisableSeccompFilterSandbox is a content_switch, so it can be > directly referred in zygote_host_impl_linux.cc. > > It seems some other NaCl related flags are also using this file. You can see > kEnableNaCl is listed in line 1592 to pass it from browser to renderer, for > example. Ahh right, thanks!
Please update the commit message. It still refers to the NACL_DANGEROUS_DISABLE_NONSFI_SANDBOX env var. Then LGTM.
The CQ bit was checked by hamaji@chromium.org
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/hamaji@chromium.org/226033002/70001
The CQ bit was unchecked by commit-bot@chromium.org
Retried try job too often on chromium_presubmit for step(s) presubmit http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=chromium_p...
+sky for chrome/browser/ui OWNER approval.
LGTM
The CQ bit was checked by hamaji@chromium.org
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/hamaji@chromium.org/226033002/70001
The CQ bit was unchecked by commit-bot@chromium.org
The commit queue went berserk retrying too often for a seemingly flaky test on builder android_dbg: http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=android_db... http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=android_db... http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=android_db... http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=android_db... http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=android_db...
The CQ bit was checked by jln@chromium.org
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/hamaji@chromium.org/226033002/70001
Message was sent while issue was closed.
Change committed as 263824 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
