|
|
Created:
4 years, 3 months ago by Corentin Wallez Modified:
4 years, 2 months ago CC:
capn, chromium-reviews Target Ref:
refs/pending/heads/master Project:
chromium Visibility:
Public. |
DescriptionMac toolchain: take output_dir into account for .dSYM
This is causing a conflict between Swiftshader's libEGL.dylib.dSYM
and ANGLE's libEGL.dylib.dSYM when building with is_asan=true on Mac
BUG=630728
Patch Set 1 #Patch Set 2 : Mac toolchain: take output_dir into account for .dSYM #Messages
Total messages: 28 (12 generated)
cwallez@chromium.org changed reviewers: + brettw@chromium.org, dpranke@chromium.org
The CQ bit was checked by cwallez@chromium.org to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: This issue passed the CQ dry run.
The CQ bit was checked by cwallez@chromium.org to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
sugoi@google.com changed reviewers: + sugoi@google.com
Note that I had 4 bots demonstrating this issue on my side: - mac_chromium_archive_rel_ng - mac_chromium_10.10_macviews - mac_chromium_asan_rel_ng - ios-simulator-cronet None of these bots are in the dry run.
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: This issue passed the CQ dry run.
Description was changed from ========== Mac toolchain: take output_dir into account for .dSYM This is causing a conflict between Swiftshader's libEGL.dylib.dSYM and ANGLE's libEGL.dylib.dSYM when building with is_asan=true on Mac BUG=630728 ========== to ========== Mac toolchain: take output_dir into account for .dSYM This is causing a conflict between Swiftshader's libEGL.dylib.dSYM and ANGLE's libEGL.dylib.dSYM when building with is_asan=true on Mac BUG=630728 ==========
lgtm.
rsesek@chromium.org changed reviewers: + rsesek@chromium.org
Does this actually work locally when doing an official build? mac_app_bundle and mac_framework_bundle specify an output_dir override, but the binary ends up being copied into a bundle. And we want the dSYMs/unstripped's to end up next to the bundle.
On 2016/09/21 21:20:44, Robert Sesek wrote: > Does this actually work locally when doing an official build? mac_app_bundle and > mac_framework_bundle specify an output_dir override, but the binary ends up > being copied into a bundle. And we want the dSYMs/unstripped's to end up next to > the bundle. Ok, the situation on Windows and Linux is that we have 2 libraries with the same names, in different directories, and we pick the right one at runtime. How do you suggest we proceed on Mac?
On 2016/09/21 at 21:33:05, sugoi wrote: > On 2016/09/21 21:20:44, Robert Sesek wrote: > > Does this actually work locally when doing an official build? mac_app_bundle and > > mac_framework_bundle specify an output_dir override, but the binary ends up > > being copied into a bundle. And we want the dSYMs/unstripped's to end up next to > > the bundle. > > Ok, the situation on Windows and Linux is that we have 2 libraries with the same names, in different directories, and we pick the right one at runtime. How do you suggest we proceed on Mac? For now ANGLE and Swiftshader are not used by Chromium on Mac so the problem wouldn't appear and we can't really test. Once we start enabling them on Mac we'll have to figure out how to bundle all of this. Right now we need this change so that both ANGLE and Swiftshader compile on Mac so we can do automated testing there. If you want I can still test doing an official build if you can point me to instructions.
On 2016/09/22 13:56:40, Corentin Wallez wrote: > On 2016/09/21 at 21:33:05, sugoi wrote: > > On 2016/09/21 21:20:44, Robert Sesek wrote: > > > Does this actually work locally when doing an official build? mac_app_bundle > and > > > mac_framework_bundle specify an output_dir override, but the binary ends up > > > being copied into a bundle. And we want the dSYMs/unstripped's to end up > next to > > > the bundle. > > > > Ok, the situation on Windows and Linux is that we have 2 libraries with the > same names, in different directories, and we pick the right one at runtime. How > do you suggest we proceed on Mac? > > For now ANGLE and Swiftshader are not used by Chromium on Mac so the problem > wouldn't appear and we can't really test. Once we start enabling them on Mac > we'll have to figure out how to bundle all of this. Right now we need this > change so that both ANGLE and Swiftshader compile on Mac so we can do automated > testing there. What is the difference between the two copies of the library? It sounds like one is for ASan and one is normal? > If you want I can still test doing an official build if you can point me to > instructions. I'm pretty sure this won't work as-is, but for testing the official build, just set |is_official_build = true| in your gn args.
On 2016/09/22 17:18:11, Robert Sesek wrote: > On 2016/09/22 13:56:40, Corentin Wallez wrote: > > On 2016/09/21 at 21:33:05, sugoi wrote: > > > On 2016/09/21 21:20:44, Robert Sesek wrote: > > > > Does this actually work locally when doing an official build? > mac_app_bundle > > and > > > > mac_framework_bundle specify an output_dir override, but the binary ends > up > > > > being copied into a bundle. And we want the dSYMs/unstripped's to end up > > next to > > > > the bundle. > > > > > > Ok, the situation on Windows and Linux is that we have 2 libraries with the > > same names, in different directories, and we pick the right one at runtime. > How > > do you suggest we proceed on Mac? > > > > For now ANGLE and Swiftshader are not used by Chromium on Mac so the problem > > wouldn't appear and we can't really test. Once we start enabling them on Mac > > we'll have to figure out how to bundle all of this. Right now we need this > > change so that both ANGLE and Swiftshader compile on Mac so we can do > automated > > testing there. > > What is the difference between the two copies of the library? It sounds like one > is for ASan and one is normal? No, one is a GPU accelerated implementation of the OpenGL ES API, one is a software implementation of the OpenGL API. Chrome chooses one based on the GPU black list. This is unrelated to ASAN. > > If you want I can still test doing an official build if you can point me to > > instructions. > > I'm pretty sure this won't work as-is, but for testing the official build, just > set |is_official_build = true| in your gn args. We want Angle and SwiftShader to compile on Mac to make sure we don't break Mac compilation as development goes forward. Angle and SwiftShader are not yet integrated into the Chrome on Mac, but that's a currently ongoing project.
On 2016/09/22 17:31:35, sugoi1 wrote: > On 2016/09/22 17:18:11, Robert Sesek wrote: > > On 2016/09/22 13:56:40, Corentin Wallez wrote: > > > On 2016/09/21 at 21:33:05, sugoi wrote: > > > > On 2016/09/21 21:20:44, Robert Sesek wrote: > > > > > Does this actually work locally when doing an official build? > > mac_app_bundle > > > and > > > > > mac_framework_bundle specify an output_dir override, but the binary ends > > up > > > > > being copied into a bundle. And we want the dSYMs/unstripped's to end up > > > next to > > > > > the bundle. > > > > > > > > Ok, the situation on Windows and Linux is that we have 2 libraries with > the > > > same names, in different directories, and we pick the right one at runtime. > > How > > > do you suggest we proceed on Mac? > > > > > > For now ANGLE and Swiftshader are not used by Chromium on Mac so the problem > > > wouldn't appear and we can't really test. Once we start enabling them on Mac > > > we'll have to figure out how to bundle all of this. Right now we need this > > > change so that both ANGLE and Swiftshader compile on Mac so we can do > > automated > > > testing there. > > > > What is the difference between the two copies of the library? It sounds like > one > > is for ASan and one is normal? > > No, one is a GPU accelerated implementation of the OpenGL ES API, one is a > software implementation of the OpenGL API. Chrome chooses one based on the GPU > black list. This is unrelated to ASAN. (I meant to say OpenGL ES API in both cases) > > > If you want I can still test doing an official build if you can point me to > > > instructions. > > > > I'm pretty sure this won't work as-is, but for testing the official build, > just > > set |is_official_build = true| in your gn args. > > We want Angle and SwiftShader to compile on Mac to make sure we don't break Mac > compilation as development goes forward. Angle and SwiftShader are not yet > integrated into the Chrome on Mac, but that's a currently ongoing project.
On 2016/09/22 at 17:18:11, rsesek wrote: > On 2016/09/22 13:56:40, Corentin Wallez wrote: > > On 2016/09/21 at 21:33:05, sugoi wrote: > > > On 2016/09/21 21:20:44, Robert Sesek wrote: > > > > Does this actually work locally when doing an official build? mac_app_bundle > > and > > > > mac_framework_bundle specify an output_dir override, but the binary ends up > > > > being copied into a bundle. And we want the dSYMs/unstripped's to end up > > next to > > > > the bundle. > > > > > > Ok, the situation on Windows and Linux is that we have 2 libraries with the > > same names, in different directories, and we pick the right one at runtime. How > > do you suggest we proceed on Mac? > > > > For now ANGLE and Swiftshader are not used by Chromium on Mac so the problem > > wouldn't appear and we can't really test. Once we start enabling them on Mac > > we'll have to figure out how to bundle all of this. Right now we need this > > change so that both ANGLE and Swiftshader compile on Mac so we can do automated > > testing there. > > What is the difference between the two copies of the library? It sounds like one is for ASan and one is normal? One is Swiftshader's (software rasterizer) implementation of the EGL API, the other one is ANGLE's (graphics API translator and validator). The code of the two libraries is different. > > > If you want I can still test doing an official build if you can point me to > > instructions. > > I'm pretty sure this won't work as-is, but for testing the official build, just set |is_official_build = true| in your gn args. Ok.
On 2016/09/22 17:33:53, Corentin Wallez wrote: > On 2016/09/22 at 17:18:11, rsesek wrote: > > On 2016/09/22 13:56:40, Corentin Wallez wrote: > > > On 2016/09/21 at 21:33:05, sugoi wrote: > > > > On 2016/09/21 21:20:44, Robert Sesek wrote: > > > > > Does this actually work locally when doing an official build? > mac_app_bundle > > > and > > > > > mac_framework_bundle specify an output_dir override, but the binary ends > up > > > > > being copied into a bundle. And we want the dSYMs/unstripped's to end up > > > next to > > > > > the bundle. > > > > > > > > Ok, the situation on Windows and Linux is that we have 2 libraries with > the > > > same names, in different directories, and we pick the right one at runtime. > How > > > do you suggest we proceed on Mac? > > > > > > For now ANGLE and Swiftshader are not used by Chromium on Mac so the problem > > > wouldn't appear and we can't really test. Once we start enabling them on Mac > > > we'll have to figure out how to bundle all of this. Right now we need this > > > change so that both ANGLE and Swiftshader compile on Mac so we can do > automated > > > testing there. > > > > What is the difference between the two copies of the library? It sounds like > one is for ASan and one is normal? > > One is Swiftshader's (software rasterizer) implementation of the EGL API, the > other one is ANGLE's (graphics API translator and validator). The code of the > two libraries is different. Do they need to be named the same thing? Having unique names is probably the easiest solution. > > > If you want I can still test doing an official build if you can point me to > > > instructions. > > > > I'm pretty sure this won't work as-is, but for testing the official build, > just set |is_official_build = true| in your gn args. > > Ok.
On 2016/09/22 18:12:07, Robert Sesek wrote: > On 2016/09/22 17:33:53, Corentin Wallez wrote: > > On 2016/09/22 at 17:18:11, rsesek wrote: > > > On 2016/09/22 13:56:40, Corentin Wallez wrote: > > > > On 2016/09/21 at 21:33:05, sugoi wrote: > > > > > On 2016/09/21 21:20:44, Robert Sesek wrote: > > > > > > Does this actually work locally when doing an official build? > > mac_app_bundle > > > > and > > > > > > mac_framework_bundle specify an output_dir override, but the binary > ends > > up > > > > > > being copied into a bundle. And we want the dSYMs/unstripped's to end > up > > > > next to > > > > > > the bundle. > > > > > > > > > > Ok, the situation on Windows and Linux is that we have 2 libraries with > > the > > > > same names, in different directories, and we pick the right one at > runtime. > > How > > > > do you suggest we proceed on Mac? > > > > > > > > For now ANGLE and Swiftshader are not used by Chromium on Mac so the > problem > > > > wouldn't appear and we can't really test. Once we start enabling them on > Mac > > > > we'll have to figure out how to bundle all of this. Right now we need this > > > > change so that both ANGLE and Swiftshader compile on Mac so we can do > > automated > > > > testing there. > > > > > > What is the difference between the two copies of the library? It sounds like > > one is for ASan and one is normal? > > > > One is Swiftshader's (software rasterizer) implementation of the EGL API, the > > other one is ANGLE's (graphics API translator and validator). The code of the > > two libraries is different. > > Do they need to be named the same thing? Having unique names is probably the > easiest solution. On Windows, I know references to DLL names prevent us from renaming the DLLs, so we can't rename them on that platform. Also, for testing purposes, we can easily drop DLLs and replace them with one another, which is really simple and fast. We'd like to avoid having different library names on different platforms, if possible, because that would complicate the code a little. At this point, all we really want is to be able to build both on Mac simultaneously, like we do on Windows and Linux, and use build bots to verify that our code changes will not break the Mac platform, and neither of these libraries will ship until we are done integrating them. Note that, currently, SwiftShader is still a component, so if it ships on Mac, it will be as a component. SwiftShader is integrated into Chromium because it aims at replacing osmesa to run layout tests, which may also apply to Mac. I don't want to close the door on any possible solution, I'm just pointing out our current use cases and goals in order to choose the best solution. > > > > If you want I can still test doing an official build if you can point me > to > > > > instructions. > > > > > > I'm pretty sure this won't work as-is, but for testing the official build, > > just set |is_official_build = true| in your gn args. > > > > Ok.
On 2016/09/22 at 18:30:48, sugoi wrote: > On 2016/09/22 18:12:07, Robert Sesek wrote: > > On 2016/09/22 17:33:53, Corentin Wallez wrote: > > > On 2016/09/22 at 17:18:11, rsesek wrote: > > > > On 2016/09/22 13:56:40, Corentin Wallez wrote: > > > > > On 2016/09/21 at 21:33:05, sugoi wrote: > > > > > > On 2016/09/21 21:20:44, Robert Sesek wrote: > > > > > > > Does this actually work locally when doing an official build? > > > mac_app_bundle > > > > > and > > > > > > > mac_framework_bundle specify an output_dir override, but the binary > > ends > > > up > > > > > > > being copied into a bundle. And we want the dSYMs/unstripped's to end > > up > > > > > next to > > > > > > > the bundle. > > > > > > > > > > > > Ok, the situation on Windows and Linux is that we have 2 libraries with > > > the > > > > > same names, in different directories, and we pick the right one at > > runtime. > > > How > > > > > do you suggest we proceed on Mac? > > > > > > > > > > For now ANGLE and Swiftshader are not used by Chromium on Mac so the > > problem > > > > > wouldn't appear and we can't really test. Once we start enabling them on > > Mac > > > > > we'll have to figure out how to bundle all of this. Right now we need this > > > > > change so that both ANGLE and Swiftshader compile on Mac so we can do > > > automated > > > > > testing there. > > > > > > > > What is the difference between the two copies of the library? It sounds like > > > one is for ASan and one is normal? > > > > > > One is Swiftshader's (software rasterizer) implementation of the EGL API, the > > > other one is ANGLE's (graphics API translator and validator). The code of the > > > two libraries is different. > > > > Do they need to be named the same thing? Having unique names is probably the > > easiest solution. > > On Windows, I know references to DLL names prevent us from renaming the DLLs, so we can't rename them on that platform. Also, for testing purposes, we can easily drop DLLs and replace them with one another, which is really simple and fast. We'd like to avoid having different library names on different platforms, if possible, because that would complicate the code a little. At this point, all we really want is to be able to build both on Mac simultaneously, like we do on Windows and Linux, and use build bots to verify that our code changes will not break the Mac platform, and neither of these libraries will ship until we are done integrating them. Note that, currently, SwiftShader is still a component, so if it ships on Mac, it will be as a component. SwiftShader is integrated into Chromium because it aims at replacing osmesa to run layout tests, which may also apply to Mac. I don't want to close the door on any possible solution, I'm just pointing out our current use cases and goals in order to choose the best solution. > > > > > > If you want I can still test doing an official build if you can point me > > to > > > > > instructions. > > > > > > > > I'm pretty sure this won't work as-is, but for testing the official build, > > > just set |is_official_build = true| in your gn args. > > > > > > Ok. Compilation worked.
On 2016/09/22 20:47:57, Corentin Wallez wrote: > Compilation worked. Robert, do you have a strong objection to landing the code in this cl? It seems to work fine and any conflict that may happen later on between SwiftShader and Angle will be our issue once we integrate both on Mac. If we hit this issue, we can always fix mac_app_bundle and mac_framework_bundle at that point to allow both libraries to coexist in the bundle. It's your call.
On 2016/09/22 20:47:57, Corentin Wallez wrote: > On 2016/09/22 at 18:30:48, sugoi wrote: > > On 2016/09/22 18:12:07, Robert Sesek wrote: > > > On 2016/09/22 17:33:53, Corentin Wallez wrote: > > > > On 2016/09/22 at 17:18:11, rsesek wrote: > > > > > On 2016/09/22 13:56:40, Corentin Wallez wrote: > > > > > > On 2016/09/21 at 21:33:05, sugoi wrote: > > > > > > > On 2016/09/21 21:20:44, Robert Sesek wrote: > > > > > > > > Does this actually work locally when doing an official build? > > > > mac_app_bundle > > > > > > and > > > > > > > > mac_framework_bundle specify an output_dir override, but the > binary > > > ends > > > > up > > > > > > > > being copied into a bundle. And we want the dSYMs/unstripped's to > end > > > up > > > > > > next to > > > > > > > > the bundle. > > > > > > > > > > > > > > Ok, the situation on Windows and Linux is that we have 2 libraries > with > > > > the > > > > > > same names, in different directories, and we pick the right one at > > > runtime. > > > > How > > > > > > do you suggest we proceed on Mac? > > > > > > > > > > > > For now ANGLE and Swiftshader are not used by Chromium on Mac so the > > > problem > > > > > > wouldn't appear and we can't really test. Once we start enabling them > on > > > Mac > > > > > > we'll have to figure out how to bundle all of this. Right now we need > this > > > > > > change so that both ANGLE and Swiftshader compile on Mac so we can do > > > > automated > > > > > > testing there. > > > > > > > > > > What is the difference between the two copies of the library? It sounds > like > > > > one is for ASan and one is normal? > > > > > > > > One is Swiftshader's (software rasterizer) implementation of the EGL API, > the > > > > other one is ANGLE's (graphics API translator and validator). The code of > the > > > > two libraries is different. > > > > > > Do they need to be named the same thing? Having unique names is probably the > > > easiest solution. > > > > On Windows, I know references to DLL names prevent us from renaming the DLLs, > so we can't rename them on that platform. Also, for testing purposes, we can > easily drop DLLs and replace them with one another, which is really simple and > fast. We'd like to avoid having different library names on different platforms, > if possible, because that would complicate the code a little. At this point, all > we really want is to be able to build both on Mac simultaneously, like we do on > Windows and Linux, and use build bots to verify that our code changes will not > break the Mac platform, and neither of these libraries will ship until we are > done integrating them. Note that, currently, SwiftShader is still a component, > so if it ships on Mac, it will be as a component. SwiftShader is integrated into > Chromium because it aims at replacing osmesa to run layout tests, which may also > apply to Mac. I don't want to close the door on any possible solution, I'm just > pointing out our current use cases and goals in order to choose the best > solution. > > I'm a little confused. If you just want the code compiled now, then how does renaming one of the targets (on Mac-only) complicate the code? I understand not wanting to select an integration method at this point, but that makes it difficult to give advice on the right way to package this into the build. > > > > > > If you want I can still test doing an official build if you can point > me > > > to > > > > > > instructions. > > > > > > > > > > I'm pretty sure this won't work as-is, but for testing the official > build, > > > > just set |is_official_build = true| in your gn args. > > > > > > > > Ok. > > Compilation worked. Do you have a "Chromium Framework.dSYM" and "Chromium.dSYM" in the root of your output directory?
On 2016/09/23 15:35:23, Robert Sesek wrote: > On 2016/09/22 20:47:57, Corentin Wallez wrote: > > On 2016/09/22 at 18:30:48, sugoi wrote: > > > On 2016/09/22 18:12:07, Robert Sesek wrote: > > > > On 2016/09/22 17:33:53, Corentin Wallez wrote: > > > > > On 2016/09/22 at 17:18:11, rsesek wrote: > > > > > > On 2016/09/22 13:56:40, Corentin Wallez wrote: > > > > > > > On 2016/09/21 at 21:33:05, sugoi wrote: > > > > > > > > On 2016/09/21 21:20:44, Robert Sesek wrote: > > > > > > > > > Does this actually work locally when doing an official build? > > > > > mac_app_bundle > > > > > > > and > > > > > > > > > mac_framework_bundle specify an output_dir override, but the > > binary > > > > ends > > > > > up > > > > > > > > > being copied into a bundle. And we want the dSYMs/unstripped's > to > > end > > > > up > > > > > > > next to > > > > > > > > > the bundle. > > > > > > > > > > > > > > > > Ok, the situation on Windows and Linux is that we have 2 libraries > > with > > > > > the > > > > > > > same names, in different directories, and we pick the right one at > > > > runtime. > > > > > How > > > > > > > do you suggest we proceed on Mac? > > > > > > > > > > > > > > For now ANGLE and Swiftshader are not used by Chromium on Mac so the > > > > problem > > > > > > > wouldn't appear and we can't really test. Once we start enabling > them > > on > > > > Mac > > > > > > > we'll have to figure out how to bundle all of this. Right now we > need > > this > > > > > > > change so that both ANGLE and Swiftshader compile on Mac so we can > do > > > > > automated > > > > > > > testing there. > > > > > > > > > > > > What is the difference between the two copies of the library? It > sounds > > like > > > > > one is for ASan and one is normal? > > > > > > > > > > One is Swiftshader's (software rasterizer) implementation of the EGL > API, > > the > > > > > other one is ANGLE's (graphics API translator and validator). The code > of > > the > > > > > two libraries is different. > > > > > > > > Do they need to be named the same thing? Having unique names is probably > the > > > > easiest solution. > > > > > > On Windows, I know references to DLL names prevent us from renaming the > DLLs, > > so we can't rename them on that platform. Also, for testing purposes, we can > > easily drop DLLs and replace them with one another, which is really simple and > > fast. We'd like to avoid having different library names on different > platforms, > > if possible, because that would complicate the code a little. At this point, > all > > we really want is to be able to build both on Mac simultaneously, like we do > on > > Windows and Linux, and use build bots to verify that our code changes will not > > break the Mac platform, and neither of these libraries will ship until we are > > done integrating them. Note that, currently, SwiftShader is still a component, > > so if it ships on Mac, it will be as a component. SwiftShader is integrated > into > > Chromium because it aims at replacing osmesa to run layout tests, which may > also > > apply to Mac. I don't want to close the door on any possible solution, I'm > just > > pointing out our current use cases and goals in order to choose the best > > solution. > > > > > I'm a little confused. If you just want the code compiled now, then how does > renaming one of the targets (on Mac-only) complicate the code? I understand not > wanting to select an integration method at this point, but that makes it > difficult to give advice on the right way to package this into the build. We only need to land compilation, but we're still going to do local development, which will require platform specific library names if we choose that option. Currently, every implementation of that API, Angle, SwiftShader, Mesa, etc uses the library names "libEGL" and "libGLESv2". "libEGL" is a standard name mentioned in Khronos API Implementers Guide: https://www.khronos.org/registry/implementers_guide.html So renaming it makes us cringe a little. > > > > > > > If you want I can still test doing an official build if you can > point > > me > > > > to > > > > > > > instructions. > > > > > > > > > > > > I'm pretty sure this won't work as-is, but for testing the official > > build, > > > > > just set |is_official_build = true| in your gn args. > > > > > > > > > > Ok. > > > > Compilation worked. > > Do you have a "Chromium Framework.dSYM" and "Chromium.dSYM" in the root of your > output directory? |