New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 633032 link

Starred by 5 users

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Mar 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 3
Type: Feature



Sign in to add a comment

Make it possible to use Mac hermetic build for WebRTC and libyuv

Project Member Reported by kjellander@chromium.org, Aug 1 2016

Issue description

Tracking bug for making it possible to set FORCE_MAC_TOOLCHAIN=1 in the environment, sync and build everything for WebRTC and libyuv.

This makes the build hermetic and independent of the currently installed Xcode version on the machine, which is great. The majority of the work on this was done in  bug 474373 .

Currently if just setting into the environment and syncing produces and error due to the hook in Chromium's DEPS file (https://cs.chromium.org/chromium/src/DEPS?rcl=0&l=729) only being executed when we roll the chromium_revision in WebRTC's DEPS file (due to the way the  sync_chromium.py script avoids re-running the Chromium hooks every time). 
We could add the hook executing build/mac_toolchain.py to WebRTC and libyuv to solve that.
 
Blocking: -626093
This doesn't actually block  Issue 626093 , since that's just trying to target all machines building chrome targets.

This maybe depends on  Issue 651267 , but that issue is pretty much complete.
Cc: erikc...@chromium.org
As we recently saw in https://codereview.webrtc.org/2447263003/#ps1 WebRTC fails build like this:

[424/3660] SOLINK libboringssl.dylib libboringssl.dylib.TOC
FAILED: libboringssl.dylib libboringssl.dylib.TOC 
export DEVELOPER_DIR=/b/c/b/mac_compile_dbg/src/build/mac_files/Xcode.app;  if [ ! -e "./libboringssl.dylib" -o ! -e "./libboringssl.dylib.TOC" ] || otool -l "./libboringssl.dylib" | grep -q LC_REEXPORT_DYLIB ; then TOOL_VERSION=1476359661 ../../build/toolchain/mac/linker_driver.py /b/c/cipd/goma/gomacc ../../third_party/llvm-build/Release+Asserts/bin/clang++ -shared  -Wl,-install_name,@rpath/"libboringssl.dylib" -stdlib=libc++ -arch x86_64 -isysroot /b/c/b/mac_compile_dbg/src/chromium/src/build/mac_files/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -mmacosx-version-min=10.7 -Wl,-ObjC -Wl,-rpath,@loader_path/. -Wl,-rpath,@loader_path/../../.. -o "./libboringssl.dylib" -Wl,-filelist,"./libboringssl.dylib.rsp"   && { otool -l "./libboringssl.dylib" | grep LC_ID_DYLIB -A 5; nm -gP "./libboringssl.dylib" | cut -f1-2 -d' ' | grep -v U$$; true; } > "./libboringssl.dylib.TOC"; else TOOL_VERSION=1476359661 ../../build/toolchain/mac/linker_driver.py /b/c/cipd/goma/gomacc ../../third_party/llvm-build/Release+Asserts/bin/clang++ -shared  -Wl,-install_name,@rpath/"libboringssl.dylib" -stdlib=libc++ -arch x86_64 -isysroot /b/c/b/mac_compile_dbg/src/chromium/src/build/mac_files/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -mmacosx-version-min=10.7 -Wl,-ObjC -Wl,-rpath,@loader_path/. -Wl,-rpath,@loader_path/../../.. -o "./libboringssl.dylib" -Wl,-filelist,"./libboringssl.dylib.rsp"   && { otool -l "./libboringssl.dylib" | grep LC_ID_DYLIB -A 5; nm -gP "./libboringssl.dylib" | cut -f1-2 -d' ' | grep -v U$$; true; } > "./libboringssl.dylib.tmp" && if ! cmp -s "./libboringssl.dylib.tmp" "./libboringssl.dylib.TOC"; then mv "./libboringssl.dylib.tmp" "./libboringssl.dylib.TOC" ; fi; fi
ld: library not found for -lc++
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Traceback (most recent call last):
  File "../../build/toolchain/mac/linker_driver.py", line 222, in <module>
    Main(sys.argv)
  File "../../build/toolchain/mac/linker_driver.py", line 72, in Main
    subprocess.check_call(compiler_driver_args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/b/c/cipd/goma/gomacc', '../../third_party/llvm-build/Release+Asserts/bin/clang++', '-shared', '-Wl,-install_name,@rpath/libboringssl.dylib', '-stdlib=libc++', '-arch', 'x86_64', '-isysroot', '/b/c/b/mac_compile_dbg/src/chromium/src/build/mac_files/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk', '-mmacosx-version-min=10.7', '-Wl,-ObjC', '-Wl,-rpath,@loader_path/.', '-Wl,-rpath,@loader_path/../../..', '-o', './libboringssl.dylib', '-Wl,-filelist,./libboringssl.dylib.rsp']' returned non-zero exit status 1

(https://build.chromium.org/p/tryserver.webrtc/builders/mac_compile_dbg/builds/9823)
when the 10.10 SDK is used (which is the one that's currently bundled in the hermetic Xcode package). 

As long as 10.11 is not available in the package, we'll be blocked.
The above issue was solved by removing the FORCE_MAC_TOOLCHAIN=1 environment variable from the bots, which was incorrectly set.
Cc: justincohen@chromium.org
An update on this: WebRTC and Chromium both now uses SDK 10.11 as target SDK, but we still get this compile error for many executable targets:
ld: library not found for -lc++

It doesn't seem related to the SDK version (which was incorrectly claimed earlier), but rather something that's different with our build.
We also get multiple linking errors due to missing Foundation library:

FAILED: RTPjitter 
export DEVELOPER_DIR=/Users/kjellander/dev/webrtc/src/build/mac_files/Xcode.app;  TOOL_VERSION=1478181388 ../../build/toolchain/mac/linker_driver.py ../../third_party/llvm-build/Release+Asserts/bin/clang++  -stdlib=libc++ -arch x86_64 -Werror -Wl,-dead_strip -isysroot /Users/kjellander/dev/webrtc/src/chromium/src/build/mac_files/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -mmacosx-version-min=10.9 -Wl,-ObjC -Wl,-pie -o "./RTPjitter" -Wl,-filelist,"./RTPjitter.rsp"  -framework Foundation
ld: framework not found Foundation
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Traceback (most recent call last):
  File "../../build/toolchain/mac/linker_driver.py", line 229, in <module>
    Main(sys.argv)
  File "../../build/toolchain/mac/linker_driver.py", line 79, in Main
    subprocess.check_call(compiler_driver_args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['../../third_party/llvm-build/Release+Asserts/bin/clang++', '-stdlib=libc++', '-arch', 'x86_64', '-Werror', '-Wl,-dead_strip', '-isysroot', '/Users/kjellander/dev/webrtc/src/chromium/src/build/mac_files/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk', '-mmacosx-version-min=10.9', '-Wl,-ObjC', '-Wl,-pie', '-o', './RTPjitter', '-Wl,-filelist,./RTPjitter.rsp', '-framework', 'Foundation']' returned non-zero exit status 1

I'm guessing this is related to the Xcode version used in that bundle (5.1.1).

webrtc builds successfully with the ios-hermetic bundle, so I'm guessing with mac moves to something newer (say >8) webrtc will also work.
Labels: -Type-Bug Type-Feature
Cc: phoglund@chromium.org
WebRTC has been on the hermetic Mac toolchain for about 2 weeks now (most work done referencing https://bugs.chromium.org/p/webrtc/issues/detail?id=8099).

It should be easy to do the same for libyuv with little changes, which I suggest we keep this bug open to track.
Cc: -ehmaldonado@chromium.org -erikc...@chromium.org -justincohen@chromium.org fbarchard@chromium.org
Owner: phoglund@chromium.org
Status: Assigned (was: Available)
For libyuv, setting export FORCE_MAC_TOOLCHAIN=1
and then do a build as usual doesnt seem any different than not setting it?
The toolchain in third_party is used.
--version says 6.0.0 (trunk 313786)

Project Member

Comment 10 by bugdroid1@chromium.org, Mar 20 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/libyuv/libyuv.git/+/6ecd76ae8da49b22ca380c7bb9fe9256730f031e

commit 6ecd76ae8da49b22ca380c7bb9fe9256730f031e
Author: Patrik Höglund <phoglund@chromium.org>
Date: Tue Mar 20 08:28:33 2018

Make libyuv use the hermetic xcode toolchain.

This is what Chromium is moving to, and what WebRTC is using.
Next step: pull xcode via CIPD.

Bug: chromium:633032 
Change-Id: I6a21aa0e5b212a3a9cb4e7cc79ebb614a89f73ed
Reviewed-on: https://chromium-review.googlesource.com/915921
Commit-Queue: Patrik Höglund <phoglund@chromium.org>
Reviewed-by: Frank Barchard <fbarchard@chromium.org>

[modify] https://crrev.com/6ecd76ae8da49b22ca380c7bb9fe9256730f031e/.gn
[modify] https://crrev.com/6ecd76ae8da49b22ca380c7bb9fe9256730f031e/DEPS
[modify] https://crrev.com/6ecd76ae8da49b22ca380c7bb9fe9256730f031e/build_overrides/build.gni

Status: Fixed (was: Assigned)
That should be it for this bug. I'll investigate if we can pull xcode via cipd on the side...

Sign in to add a comment