Make_distrib script fails on macOS Monterey

Issue #3274 new
Patrick Heyer created an issue

After successfully building CEF from branch 4896, the make_distrib script seems to fail because it tries to use nm as provided by the default Xcode toolchain to find exported symbols, but because Chromium has been built with its own custom toolchain, the files are incompatible:

Finding exported symbols...
Traceback (most recent call last):
  File "make_distrib.py", line 1045, in <module>
    os.path.join(dst_dir, 'cef_sandbox.a'))
  File "make_distrib.py", line 372, in combine_libs
    for symbol in get_exported_symbols(os.path.join(build_dir, libs[0])):
  File "make_distrib.py", line 326, in get_exported_symbols
    raise Exception('ERROR: nm failed: %s' % result['err'])
Exception: ERROR: nm failed: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: error: /Users/***/***/chromium_git/chromium/src/out/Release_GN_x64/obj/cef/libcef_sandbox.a(sandbox_mac.o) Unknown attribute kind (114) (Producer: 'LLVM15.0.0git' Reader: 'LLVM APPLE_1_1300.0.29.30_0')

As an experiment I ensured that the nm binary found in the path is the one created by Chromium in /Users/***/***/chromium_git/chromium/src/third_party/llvm-build/Release+Asserts/bin , which allows this step to succeed, but then it fails when invoking the linker via ld:

ld: could not parse object file /Users/***/***/chromium_git/chromium/src/out/Release_GN_x64/obj/buildtools/third_party/libc++/libc++/algorithm.o: 'Unknown attribute kind (114) (Producer: 'LLVM15.0.0git' Reader: 'LLVM APPLE_1_1300.0.29.30_0')', using libLTO version 'LLVM version 13.0.0, (clang-1300.0.29.30)' file '/Users/patthemav/GitHub/CEF_x86_64/chromium_git/chromium/src/out/Release_GN_x64/obj/buildtools/third_party/libc++/libc++/algorithm.o'

Steps to Reproduce

Environment:

  • ProductName: macOS
  • ProductVersion: 12.2.1
  • BuildVersion: 21D62
  • CEF Branch: 4896
export GN_DEFINES="is_official_build=true proprietary_codecs=true ffmpeg_branding=Chrome"
export PATH=/Users/***/***/depot_tools:$PATH
  1. Run python ../automate/automate-git.py --download-dir=/Users/***/***/chromium_git --depot-tools-dir=/Users/***/***/depot_tools --no-distrib --x64-build --branch=4896 --no-build
  2. Change into the /Users/***/***/chromium_git/chromium_git/chromium/src/cef directory
  3. Run cef_create_projects.sh
  4. Change into the /Users/***/***/chromium_git/chromium_git/chromium/src/ directory
  5. Run ninja -C out/Release_GN_x64 cef
  6. Change into the /Users/***/***/chromium_git/chromium_git/chromium/src/cef/tools directory
  7. Run ./make_distrib.sh --ninja-build --minimal --x64-build
  8. Observe error

Comments (7)

  1. Patrick Heyer reporter

    I got version 13.2.1 (13C100) here. But good point, let me check some common-ish macOS developer tools gotchas.

    Edit: So I checked if there are other versions of the developer tools installed but that’s the only version on my system and also that’s the version selected from Xcode’s preferences.

    Edit 2: Tried the same on my M1 Mac, compilation went fine, but make_distrib doesn’t fail over nm (probably because it’s arm64) and fails straight at ld with the same error as above. System is also on newest macOS and Xcode 13.2.1.

  2. Patrick Heyer reporter

    @Marshall Greenblatt FWIW I still get those issues with CEF branch 4951:

    ld: could not parse object file /Users/***/***/CEF_arm64/chromium_git/chromium/src/out/Release_GN_arm64/obj/buildtools/third_party/libc++/libc++/algorithm.o: 'Not an int attribute (Producer: 'LLVM15.0.0git' Reader: 'LLVM APPLE_1_1316.0.21.2.3_0')', using libLTO version 'LLVM version 13.1.6, (clang-1316.0.21.2.3)' file '/Users/***/***/CEF_arm64/chromium_git/chromium/src/out/Release_GN_arm64/obj/buildtools/third_party/libc++/libc++/algorithm.o'
    

    I’ve got Xcode 13.3.1 installed, but during the automation script I got the following events:

    Apply runhooks.patch in /Users/***/***/CEF_arm64/chromium_git/chromium/src
        9   5   build/toolchain/win/setup_toolchain.py
        11  0   build/vs_toolchain.py
    ... successfully applied.
    -------- Running "gclient runhooks --jobs 16" in "/Users/***/***/CEF_arm64/chromium_git/chromium"...
    
    ________ running 'python3 src/build/mac_toolchain.py' in '/Users/***/***/CEF_arm64/chromium_git/chromium'
    Skipping Mac toolchain installation for mac
    Running hooks:  14% (18/122) clang               
    ________ running 'python3 src/tools/clang/scripts/update.py' in '/Users/***/***/CEF_arm64/chromium_git/chromium'
    Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Mac_arm64/clang-llvmorg-15-init-3677-g8133778d-4.tgz .......... Done.
    Running hooks:  19% (24/122) objdump       
    ________ running 'python3 src/tools/clang/scripts/update.py --package=objdump' in '/Users/***/***/CEF_arm64/chromium_git/chromium'
    Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Mac_arm64/llvmobjdump-llvmorg-15-init-3677-g8133778d-4.tgz .......... Done.
    Running hooks:  24% (30/122) clang_format_mac_arm64
    ________ running 'python3 src/third_party/depot_tools/download_from_google_storage.py --no_resume --no_auth --bucket chromium-clang-format -s src/buildtools/mac/clang-format.arm64.sha1 -o src/buildtools/mac/clang-format' in '/Users/***/***/CEF_arm64/chromium_git/chromium'
    0> Downloading src/buildtools/mac/clang-format@5ba974b3b37f9f4e3b44fdde11d7ef2ab71619ab...
    Downloading 1 files took 5.908320 second(s)
    

    So it seems to use the custom build toolchain and not the one provided by Xcode.

  3. Marshall Greenblatt

    Chromium bundles its own version of clang but still requires Xcode for SDKs, some tools, etc.

  4. Patrick Heyer reporter

    Yeah that makes sense - I cannot figure out how to ensure that the libraries built by Chromium’s build system use the native build tools, because those are what CEF’s make_distrib script seems to use.

    I nuked the entire directory, re-downloaded the automate-git.py script and also did a clean checkout of depot_tools and removed everything in the chromium_git subdirectory, and yet it fails. I also temporarily removed pyenv from the equation and also removed the Homebrew paths to ensure an almost vanilla-macOS with no success.

  5. Patrick Heyer reporter

    I’ve had another person run through the same steps and they ended up with the same issue on their machine (I wanted to ensure that it’s not an issue local to mine).

    Running the whole script fully automatic however (e.g. python3 ../automate/automate-git.py --download-dir=/Users/***/***/CEF_arm64/chromium_git --depot-tools-dir=/Users/***/**/CEF_arm64/depot_tools --arm64-build --branch=4951) doesn’t result in the same issue. Once you trigger the build manually, the problem reoccurs.

  6. pkv

    I can confirm the bug w/ cef 5060. I have a similar error message with nm:

    Exception: ERROR: nm failed: 
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: 
    error: /Volumes/mac/code/chromium_git/chromium/src/out/Release_GN_x64/obj/cef/libcef_sandbox.a(sandbox_mac.o): 
    Not an int attribute (Producer: 'LLVM15.0.0git' Reader: 'LLVM APPLE_1_1316.0.21.2.5_0')
    

    Xcode version is 13.4.1 on macos 12.4.

  7. Log in to comment