New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 653337 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Oct 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 3
Type: Bug

Blocked on:
issue 654598



Sign in to add a comment

Mac debug (non-component) build fails verify_order step

Project Member Reported by erikc...@chromium.org, Oct 6 2016

Issue description

This issue has the same symptoms as  Issue 630629 , but likely has a different root cause.

chrome revision: 0952e991d96c311fc55a4cf26506d4f5357f2f7d
toolchain: Xcode 5.1.1

gn args:
"""
  5 dcheck_always_on = false                                                        
  6 is_component_build = false                                                      
  7 is_debug = true                                                                 
  8 symbol_level = 0                                                                
  9 use_goma = true                                                                 
 10 enable_nacl = true     
"""

erikchen@erikchen-macpro ~/projects/chromium/src  (temp14)$ ninja -C out/gn -j 200 -k 1000 interactive_ui_tests
ninja: Entering directory `out/gn'
[1/11] ACTION //chrome:verify_chrome_framework_order(//build/toolchain/mac:clang_x64)
FAILED: obj/chrome/run_verify_chrome_framework_order.stamp 
python ../../chrome/tools/build/mac/run_verify_order.py --stamp obj/chrome/run_verify_chrome_framework_order.stamp _ChromeMain Chromium\ Framework.framework/Chromium\ Framework
../../chrome/tools/build/mac/verify_order: unordered symbols in Chromium Framework.framework/Chromium Framework:
__ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6insertIPKcEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorISA_EE5valueENS_11__wrap_iterIPcEEE4typeENSB_IS8_EESA_SA_
__ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6assignIPKcEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorISA_EE5valueERS5_E4typeESA_SA_
__ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6assignIPKhEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorISA_EE5valueERS5_E4typeESA_SA_
__ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6assignIPKwEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorISA_EE5valueERS5_E4typeESA_SA_
__ZNSt3__112basic_stringIwNS_11char_traitsIwEENS_9allocatorIwEEE6assignIPKcEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorISA_EE5valueERS5_E4typeESA_SA_
__ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6assignIPKtEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorISA_EE5valueERS5_E4typeESA_SA_
__ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6assignIPhEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorIS9_EE5valueERS5_E4typeES9_S9_
__ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6appendIPKcEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorISA_EE5valueERS5_E4typeESA_SA_
__ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6assignIPaEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorIS9_EE5valueERS5_E4typeES9_S9_
Traceback (most recent call last):
  File "../../chrome/tools/build/mac/run_verify_order.py", line 23, in <module>
    [ os.path.join(this_script_dir, 'verify_order') ] + unknown_args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['../../chrome/tools/build/mac/verify_order', '_ChromeMain', 'Chromium Framework.framework/Chromium Framework']' returned non-zero exit status 1
ninja: build stopped: cannot make progress due to previous errors.

 

Comment 1 by mark@chromium.org, Oct 6 2016

Can you figure out where those symbols are coming from by looking at object file symbol tables?
Cc: miguelg@chromium.org
Miguel reported this issue to me via chat a few weeks ago, and simply syncing resolved the issue.

Knowing where these symbols is coming from would be helpful. It seems like something may not be getting (re-)built correctly.
Based on https://bugs.chromium.org/p/chromium/issues/detail?id=630634, it looks like we just don't want to support this build configurations [no tests == not supported]. 

I can repro this with a fully clean build (rm -rf out/gn) on tip of tree: f2b92bd425480c8e75756a37ad7f30e6cbd12abc.

Unfortunately, these symbols appear in more than one library, so I haven't traced down a root cause:
__ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6insertIPKcEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorISA_EE5valueENS_11__wrap_iterIPcEEE4typeENSB_IS8_EESA_SA_

demangled:
std::__1::enable_if<(__is_forward_iterator<unsigned short const*>::value) && (__libcpp_string_gets_noexcept_iterator<unsigned short const*>::value), std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&>::type std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::assign<unsigned short const*>(unsigned short const*, unsigned short const*)

shows up in:
out/gn/obj/net/libnet.a 
out/gn/obj/chrome/browser/extensions/libextensions.a 
out/gn/obj/components/nacl/renderer/plugin/libnacl_trusted_plugin.a  
out/gn/obj/chrome/browser/extensions/extensions/extension_creator.o 
out/gn/obj/components/nacl/renderer/plugin/nacl_trusted_plugin/pnacl_translate_thread.o
out/gn/obj/content/browser/browser/indexed_db_leveldb_coding.o
out/gn/obj/media/cast/receiver/frame_buffer.o
out/gn/obj/net/net/hpack_decoder.o 

__ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6assignIPKtEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorISA_EE5valueERS5_E4typeESA_SA_

demangled:
std::__1::enable_if<(__is_forward_iterator<unsigned short const*>::value) && (__libcpp_string_gets_noexcept_iterator<unsigned short const*>::value), std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&>::type std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::assign<unsigned short const*>(unsigned short const*, unsigned short const*)

only shows up in:
out/gn/obj/base/libbase.a 
out/gn/obj/base/base/utf_string_conversions.o





Also, what SDK are you using?
I'm using the Xcode 5 toolchain with 10.10 SDK.
Cc: tapted@chromium.org
Owner: erikc...@chromium.org
Status: Assigned (was: Untriaged)
taking out of triage queue. My two cents - I think this config is already "unsupported" (or permanently broken) on Windows, and the outcome from  Issue 630634  was to default debug builds to component builds. But also, some things are only testable in non-component builds, and it would be nice to have the option of debug symbols still... - maybe we can warn about it when gn runs to say it's supported only on a "Best Effort" basis.
Status: WontFix (was: Assigned)
you can still have symbols in release builds, you just also get optimization.

Comment 9 by mark@chromium.org, Oct 7 2016

WontFix because you’re not having the problem anymore, or something else?
Based on c#7 [and lack of bot coverage], it seems like this config is unsupported. I just won't build with it in the future.

Comment 11 by mark@chromium.org, Oct 7 2016

Owner: mark@chromium.org
Status: Assigned (was: WontFix)
This config should work. It certainly worked very recently. Maybe not with the Xcode 5-10.10 SDK combination, but it ought to work with a more appropriate combination. If it’s broken with 8/10.10, I’ll fix it.
If you want it to work, consider adding bot coverage. This issue would be trivially caught by the CQ/waterfall. 

[fingers crossed that we will soon all be on the same xcode toolchain [and a sane one at that]]

Comment 13 by mark@chromium.org, Oct 7 2016

Xcode 8.0 8A218a
10.10 SDK
Checked-in clang 282487
Xcode’s ld 274.1
is_component_build = false
is_debug = true

I found

__ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6insertIPKcEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorISA_EE5valueENS_11__wrap_iterIPcEEE4typeENSB_IS8_EESA_SA_

in

obj/chrome/browser/extensions/extensions/extension_creator.o
obj/components/nacl/renderer/plugin/nacl_trusted_plugin/pnacl_translate_thread.o
obj/content/browser/browser/indexed_db_leveldb_coding.o
obj/media/cast/receiver/frame_buffer.o
obj/net/net/hpack_decoder.o

In each case, as weak external (N_EXT | N_WEAK_DEF). The private bit (N_PEXT) is not set.

Comment 14 by mark@chromium.org, Oct 7 2016

In a non-debug build, this symbol isn’t even present in

obj/chrome/browser/extensions/extensions/extension_creator.o
obj/media/cast/receiver/frame_buffer.o

It still does show up in

obj/components/nacl/renderer/plugin/nacl_trusted_plugin/pnacl_translate_thread.o
obj/content/browser/browser/indexed_db_leveldb_coding.o
obj/net/net/hpack_decoder.o

but as “weak external automatically hidden”, or (N_EXT | N_WEAK_DEF | N_WEAK_REF).

Comment 15 by mark@chromium.org, Oct 7 2016

Same problem as comment 13 with debug and the 10.12 SDK.

Comment 16 by mark@chromium.org, Oct 7 2016

Owner: thakis@chromium.org
Looks like a clang and libc++ problem concerning visibility of certain enable_if constructs internal to libc++.

Here’s a reduced testcase, based on what I saw happening at net/spdy/hpack/hpack_decoder.cc:55 (https://chromium.googlesource.com/chromium/src/+/766306f5e9b9c799703d9ff98384cd86553eb8b9/net/spdy/hpack/hpack_decoder.cc#55).

mark@garbage bash$ cat test.cc
#include <string.h>
#include <string>
int main(int argc, char* argv[]) {
  std::string b(argv[0], strlen(argv[0]));
  b.insert(b.end(), argv[1], argv[1] + strlen(argv[1]));
  return 0;
}

# With the system’s clang and libc++ from Xcode 8.0 8A218a, it’s “weak private
# external” (N_EXT | N_PEXT + N_WEAK_DEF). That’s good.

mark@garbage bash$ clang++ -fvisibility=hidden -fvisibility-inlines-hidden -std=c++11 -isysroot /SDKs/MacOSX10.12.sdk -c test.cc -o test.o && nm -m test.o | grep insert
0000000000000440 (__TEXT,__text) weak private external __ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6insertIPcEENS_9enable_ifIXsr21__is_forward_iteratorIT_EE5valueENS_11__wrap_iterIS7_EEE4typeENSA_IPKcEES9_S9_

mark@garbage bash$ clang++ -v
Apple LLVM version 8.0.0 (clang-800.0.38)
Target: x86_64-apple-darwin16.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

# With Chrome’s trunk clang and libc++, clang 282487 from Chrome 910d79fbfedb, it’s
# “weak external” (N_EXT | N_WEAK_DEF), no “private” at all. That’s bad.

mark@garbage bash$ /chrome/chrome/src/third_party/llvm-build/Release+Asserts/bin/clang++ -fvisibility=hidden -fvisibility-inlines-hidden -std=c++11 -isysroot /SDKs/MacOSX10.12.sdk -c test.cc -o test.o && nm -m test.o | grep insert
0000000000001dd0 (__TEXT,__text) weak external __ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6insertIPKcEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorISA_EE5valueENS_11__wrap_iterIPcEEE4typeENSB_IS8_EESA_SA_
00000000000007d0 (__TEXT,__text) weak external __ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6insertIPcEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorIS9_EE5valueENS_11__wrap_iterIS7_EEE4typeENSA_IPKcEES9_S9_

mark@garbage bash$ /chrome/chrome/src/third_party/llvm-build/Release+Asserts/bin/clang++ -v
clang version 4.0.0 (trunk 282487)
Target: x86_64-apple-darwin16.0.0
Thread model: posix
InstalledDir: /chrome/chrome/src/third_party/llvm-build/Release+Asserts/bin

# The symbols corresponding to these enable_if constructs will survive in a final
# linked executable’s symbol table when they’re not “private” or “automatically
# hidden.”

mark@garbage bash$ /chrome/chrome/src/third_party/llvm-build/Release+Asserts/bin/clang++ -fvisibility=hidden -fvisibility-inlines-hidden -std=c++11 -isysroot /SDKs/MacOSX10.12.sdk test.cc -Wl,-dead_strip -o test && strip test && nm -m test | grep insert
00000001000028d0 (__TEXT,__text) weak external __ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6insertIPKcEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorISA_EE5valueENS_11__wrap_iterIPcEEE4typeENSB_IS8_EESA_SA_
00000001000012e0 (__TEXT,__text) weak external __ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6insertIPcEENS_9enable_ifIXaasr21__is_forward_iteratorIT_EE5valuesr38__libcpp_string_gets_noexcept_iteratorIS9_EE5valueENS_11__wrap_iterIS7_EEE4typeENSA_IPKcEES9_S9_

# Meanwhile, mixing the system clang with Chrome’s libc++ gives “weak external
# automatically hidden” (N_EXT + N_WEAK_DEF | N_WEAK_REF), and mixing Chrome’s clang
# with the system libc++ gives “weak private external” (N_EXT | N_PEXT + N_WEAK_DEF).
# These are also both fine.
#
# Chrome’s clang with its own libc++ also gives “weak external automatically hidden”
# (N_EXT + N_WEAK_DEF | N_WEAK_REF) but only at -O1 and above. This problem only occurs
# without optimization.

The system’s libc++ headers are in /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1. Chrome’s are in /chrome/chrome/src/third_party/llvm-build/Release+Asserts/include/c++/v1.
Great comment, thanks! I'll have a look if some recent change broke this.

Comment 18 by mark@chromium.org, Oct 7 2016

I suspect that libc++ r281681 is related.

http://llvm.org/viewvc/llvm-project?view=revision&revision=281681

Thanks for taking it from here.
It's due to http://llvm.org/viewvc/llvm-project?view=revision&revision=281673 . I'll file upstream.
https://codereview.chromium.org/2409613002/ has the fix, maybe it'll help (didn't verify the fix)
Blockedon: 654598
Status: Fixed (was: Assigned)
I just ran `ninja -C out/gn -j 200 -k 1000 obj/chrome/run_verify_chrome_framework_order.stamp` with the gn config from comment 0, and things built fine. Seems like the libc++ fix that was brought in in the clang roll from 3 days ago (see blocking bug) helped.

Sign in to add a comment