GYP error on checking out new chromium checkout |
||||||
Issue descriptionVersion: 2d26a030a05d92fbb7489d62cfb7759b8f045335 (ToT on 4/26/2016) OS: OS X El Capitan 10.11.4 What steps will reproduce the problem? (1) In some directory with plenty of space, do "fetch chromium" (2) When it errors with "gclient sync" returning non-zero exit status, re-execute gclient sync. What is the expected output? A long wait, and then a clean chromium checkout ready to build. What do you see instead? A failure in generating the XCode projects because of duplicate mksnapshot entries. See below. rdsmith-macbookpro:../Sandboxen/testchrome $ fetch chromium Running: gclient root Running: gclient config --spec 'solutions = [ { "managed": False, "name": "src", "url": "https://chromium.googlesource.com/chromium/src.git", "custom_deps": {}, "deps_file": ".DEPS.git", "safesync_url": "", }, ] ' Running: gclient sync Error: Command '/usr/bin/python src/build/gyp_chromium --running-as-hook' returned non-zero exit status 1 in /Users/rdsmith/Sandboxen/testchrome Traceback (most recent call last): File "/Users/rdsmith/Sandboxen/depot_tools/fetch.py", line 346, in <module> sys.exit(main()) File "/Users/rdsmith/Sandboxen/depot_tools/fetch.py", line 341, in main return run(options, spec, root) File "/Users/rdsmith/Sandboxen/depot_tools/fetch.py", line 335, in run return checkout.init() File "/Users/rdsmith/Sandboxen/depot_tools/fetch.py", line 142, in init self.run_gclient(*sync_cmd) File "/Users/rdsmith/Sandboxen/depot_tools/fetch.py", line 76, in run_gclient return self.run(cmd_prefix + cmd, **kwargs) File "/Users/rdsmith/Sandboxen/depot_tools/fetch.py", line 66, in run return subprocess.check_output(cmd, **kwargs) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 573, in check_output raise CalledProcessError(retcode, cmd, output=output) subprocess.CalledProcessError: Command '('gclient', 'sync')' returned non-zero exit status 2 rdsmith-macbookpro:../Sandboxen/testchrome $ gclient sync Syncing projects: 100% (64/64), done. ________ running '/usr/bin/python src/build/landmines.py' in '/Users/rdsmith/Sandboxen/testchrome' ________ running '/usr/bin/python src/tools/remove_stale_pyc_files.py src/android_webview/tools src/build/android src/gpu/gles2_conform_support src/infra src/ppapi src/printing src/third_party/catapult src/third_party/closure_compiler/build src/tools' in '/Users/rdsmith/Sandboxen/testchrome' ________ running '/usr/bin/python src/build/download_nacl_toolchains.py --mode nacl_core_sdk sync --extract' in '/Users/rdsmith/Sandboxen/testchrome' ________ running '/usr/bin/python src/build/android/play_services/update.py download' in '/Users/rdsmith/Sandboxen/testchrome' ________ running '/usr/bin/python src/build/linux/sysroot_scripts/install-sysroot.py --running-as-hook' in '/Users/rdsmith/Sandboxen/testchrome' ________ running '/usr/bin/python src/build/vs_toolchain.py update' in '/Users/rdsmith/Sandboxen/testchrome' ________ running '/usr/bin/python src/build/mac_toolchain.py' in '/Users/rdsmith/Sandboxen/testchrome' Using local toolchain. ________ running '/usr/bin/python src/third_party/binutils/download.py' in '/Users/rdsmith/Sandboxen/testchrome' ________ running '/usr/bin/python src/tools/clang/scripts/update.py --if-needed' in '/Users/rdsmith/Sandboxen/testchrome' Updating Clang to 267383-1... Clang is already up to date. ________ running '/usr/bin/python src/build/util/lastchange.py -o src/build/util/LASTCHANGE' in '/Users/rdsmith/Sandboxen/testchrome' ________ running '/usr/bin/python src/build/util/lastchange.py --git-hash-only -s src/third_party/WebKit -o src/build/util/LASTCHANGE.blink' in '/Users/rdsmith/Sandboxen/testchrome' ________ running 'download_from_google_storage --no_resume --platform=win32 --no_auth --bucket chromium-gn -s src/buildtools/win/gn.exe.sha1' in '/Users/rdsmith/Sandboxen/testchrome' The current platform doesn't match "win32", skipping. ________ running 'download_from_google_storage --no_resume --platform=darwin --no_auth --bucket chromium-gn -s src/buildtools/mac/gn.sha1' in '/Users/rdsmith/Sandboxen/testchrome' 0> File src/buildtools/mac/gn exists and SHA1 matches. Skipping. Success! Downloading 1 files took 0.014553 second(s) ________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-gn -s src/buildtools/linux64/gn.sha1' in '/Users/rdsmith/Sandboxen/testchrome' The current platform doesn't match "linux*", skipping. ________ running 'download_from_google_storage --no_resume --platform=win32 --no_auth --bucket chromium-clang-format -s src/buildtools/win/clang-format.exe.sha1' in '/Users/rdsmith/Sandboxen/testchrome' The current platform doesn't match "win32", skipping. ________ running 'download_from_google_storage --no_resume --platform=darwin --no_auth --bucket chromium-clang-format -s src/buildtools/mac/clang-format.sha1' in '/Users/rdsmith/Sandboxen/testchrome' 0> File src/buildtools/mac/clang-format exists and SHA1 matches. Skipping. Success! Downloading 1 files took 0.012053 second(s) ________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-clang-format -s src/buildtools/linux64/clang-format.sha1' in '/Users/rdsmith/Sandboxen/testchrome' The current platform doesn't match "linux*", skipping. ________ running 'download_from_google_storage --no_resume --platform=darwin --no_auth --bucket chromium-libcpp -s src/third_party/libc++-static/libc++.a.sha1' in '/Users/rdsmith/Sandboxen/testchrome' 0> File src/third_party/libc++-static/libc++.a exists and SHA1 matches. Skipping. Success! Downloading 1 files took 0.019827 second(s) ________ running 'download_from_google_storage --no_resume --platform=win32 --no_auth --bucket chromium-luci -d src/tools/luci-go/win64' in '/Users/rdsmith/Sandboxen/testchrome' The current platform doesn't match "win32", skipping. ________ running 'download_from_google_storage --no_resume --platform=darwin --no_auth --bucket chromium-luci -d src/tools/luci-go/mac64' in '/Users/rdsmith/Sandboxen/testchrome' 0> File src/tools/luci-go/mac64/isolate exists and SHA1 matches. Skipping. Success! Downloading 1 files took 0.042021 second(s) ________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-luci -d src/tools/luci-go/linux64' in '/Users/rdsmith/Sandboxen/testchrome' The current platform doesn't match "linux*", skipping. ________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-eu-strip -s src/build/linux/bin/eu-strip.sha1' in '/Users/rdsmith/Sandboxen/testchrome' The current platform doesn't match "linux*", skipping. ________ running 'download_from_google_storage --no_resume --platform=win32 --no_auth --bucket chromium-drmemory -s src/third_party/drmemory/drmemory-windows-sfx.exe.sha1' in '/Users/rdsmith/Sandboxen/testchrome' The current platform doesn't match "win32", skipping. ________ running '/usr/bin/python src/build/get_syzygy_binaries.py --output-dir=src/third_party/syzygy/binaries --revision=1a6e67fecbd5363c425107b2dee4be8f23dbea35 --overwrite' in '/Users/rdsmith/Sandboxen/testchrome' INFO:get_syzygy_binaries.py:Output directory does not exist, skipping cleanup. ________ running '/usr/bin/python src/build/get_syzygy_binaries.py --output-dir=src/third_party/kasko/binaries --revision=266a18d9209be5ca5c5dcd0620942b82a2d238f3 --resource=kasko.zip --resource=kasko_symbols.zip --overwrite' in '/Users/rdsmith/Sandboxen/testchrome' INFO:get_syzygy_binaries.py:Unexpected output directory, skipping cleanup. ________ running 'download_from_google_storage --no_resume --platform=win32 --directory --recursive --no_auth --num_threads=16 --bucket chromium-apache-win32 src/third_party/apache-win32' in '/Users/rdsmith/Sandboxen/testchrome' The current platform doesn't match "win32", skipping. ________ running 'download_from_google_storage --no_resume --platform=linux* --extract --no_auth --bucket chromium-fonts -s src/third_party/blimp_fonts/font_bundle.tar.gz.sha1' in '/Users/rdsmith/Sandboxen/testchrome' The current platform doesn't match "linux*", skipping. ________ running '/usr/bin/python src/third_party/instrumented_libraries/scripts/download_binaries.py' in '/Users/rdsmith/Sandboxen/testchrome' ________ running '/usr/bin/python src/build/android/download_doclava.py' in '/Users/rdsmith/Sandboxen/testchrome' ________ running '/usr/bin/python src/build/gyp_chromium --running-as-hook' in '/Users/rdsmith/Sandboxen/testchrome' Updating projects from gyp files... Traceback (most recent call last): File "src/build/gyp_chromium", line 12, in <module> execfile(__file__ + '.py') File "src/build/gyp_chromium.py", line 357, in <module> sys.exit(main()) File "src/build/gyp_chromium.py", line 344, in main gyp_rc = gyp.main(args) File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/__init__.py", line 538, in main return gyp_main(args) File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/__init__.py", line 523, in gyp_main generator.GenerateOutput(flat_list, targets, data, params) File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/generator/xcode.py", line 1141, in GenerateOutput AddSourceToTarget(source, type, pbxp, xct) File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/generator/xcode.py", line 507, in AddSourceToTarget xct.SourcesPhase().AddFile(source) File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/xcodeproj_file.py", line 1855, in AddFile file_ref = file_group.AddOrGetFileByPath(path, hierarchical) File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/xcodeproj_file.py", line 1304, in AddOrGetFileByPath hierarchical) File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/xcodeproj_file.py", line 1304, in AddOrGetFileByPath hierarchical) File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/xcodeproj_file.py", line 1299, in AddOrGetFileByPath assert group_ref.__class__ == PBXGroup AssertionError Error: Command '/usr/bin/python src/build/gyp_chromium --running-as-hook' returned non-zero exit status 1 in /Users/rdsmith/Sandboxen/testchrome Hook '/usr/bin/python src/build/gyp_chromium --running-as-hook' took 126.81 secs rdsmith-macbookpro:../Sandboxen/testchrome $ Please use labels and text to provide additional information. Note that this happens with existing checkouts as well; I tried it with a new checkout just to rule out configuration problems.
,
Apr 27 2016
Are you using xcodebuild? Did you set the GYP_GENERATORS environment variable to xcode? Could you try running with: GYP_GENERATORS=ninja gclient sync I thought, xcode is not officially supported anymore by chromium and I couldn't find a single bot that tests it. Though, the error thrown might indicate that after the move of the v8.gyp file, there's still a duplicate build path somewhere. The version you fetched should include all fixes and there shouldn't be any reference to the old gyp file anymore. I need to find somebody with a mac to repro this.
,
Apr 27 2016
,
Apr 27 2016
Couldn't repro. With GYP_GENERATORS=ninja it passes and with GYP_GENERATORS=xcode it errors out differently with a key error in chrome/test/media_router/media_router_tests.gypi
,
Apr 27 2016
ok - we were able to repro with the hybrid GYP_GENERATORS="ninja,xcode-ninja" Looking more.
,
Apr 27 2016
Ah, thank you, I had forgotten about my GYP_GENERATORS environment variable. I don't know whether xcode is supported; I haven't used it myself in a while. Dropping the priority; at least personally, if I have a workaround that doesn't interfere with my development abilities, I'm good.
,
Apr 27 2016
Note also the related http://crbug.com/606547 which was fixed. I don't know why the fixes for the other bug didn't fix this too.
,
Apr 27 2016
xcode-ninja is supposed to work. machenbach, you can repro, yes?
,
Apr 27 2016
Yes I could repro with: GYP_GENERATORS="ninja,xcode-ninja" gclient runhooks Not my laptop unfortunately and I didn't manage to set up a new mac for this. I have also no idea why it fails like this. My intuition was: 1) Copy a gyp file in v8 from A to B. 2) Update references in embedder code (here: chromium,pdfium) from A to B. 3) Delete A in v8. After 1, http://crbug.com/606547 happened which was fixed by 2. If this is something similar, I have no idea where another reference could be. It should also fail differently when deleting the old file in v8, but it doesn't.
,
Apr 27 2016
Found a comp. Added a bit output to gyp where the error happens: printing next_dir and group_ref.__class__ .. <class 'gyp.xcodeproj_file.PBXGroup'> v8 <class 'gyp.xcodeproj_file.PBXGroup'> include <class 'gyp.xcodeproj_file.PBXGroup'> .. <class 'gyp.xcodeproj_file.PBXGroup'> v8 <class 'gyp.xcodeproj_file.PBXGroup'> .. <class 'gyp.xcodeproj_file.PBXGroup'> v8 <class 'gyp.xcodeproj_file.PBXGroup'> src <class 'gyp.xcodeproj_file.PBXFileReference'> I wonder if that's due to some change within the v8.gyp file (https://codereview.chromium.org/1920793002/diff/120001/src/v8.gyp). Looking more tomorrow.
,
Apr 27 2016
Fun fact: This also repro's in v8 stand-alone with: GYP_GENERATORS="xcode-ninja" gclient runhooks I'll go from there to analyze more.
,
Apr 27 2016
,
Apr 28 2016
Fix coming: https://codereview.chromium.org/1932703002/
,
Apr 28 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build.git/+/44767bb9b999d1e7089b5f61d9cb72569d94d4fc commit 44767bb9b999d1e7089b5f61d9cb72569d94d4fc Author: machenbach@chromium.org <machenbach@chromium.org> Date: Thu Apr 28 12:40:48 2016 V8: Turn xcode builder into xcode-ninja builder Xcode is not officially supported by chromium, while the hybrid xcode-ninja is. Therefore we should make sure that the latter doesn't break. BUG= 606898 Review-Url: https://codereview.chromium.org/1930883002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/build@300276 0039d316-1c4b-4281-b951-d872f2087c98 [modify] https://crrev.com/44767bb9b999d1e7089b5f61d9cb72569d94d4fc/scripts/slave/recipe_modules/v8/builders.py [modify] https://crrev.com/44767bb9b999d1e7089b5f61d9cb72569d94d4fc/scripts/slave/recipe_modules/v8/chromium_config.py [modify] https://crrev.com/44767bb9b999d1e7089b5f61d9cb72569d94d4fc/scripts/slave/recipes/v8.expected/full_client_v8_V8_Mac64___xcode.json
,
Apr 28 2016
The following revision refers to this bug: http://goto.ext.google.com/viewvc/chrome-internal?view=rev&revision=87154 ------------------------------------------------------------------ r87154 | recipe-roller@chromium.org | 2016-04-28T12:42:51.478641Z -----------------------------------------------------------------
,
Apr 28 2016
The following revision refers to this bug: https://chromium.googlesource.com/v8/v8.git/+/e7e7124c835e4600f1ca0d1b9518b6b1c74123bb commit e7e7124c835e4600f1ca0d1b9518b6b1c74123bb Author: machenbach <machenbach@chromium.org> Date: Thu Apr 28 12:43:26 2016 [build] Fix empty action inputs for xcode-ninja builds BUG= chromium:606898 LOG=n Review-Url: https://codereview.chromium.org/1932703002 Cr-Commit-Position: refs/heads/master@{#35860} [modify] https://crrev.com/e7e7124c835e4600f1ca0d1b9518b6b1c74123bb/src/v8.gyp
,
Apr 28 2016
I'll set this to fixed, once this has rolled into chromium.
,
Apr 28 2016
Thanks very much!
,
Apr 28 2016
Roll with fix: https://codereview.chromium.org/1930583003/
,
Apr 28 2016
,
Apr 28 2016
I still see this issue on iOS when I add dependency to mojo/mojo_edk.gyp:mojo_system_impl or services/shell/shell.gyp:shell_public. This is CL which triggers the problem: https://codereview.chromium.org/1906993003/patch/60001/70021 Should I file a separate bug or reopen this one? We trying to add Mojo support for iOS WebUI and this bug blocks our work.
,
Apr 28 2016
Is this already downstream and you still get the problem? https://chromium.googlesource.com/chromium/src/+/848a8bf749b03e58e82335cac14cfa0a2a8ada35
,
Apr 28 2016
My code is upstream, so I've tried runhooks upstream and still see the problem.
,
Apr 28 2016
eugenebut: It sounds like your CL triggers this maybe? Do you only get that with your patch, or without it too?
,
Apr 28 2016
Everything works well now, so I probably was doing something weird. Sorry for the trouble.
,
Apr 29 2016
Alright. If there should still be anything wrong, please reopen this ticket and provide detailed repro instructions. Cheers |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by rdsmith@chromium.org
, Apr 26 2016Components: Blink>JavaScript