New issue
Advanced search Search tips

Issue 606898 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: Bug



Sign in to add a comment

GYP error on checking out new chromium checkout

Project Member Reported by rdsmith@chromium.org, Apr 26 2016

Issue description

Version: 2d26a030a05d92fbb7489d62cfb7759b8f045335 (ToT on 4/26/2016)
OS: OS X El Capitan 10.11.4

What steps will reproduce the problem?
(1) In some directory with plenty of space, do "fetch chromium"
(2) When it errors with "gclient sync" returning non-zero exit status, re-execute gclient sync.

What is the expected output?

A long wait, and then a clean chromium checkout ready to build.

What do you see instead?

A failure in generating the XCode projects because of duplicate mksnapshot entries.  See below.


rdsmith-macbookpro:../Sandboxen/testchrome $ fetch chromium
Running: gclient root
Running: gclient config --spec 'solutions = [
  {
    "managed": False,
    "name": "src",
    "url": "https://chromium.googlesource.com/chromium/src.git",
    "custom_deps": {},
    "deps_file": ".DEPS.git",
    "safesync_url": "",
  },
]
'
Running: gclient sync
Error: Command '/usr/bin/python src/build/gyp_chromium --running-as-hook' returned non-zero exit status 1 in /Users/rdsmith/Sandboxen/testchrome
Traceback (most recent call last):
  File "/Users/rdsmith/Sandboxen/depot_tools/fetch.py", line 346, in <module>
    sys.exit(main())
  File "/Users/rdsmith/Sandboxen/depot_tools/fetch.py", line 341, in main
    return run(options, spec, root)
  File "/Users/rdsmith/Sandboxen/depot_tools/fetch.py", line 335, in run
    return checkout.init()
  File "/Users/rdsmith/Sandboxen/depot_tools/fetch.py", line 142, in init
    self.run_gclient(*sync_cmd)
  File "/Users/rdsmith/Sandboxen/depot_tools/fetch.py", line 76, in run_gclient
    return self.run(cmd_prefix + cmd, **kwargs)
  File "/Users/rdsmith/Sandboxen/depot_tools/fetch.py", line 66, in run
    return subprocess.check_output(cmd, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '('gclient', 'sync')' returned non-zero exit status 2
rdsmith-macbookpro:../Sandboxen/testchrome $ gclient sync
Syncing projects: 100% (64/64), done.

________ running '/usr/bin/python src/build/landmines.py' in '/Users/rdsmith/Sandboxen/testchrome'

________ running '/usr/bin/python src/tools/remove_stale_pyc_files.py src/android_webview/tools src/build/android src/gpu/gles2_conform_support src/infra src/ppapi src/printing src/third_party/catapult src/third_party/closure_compiler/build src/tools' in '/Users/rdsmith/Sandboxen/testchrome'

________ running '/usr/bin/python src/build/download_nacl_toolchains.py --mode nacl_core_sdk sync --extract' in '/Users/rdsmith/Sandboxen/testchrome'

________ running '/usr/bin/python src/build/android/play_services/update.py download' in '/Users/rdsmith/Sandboxen/testchrome'

________ running '/usr/bin/python src/build/linux/sysroot_scripts/install-sysroot.py --running-as-hook' in '/Users/rdsmith/Sandboxen/testchrome'

________ running '/usr/bin/python src/build/vs_toolchain.py update' in '/Users/rdsmith/Sandboxen/testchrome'

________ running '/usr/bin/python src/build/mac_toolchain.py' in '/Users/rdsmith/Sandboxen/testchrome'
Using local toolchain.

________ running '/usr/bin/python src/third_party/binutils/download.py' in '/Users/rdsmith/Sandboxen/testchrome'

________ running '/usr/bin/python src/tools/clang/scripts/update.py --if-needed' in '/Users/rdsmith/Sandboxen/testchrome'
Updating Clang to 267383-1...
Clang is already up to date.

________ running '/usr/bin/python src/build/util/lastchange.py -o src/build/util/LASTCHANGE' in '/Users/rdsmith/Sandboxen/testchrome'

________ running '/usr/bin/python src/build/util/lastchange.py --git-hash-only -s src/third_party/WebKit -o src/build/util/LASTCHANGE.blink' in '/Users/rdsmith/Sandboxen/testchrome'

________ running 'download_from_google_storage --no_resume --platform=win32 --no_auth --bucket chromium-gn -s src/buildtools/win/gn.exe.sha1' in '/Users/rdsmith/Sandboxen/testchrome'
The current platform doesn't match "win32", skipping.

________ running 'download_from_google_storage --no_resume --platform=darwin --no_auth --bucket chromium-gn -s src/buildtools/mac/gn.sha1' in '/Users/rdsmith/Sandboxen/testchrome'
0> File src/buildtools/mac/gn exists and SHA1 matches. Skipping.
Success!
Downloading 1 files took 0.014553 second(s)

________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-gn -s src/buildtools/linux64/gn.sha1' in '/Users/rdsmith/Sandboxen/testchrome'
The current platform doesn't match "linux*", skipping.

________ running 'download_from_google_storage --no_resume --platform=win32 --no_auth --bucket chromium-clang-format -s src/buildtools/win/clang-format.exe.sha1' in '/Users/rdsmith/Sandboxen/testchrome'
The current platform doesn't match "win32", skipping.

________ running 'download_from_google_storage --no_resume --platform=darwin --no_auth --bucket chromium-clang-format -s src/buildtools/mac/clang-format.sha1' in '/Users/rdsmith/Sandboxen/testchrome'
0> File src/buildtools/mac/clang-format exists and SHA1 matches. Skipping.
Success!
Downloading 1 files took 0.012053 second(s)

________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-clang-format -s src/buildtools/linux64/clang-format.sha1' in '/Users/rdsmith/Sandboxen/testchrome'
The current platform doesn't match "linux*", skipping.

________ running 'download_from_google_storage --no_resume --platform=darwin --no_auth --bucket chromium-libcpp -s src/third_party/libc++-static/libc++.a.sha1' in '/Users/rdsmith/Sandboxen/testchrome'
0> File src/third_party/libc++-static/libc++.a exists and SHA1 matches. Skipping.
Success!
Downloading 1 files took 0.019827 second(s)

________ running 'download_from_google_storage --no_resume --platform=win32 --no_auth --bucket chromium-luci -d src/tools/luci-go/win64' in '/Users/rdsmith/Sandboxen/testchrome'
The current platform doesn't match "win32", skipping.

________ running 'download_from_google_storage --no_resume --platform=darwin --no_auth --bucket chromium-luci -d src/tools/luci-go/mac64' in '/Users/rdsmith/Sandboxen/testchrome'
0> File src/tools/luci-go/mac64/isolate exists and SHA1 matches. Skipping.
Success!
Downloading 1 files took 0.042021 second(s)

________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-luci -d src/tools/luci-go/linux64' in '/Users/rdsmith/Sandboxen/testchrome'
The current platform doesn't match "linux*", skipping.

________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-eu-strip -s src/build/linux/bin/eu-strip.sha1' in '/Users/rdsmith/Sandboxen/testchrome'
The current platform doesn't match "linux*", skipping.

________ running 'download_from_google_storage --no_resume --platform=win32 --no_auth --bucket chromium-drmemory -s src/third_party/drmemory/drmemory-windows-sfx.exe.sha1' in '/Users/rdsmith/Sandboxen/testchrome'
The current platform doesn't match "win32", skipping.

________ running '/usr/bin/python src/build/get_syzygy_binaries.py --output-dir=src/third_party/syzygy/binaries --revision=1a6e67fecbd5363c425107b2dee4be8f23dbea35 --overwrite' in '/Users/rdsmith/Sandboxen/testchrome'
INFO:get_syzygy_binaries.py:Output directory does not exist, skipping cleanup.

________ running '/usr/bin/python src/build/get_syzygy_binaries.py --output-dir=src/third_party/kasko/binaries --revision=266a18d9209be5ca5c5dcd0620942b82a2d238f3 --resource=kasko.zip --resource=kasko_symbols.zip --overwrite' in '/Users/rdsmith/Sandboxen/testchrome'
INFO:get_syzygy_binaries.py:Unexpected output directory, skipping cleanup.

________ running 'download_from_google_storage --no_resume --platform=win32 --directory --recursive --no_auth --num_threads=16 --bucket chromium-apache-win32 src/third_party/apache-win32' in '/Users/rdsmith/Sandboxen/testchrome'
The current platform doesn't match "win32", skipping.

________ running 'download_from_google_storage --no_resume --platform=linux* --extract --no_auth --bucket chromium-fonts -s src/third_party/blimp_fonts/font_bundle.tar.gz.sha1' in '/Users/rdsmith/Sandboxen/testchrome'
The current platform doesn't match "linux*", skipping.

________ running '/usr/bin/python src/third_party/instrumented_libraries/scripts/download_binaries.py' in '/Users/rdsmith/Sandboxen/testchrome'

________ running '/usr/bin/python src/build/android/download_doclava.py' in '/Users/rdsmith/Sandboxen/testchrome'

________ running '/usr/bin/python src/build/gyp_chromium --running-as-hook' in '/Users/rdsmith/Sandboxen/testchrome'
Updating projects from gyp files...
Traceback (most recent call last):
  File "src/build/gyp_chromium", line 12, in <module>
    execfile(__file__ + '.py')
  File "src/build/gyp_chromium.py", line 357, in <module>
    sys.exit(main())
  File "src/build/gyp_chromium.py", line 344, in main
    gyp_rc = gyp.main(args)
  File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/__init__.py", line 538, in main
    return gyp_main(args)
  File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/__init__.py", line 523, in gyp_main
    generator.GenerateOutput(flat_list, targets, data, params)
  File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/generator/xcode.py", line 1141, in GenerateOutput
    AddSourceToTarget(source, type, pbxp, xct)
  File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/generator/xcode.py", line 507, in AddSourceToTarget
    xct.SourcesPhase().AddFile(source)
  File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/xcodeproj_file.py", line 1855, in AddFile
    file_ref = file_group.AddOrGetFileByPath(path, hierarchical)
  File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/xcodeproj_file.py", line 1304, in AddOrGetFileByPath
    hierarchical)
  File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/xcodeproj_file.py", line 1304, in AddOrGetFileByPath
    hierarchical)
  File "/Users/rdsmith/Sandboxen/testchrome/src/tools/gyp/pylib/gyp/xcodeproj_file.py", line 1299, in AddOrGetFileByPath
    assert group_ref.__class__ == PBXGroup
AssertionError
Error: Command '/usr/bin/python src/build/gyp_chromium --running-as-hook' returned non-zero exit status 1 in /Users/rdsmith/Sandboxen/testchrome
Hook '/usr/bin/python src/build/gyp_chromium --running-as-hook' took 126.81 secs
rdsmith-macbookpro:../Sandboxen/testchrome $ 


Please use labels and text to provide additional information.

Note that this happens with existing checkouts as well; I tried it with a new checkout just to rule out configuration problems.

 
Cc: jochen@chromium.org marja@chromium.org
Components: Blink>JavaScript
Bisect reveals this occurred due to an auto-roll of V8; see https://codereview.chromium.org/1915963002.  Jochen, Marja: You're the people in the OWNERS file I recognize; can either of you help me route this appropriately?  



Cc: -marja@chromium.org thakis@chromium.org
Are you using xcodebuild? Did you set the GYP_GENERATORS environment variable to xcode? Could you try running with:
GYP_GENERATORS=ninja gclient sync

I thought, xcode is not officially supported anymore by chromium and I couldn't find a single bot that tests it.

Though, the error thrown might indicate that after the move of the v8.gyp file, there's still a duplicate build path somewhere. The version you fetched should include all fixes and there shouldn't be any reference to the old gyp file anymore.

I need to find somebody with a mac to repro this.
Owner: machenb...@chromium.org
Status: Assigned (was: Untriaged)
Couldn't repro. With GYP_GENERATORS=ninja it passes and with GYP_GENERATORS=xcode it errors out differently with a key error in chrome/test/media_router/media_router_tests.gypi
ok - we were able to repro with the hybrid GYP_GENERATORS="ninja,xcode-ninja"

Looking more.

Labels: -Pri-1 Pri-2
Ah, thank you, I had forgotten about my GYP_GENERATORS environment variable.  I don't know whether xcode is supported; I haven't used it myself in a while.  

Dropping the priority; at least personally, if I have a workaround that doesn't interfere with my development abilities, I'm good.

Note also the related  http://crbug.com/606547  which was fixed. I don't know why the fixes for the other bug didn't fix this too.

Comment 8 by thakis@chromium.org, Apr 27 2016

xcode-ninja is supposed to work. machenbach, you can repro, yes?
Yes I could repro with:
GYP_GENERATORS="ninja,xcode-ninja" gclient runhooks

Not my laptop unfortunately and I didn't manage to set up a new mac for this.

I have also no idea why it fails like this. My intuition was:
1) Copy a gyp file in v8 from A to B.
2) Update references in embedder code (here: chromium,pdfium) from A to B.
3) Delete A in v8.

After 1,  http://crbug.com/606547  happened which was fixed by 2. If this is something similar, I have no idea where another reference could be. It should also fail differently when deleting the old file in v8, but it doesn't.
Found a comp. Added a bit output to gyp where the error happens:
printing next_dir and group_ref.__class__

  ..  <class 'gyp.xcodeproj_file.PBXGroup'>
  v8  <class 'gyp.xcodeproj_file.PBXGroup'>
  include  <class 'gyp.xcodeproj_file.PBXGroup'>
  ..  <class 'gyp.xcodeproj_file.PBXGroup'>
  v8  <class 'gyp.xcodeproj_file.PBXGroup'>
  ..  <class 'gyp.xcodeproj_file.PBXGroup'>
  v8  <class 'gyp.xcodeproj_file.PBXGroup'>
  src  <class 'gyp.xcodeproj_file.PBXFileReference'> 

I wonder if that's due to some change within the v8.gyp file (https://codereview.chromium.org/1920793002/diff/120001/src/v8.gyp). Looking more tomorrow.
Fun fact: This also repro's in v8 stand-alone with:
GYP_GENERATORS="xcode-ninja" gclient runhooks

I'll go from there to analyze more.
Cc: eugene...@chromium.org
Project Member

Comment 14 by bugdroid1@chromium.org, Apr 28 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/44767bb9b999d1e7089b5f61d9cb72569d94d4fc

commit 44767bb9b999d1e7089b5f61d9cb72569d94d4fc
Author: machenbach@chromium.org <machenbach@chromium.org>
Date: Thu Apr 28 12:40:48 2016

V8: Turn xcode builder into xcode-ninja builder

Xcode is not officially supported by chromium, while the
hybrid xcode-ninja is. Therefore we should make sure that
the latter doesn't break.

BUG= 606898 

Review-Url: https://codereview.chromium.org/1930883002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/build@300276 0039d316-1c4b-4281-b951-d872f2087c98

[modify] https://crrev.com/44767bb9b999d1e7089b5f61d9cb72569d94d4fc/scripts/slave/recipe_modules/v8/builders.py
[modify] https://crrev.com/44767bb9b999d1e7089b5f61d9cb72569d94d4fc/scripts/slave/recipe_modules/v8/chromium_config.py
[modify] https://crrev.com/44767bb9b999d1e7089b5f61d9cb72569d94d4fc/scripts/slave/recipes/v8.expected/full_client_v8_V8_Mac64___xcode.json

Project Member

Comment 15 by bugdroid1@chromium.org, Apr 28 2016

The following revision refers to this bug:
  http://goto.ext.google.com/viewvc/chrome-internal?view=rev&revision=87154

------------------------------------------------------------------
r87154 | recipe-roller@chromium.org | 2016-04-28T12:42:51.478641Z

-----------------------------------------------------------------
Project Member

Comment 16 by bugdroid1@chromium.org, Apr 28 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/e7e7124c835e4600f1ca0d1b9518b6b1c74123bb

commit e7e7124c835e4600f1ca0d1b9518b6b1c74123bb
Author: machenbach <machenbach@chromium.org>
Date: Thu Apr 28 12:43:26 2016

[build] Fix empty action inputs for xcode-ninja builds

BUG= chromium:606898 
LOG=n

Review-Url: https://codereview.chromium.org/1932703002
Cr-Commit-Position: refs/heads/master@{#35860}

[modify] https://crrev.com/e7e7124c835e4600f1ca0d1b9518b6b1c74123bb/src/v8.gyp

I'll set this to fixed, once this has rolled into chromium.
Thanks very much!
Status: Fixed (was: Assigned)
I still see this issue on iOS when I add dependency to mojo/mojo_edk.gyp:mojo_system_impl or services/shell/shell.gyp:shell_public.


This is CL which triggers the problem:
https://codereview.chromium.org/1906993003/patch/60001/70021


Should I file a separate bug or reopen this one? We trying to add Mojo support for iOS WebUI and this bug blocks our work.
Is this already downstream and you still get the problem?
https://chromium.googlesource.com/chromium/src/+/848a8bf749b03e58e82335cac14cfa0a2a8ada35
My code is upstream, so I've tried runhooks upstream and still see the problem.
eugenebut: It sounds like your CL triggers this maybe? Do you only get that with your patch, or without it too?
Everything works well now, so I probably was doing something weird. Sorry for the trouble.
Alright. If there should still be anything wrong, please reopen this ticket and provide detailed repro instructions. Cheers

Sign in to add a comment