New issue
Advanced search Search tips

Issue 763979 link

Starred by 1 user

Issue metadata

Status: Untriaged
Owner: ----
Cc:
EstimatedDays: ----
NextAction: 2019-07-09
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

nyan_kitty-paladin failed at the buildPackage stage due to gsutil no URLs match error

Project Member Reported by shuqianz@chromium.org, Sep 11 2017

Issue description

https://uberchromegw.corp.google.com/i/chromeos/builders/nyan_kitty-paladin/builds/2891/steps/BuildPackages/logs/stdio

Completed chromeos-base/autotest-all-0.0.1-r39 (in 0m4.6s)
Pending 0/617, [Time 11:43:21 | Elapsed 7m31.4s | Load 1.95 8.25 7.15]
Merge complete
Done
Builds complete
INFO    : fetching the debug symbols
cros_install_debug_syms: Unhandled exception:
Traceback (most recent call last):
  File "/mnt/host/source/chromite/bin/cros_install_debug_syms", line 168, in <module>
    DoMain()
  File "/mnt/host/source/chromite/bin/cros_install_debug_syms", line 164, in DoMain
    commandline.ScriptWrapperMain(FindTarget)
  File "/mnt/host/source/chromite/lib/commandline.py", line 910, in ScriptWrapperMain
    ret = target(argv[1:])
  File "/mnt/host/source/chromite/scripts/cros_install_debug_syms.py", line 322, in main
    processes=options.jobs)
  File "/mnt/host/source/chromite/lib/parallel.py", line 809, in RunTasksInProcessPool
    queue.put((idx, input_args))
  File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/mnt/host/source/chromite/lib/parallel.py", line 750, in BackgroundTaskRunner
    queue.put(_AllTasksComplete())
  File "/mnt/host/source/chromite/lib/parallel.py", line 750, in BackgroundTaskRunner
    queue.put(_AllTasksComplete())
  File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/mnt/host/source/chromite/lib/parallel.py", line 561, in ParallelTasks
    raise BackgroundFailure(exc_infos=errors)
chromite.lib.parallel.BackgroundFailure: <class 'chromite.lib.gs.GSNoSuchKey'>: return code: 1; command: /mnt/host/source/.cache/common/gsutil_4.27.tar.gz/gsutil/gsutil -o 'Boto:num_retries=10' cp -v -- gs://chromeos-prebuilt/board/nyan_kitty/paladin-R63-9920.0.0-rc3 /build/nyan_kitty/packages/chromeos-base/audiotest-0.0.1-r32.debug.tbz2
Omitting prefix "gs://chromeos-prebuilt/board/nyan_kitty/paladin-R63-9920.0.0-rc3/". (Did you mean to do cp -r?)
CommandException: No URLs matched

cmd=['/mnt/host/source/.cache/common/gsutil_4.27.tar.gz/gsutil/gsutil', '-o', 'Boto:num_retries=10', 'cp', '-v', '--', 'gs://chromeos-prebuilt/board/nyan_kitty/paladin-R63-9920.0.0-rc3', u'/build/nyan_kitty/packages/chromeos-base/audiotest-0.0.1-r32.debug.tbz2'], extra env={'BOTO_CONFIG': u'/mnt/host/source/src/private-overlays/chromeos-overlay/googlestorage_account.boto'}
Traceback (most recent call last):
  File "/mnt/host/source/chromite/lib/parallel.py", line 602, in TaskRunner
    task(*x, **task_kwargs)
  File "/mnt/host/source/chromite/lib/parallel.py", line 800, in <lambda>
    fn = lambda idx, task_args: out_queue.put((idx, task(*task_args)))
  File "/mnt/host/source/chromite/scripts/cros_install_debug_syms.py", line 94, in Install
    self._gs_context.Copy(url, archive, debug_level=logging.DEBUG)
  File "/mnt/host/source/chromite/lib/gs.py", line 870, in Copy
    raise
  File "/mnt/host/source/chromite/lib/gs.py", line 847, in Copy
    result = self.DoCommand(cmd, **kwargs)
  File "/mnt/host/source/chromite/lib/gs.py", line 781, in DoCommand
    extra_env=extra_env, **kwargs)
  File "/mnt/host/source/chromite/lib/retry_stats.py", line 180, in RetryWithStats
    *args, **kwargs)
  File "/mnt/host/source/chromite/lib/retry_util.py", line 243, in GenericRetry
    return _run()
  File "/mnt/host/source/chromite/lib/retry_util.py", line 200, in _Wrapper
    if not self._handler(e):
  File "/mnt/host/source/chromite/lib/gs.py", line 661, in _RetryFilter
    raise GSNoSuchKey(e)
GSNoSuchKey: return code: 1; command: /mnt/host/source/.cache/common/gsutil_4.27.tar.gz/gsutil/gsutil -o 'Boto:num_retries=10' cp -v -- gs://chromeos-prebuilt/board/nyan_kitty/paladin-R63-9920.0.0-rc3 /build/nyan_kitty/packages/chromeos-base/audiotest-0.0.1-r32.debug.tbz2
Omitting prefix "gs://chromeos-prebuilt/board/nyan_kitty/paladin-R63-9920.0.0-rc3/". (Did you mean to do cp -r?)
CommandException: No URLs matched

cmd=['/mnt/host/source/.cache/common/gsutil_4.27.tar.gz/gsutil/gsutil', '-o', 'Boto:num_retries=10', 'cp', '-v', '--', 'gs://chromeos-prebuilt/board/nyan_kitty/paladin-R63-9920.0.0-rc3', u'/build/nyan_kitty/packages/chromeos-base/audiotest-0.0.1-r32.debug.tbz2'], extra env={'BOTO_CONFIG': u'/mnt/host/source/src/private-overlays/chromeos-overlay/googlestorage_account.boto'}

ERROR   : Sun Sep 10 11:47:46 PDT 2017
ERROR   :  PGID  PPID   PID     ELAPSED     TIME %CPU COMMAND
ERROR   : Arguments of 10: ./build_packages '--board=nyan_kitty' '--accept_licenses=@CHROMEOS' '--withdebugsymbols' '--skip_chroot_upgrade' '--withevents' '--eventfile=/mnt/host/source/buildbot_archive/nyan_kitty-paladin/R63-9929.0.0-rc1/build-events.json' 'virtual/target-os' 'virtual/target-os-dev' 'virtual/target-os-test' 'virtual/target-os-factory' 'virtual/target-os-factory-shim' 'chromeos-base/autotest-all'
ERROR   : Backtrace:  (most recent call is last)
ERROR   :  build_packages:338:main(), called: die_err_trap  
ERROR   : 
ERROR   : Command failed:
ERROR   :   Command 'sudo -E "${GCLIENT_ROOT}/chromite/bin/cros_install_debug_syms" "--board=${FLAGS_board}" "--all"' exited with nonzero code: 1
11:47:46: ERROR: 
return code: 1; command: /b/c/cbuild/repository/chromite/bin/cros_sdk 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmpcd5Mfg' 'USE=chrome_internal' 'FEATURES=separatedebug' -- ./build_packages '--board=nyan_kitty' '--accept_licenses=@CHROMEOS' --withdebugsymbols --skip_chroot_upgrade --withevents '--eventfile=/mnt/host/source/buildbot_archive/nyan_kitty-paladin/R63-9929.0.0-rc1/build-events.json' virtual/target-os virtual/target-os-dev virtual/target-os-test virtual/target-os-factory virtual/target-os-factory-shim chromeos-base/autotest-all
cmd=['/b/c/cbuild/repository/chromite/bin/cros_sdk', 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmpcd5Mfg', u'USE=chrome_internal', 'FEATURES=separatedebug', '--', './build_packages', u'--board=nyan_kitty', '--accept_licenses=@CHROMEOS', '--withdebugsymbols', '--skip_chroot_upgrade', '--withevents', u'--eventfile=/mnt/host/source/buildbot_archive/nyan_kitty-paladin/R63-9929.0.0-rc1/build-events.json', 'virtual/target-os', 'virtual/target-os-dev', 'virtual/target-os-test', 'virtual/target-os-factory', 'virtual/target-os-factory-shim', 'chromeos-base/autotest-all'], cwd=/b/c/cbuild/repository, extra env={'PARALLEL_EMERGE_STATUS_FILE': '/tmp/tmpcd5Mfg', 'USE': u'chrome_internal', 'FEATURES': 'separatedebug'}

@@@STEP_FAILURE@@@
11:47:46: ERROR: ./build_packages failed (code=1)
11:47:46: INFO: Translating result ./build_packages failed (code=1) to fail.
11:47:46: INFO: Running cidb query on pid 26739, repr(query) starts with <sqlalchemy.sql.expression.Update object at 0x7ff9fc207290>
11:47:46: INFO: Running cidb query on pid 26739, repr(query) starts with <sqlalchemy.sql.expression.Insert object at 0x7ff9fc23f950>
************************************************************
** Finished Stage BuildPackages - Sun, 10 Sep 2017 11:47:46 -0700 (PDT)
************************************************************
11:47:47: INFO: Running cidb query on pid 26739, repr(query) starts with <sqlalchemy.sql.expression.Insert object at 0x7ff9fc207390>



I ran the gsutil command to manually check the prebuilt.
$ gsutil ls gs://chromeos-prebuilt/board/nyan_kitty/paladin-R63-9920.0.0-rc3
gs://chromeos-prebuilt/board/nyan_kitty/paladin-R63-9920.0.0-rc3/packages/

The image is there, it seems that the build/nyan_kitty/packages/chromeos-base/ is missing?

 
With later builds (#2905 onward) this problem was no show, though? 
Cc: -seobrien@chromium.org
Labels: Pri-3
NextAction: 2019-07-09
Downgrading P2s that haven't been modified in more than 6 months, which have no component or owner.

Sign in to add a comment