New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 789085 link

Starred by 1 user

Issue metadata

Status: Duplicate
Owner: ----
Closed: Nov 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

-release: Archive/Unittest race: File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 668, in aux_get, raise KeyError(mycpv)

Project Member Reported by drinkcat@chromium.org, Nov 28 2017

Issue description

Seen on hana Archive stage: https://luci-milo.appspot.com/buildbot/chromeos/hana-release/1253

12:34:21: INFO: RunCommand: /b/c/cbuild/repository/chromite/bin/cros_sdk 'USE=-cros-debug chrome_internal' 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmpdxWhLh' -- ./mod_image_for_recovery.sh '--board=hana' '--image=/mnt/host/source/src/build/images/hana/R64-10166.0.0/tmpBd_h7I/chromiumos_base_image.bin' in /b/c/cbuild/repository
12:37:36: ERROR: 
return code: 1; command: /b/c/cbuild/repository/chromite/bin/cros_sdk 'USE=-cros-debug chrome_internal' 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmpdxWhLh' -- ./mod_image_for_recovery.sh '--board=hana' '--image=/mnt/host/source/src/build/images/hana/R64-10166.0.0/tmpBd_h7I/chromiumos_base_image.bin'
!!! CONFIG_PROTECT is empty for '/build/hana/'
Traceback (most recent call last):
  File "/usr/bin/emerge", line 51, in <module>
    retval = emerge_main()
  File "/usr/lib64/portage/pym/_emerge/main.py", line 1057, in emerge_main
    return run_action(emerge_config)
...
  File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 1097, in _hash_pkg
    cpv, ["COUNTER", "_mtime_"])
  File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 668, in aux_get
    raise KeyError(mycpv)
KeyError: u'chromeos-base/ci-results-archiver-0.0.1-r1'

It looks like UnitTest stage is running at the same time and re-installing that package:

Pending 32/57, Building 10/19, [Time 12:37:22 | Elapsed 2m57.7s | Load 86.81 49.41 24.15]
Started chromeos-base/bootstat-0.0.2-r2621 (logged in /tmp/bootstat-0.0.2-r2621-Jw6odL)
Started chromeos-base/chromeos-config-tools-0.0.2-r786 (logged in /tmp/chromeos-config-tools-0.0.2-r786-IlG19s)
Started chromeos-base/libcontainer-0.0.1-r1067 (logged in /tmp/libcontainer-0.0.1-r1067-_wwb8d)
Started chromeos-base/metrics-0.0.1-r2591 (logged in /tmp/metrics-0.0.1-r2591-2TFmCl)
Completed chromeos-base/ci-results-archiver-0.0.1-r1 (in 2m23.0s)
Still building autotest-0.0.2-r8884 (2m13.2s). Logs in /tmp/autotest-0.0.2-r8884-gRoPdd
Still building quipper-0.0.1-r2168 (2m5.4s). Logs in /tmp/quipper-0.0.1-r2168-t1qMkr
Still building chromite-0.0.2-r3549 (2m27.6s). Logs in /tmp/chromite-0.0.2-r3549-F5DTe5
Completed chromeos-base/chromeos-config-tools-0.0.2-r786 (in 0m39.1s)
Pending 30/57, Building 10/18, [Time 12:38:01 | Elapsed 3m36.8s | Load 54.75 46.0 24.08]

But also on https://luci-milo.appspot.com/buildbot/chromeos/nyan_blaze-release/1715
12:35:55: INFO: Done. Using 23 gconv modules. Removed 223 unused modules (4480.2 KiB) and 6 unused dependencies (836.0 KiB)
insert_container_publickey.sh: INFO   : Container verification key was installed. Do not forget to resign the image!
INFO    : The target platform does not use bootloader templates.
INFO    : Building dev-install package lists
Traceback (most recent call last):
  File "/usr/bin/emerge", line 51, in <module>
    retval = emerge_main()
  File "/usr/lib64/portage/pym/_emerge/main.py", line 1057, in emerge_main
    return run_action(emerge_config)
...
  File "/usr/lib64/portage/pym/_emerge/FakeVartree.py", line 276, in _pkg
    self._real_vardb.aux_get(cpv, self._db_keys)),
  File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 668, in aux_get
    raise KeyError(mycpv)
KeyError: u'chromeos-base/quipper-0.0.1-r2168'
ERROR   : `emerge-nyan_blaze virtual/target-os` failed

And Unittest is running at about the same time again, doing something with quipper:
Pending 0/53, Building 1/1, [Time 12:35:23 | Elapsed 15m52.8s | Load 9.62 41.22 41.41]
Completed chromeos-base/quipper-0.0.1-r2168 (in 14m39.4s)
Pending 0/53, [Time 12:36:01 | Elapsed 16m30.7s | Load 6.99 37.07 40.02]
 
This looks the same as https://bugs.chromium.org/p/chromium/issues/detail?id=789058, but with different symptoms...

Comment 2 by vapier@chromium.org, Nov 28 2017

Mergedinto: 789058
Status: Duplicate (was: Available)
i agree it's probably the same as issue 789058.  UnitTestStage is updating the vdb by installing packages, but the ArchiveStage is also updating the vdb by building recovery related packages.  i think the solution will be the same, so lets dupe it over.

Sign in to add a comment