intermittent failures due to Permission denied: '/proc/sys/fs/binfmt_misc/qemu-arm' |
||||||||||||
Issue descriptionimageloader-0.0.1-r1250: >>> Emerging (1 of 1) chromeos-base/imageloader-0.0.1-r1250::chromiumos for /build/arm-generic/ imageloader-0.0.1-r1250: * Running stacked hooks for pre_pkg_setup imageloader-0.0.1-r1250: * sysroot_build_bin_dir ... imageloader-0.0.1-r1250: [ ok ] imageloader-0.0.1-r1250: * Running stacked hooks for post_pkg_setup imageloader-0.0.1-r1250: * python_eclass_hack ... imageloader-0.0.1-r1250: [ ok ] imageloader-0.0.1-r1250: * Running stacked hooks for pre_src_unpack imageloader-0.0.1-r1250: * python_multilib_setup ... imageloader-0.0.1-r1250: [ ok ] imageloader-0.0.1-r1250: >>> Unpacking source... imageloader-0.0.1-r1250: * /mnt/host/source/src/platform2 is not at rev 88481efde4161ad9e80bfc44769804208c64da06 imageloader-0.0.1-r1250: Cloning into '/build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1'... imageloader-0.0.1-r1250: done. imageloader-0.0.1-r1250: >>> Source unpacked in /build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work imageloader-0.0.1-r1250: * Running stacked hooks for post_src_unpack imageloader-0.0.1-r1250: * asan_init ... imageloader-0.0.1-r1250: [ ok ] imageloader-0.0.1-r1250: >>> Preparing source in /build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1/imageloader ... imageloader-0.0.1-r1250: >>> Source prepared. imageloader-0.0.1-r1250: >>> Configuring source in /build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1/imageloader ... imageloader-0.0.1-r1250: /build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1/common-mk/platform2.py --libdir=/usr/lib --use_flags=arm cros-debug elibc_glibc kernel_linux test userland_GNU --jobs=32 --action=configure --cache_dir=/build/arm-generic/var/cache/portage/chromeos-base/imageloader /build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1/imageloader/imageloader.gyp imageloader-0.0.1-r1250: >>> Source configured. imageloader-0.0.1-r1250: >>> Compiling source in /build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1/imageloader ... imageloader-0.0.1-r1250: /build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1/common-mk/platform2.py --libdir=/usr/lib --use_flags=arm cros-debug elibc_glibc kernel_linux test userland_GNU --jobs=32 --action=compile --cache_dir=/build/arm-generic/var/cache/portage/chromeos-base/imageloader all imageloader-0.0.1-r1250: >>> Source compiled. imageloader-0.0.1-r1250: >>> Test phase: chromeos-base/imageloader-0.0.1-r1250 imageloader-0.0.1-r1250: /build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1/common-mk/platform2_test.py --action=pre_test --sysroot=/build/arm-generic -- imageloader-0.0.1-r1250: ERROR: ld.so: object 'libsandbox.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. imageloader-0.0.1-r1250: /build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1/common-mk/platform2_test.py --action=run --sysroot=/build/arm-generic -- /build/arm-generic/var/cache/portage/chromeos-base/imageloader/out/Default/run_tests imageloader-0.0.1-r1250: ERROR: ld.so: object 'libsandbox.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. imageloader-0.0.1-r1250: Traceback (most recent call last): imageloader-0.0.1-r1250: File "/build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1/common-mk/platform2_test.py", line 520, in <module> imageloader-0.0.1-r1250: main(sys.argv[1:]) imageloader-0.0.1-r1250: File "/build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1/common-mk/platform2_test.py", line 516, in main imageloader-0.0.1-r1250: getattr(p2test, options.action)() imageloader-0.0.1-r1250: File "/build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1/common-mk/platform2_test.py", line 294, in run imageloader-0.0.1-r1250: self.pre_test() imageloader-0.0.1-r1250: File "/build/arm-generic/tmp/portage/chromeos-base/imageloader-0.0.1-r1250/work/imageloader-0.0.1/common-mk/platform2_test.py", line 284, in pre_test imageloader-0.0.1-r1250: self.qemu.RegisterBinfmt() imageloader-0.0.1-r1250: File "/usr/lib64/python2.7/site-packages/chromite/lib/qemu.py", line 295, in RegisterBinfmt imageloader-0.0.1-r1250: osutils.WriteFile(self.binfmt_path, '-1') imageloader-0.0.1-r1250: File "/usr/lib64/python2.7/site-packages/chromite/lib/osutils.py", line 130, in WriteFile imageloader-0.0.1-r1250: with open(write_path, mode) as f: imageloader-0.0.1-r1250: IOError: [Errno 13] Permission denied: '/proc/sys/fs/binfmt_misc/qemu-arm' There have not been any recent changes to this package, last ebuild uprev was about one week ago. Let's see if this paladin cycles green.
,
Mar 22 2018
,
Mar 22 2018
If you go through the logs, there's a similar failure in the libcontainer package: libcontainer-0.0.1-r1274: IOError: [Errno 13] Permission denied: '/proc/sys/fs/binfmt_misc/qemu-arm'
,
Mar 23 2018
Sheriff, can you please find an owner for this?
,
Mar 23 2018
The builder cycled green and I don't have a good idea about the root cause.
,
Apr 4 2018
Again in
,
Apr 4 2018
https://luci-milo.appspot.com/buildbot/chromeos/veyron_tiger-paladin/2906
,
Apr 4 2018
,
Apr 4 2018
,
Apr 9 2018
Didn't make too much headway on this. I believe the builders are on kernel 3.13, so I looked upstream to see if we were missing patches or if there were known binfmt issues. I didn't dig super deep, but didn't uncover any obvious culprits. Other things that might be worth checking: * any new chromite changes surrounding binfmt_misc * uprevs of qemu or other packages Moving to this week's sheriff. Maybe worth a look from someone on the CI team as well.
,
Apr 9 2018
I logged in via SSH and poked around and didn't see anything immediately obvious. Both of the bots that this failed on were kernel 3.13 on Trusty which is very old at this point. They will get upgraded by virtue of the transition to Swarming so perhaps we can just wait to see if this exists after that. They had both recently been rebooted.
,
Apr 23 2018
the binfmt/qemu logic in question was written long enough ago that i wouldn't worry about this being a "is the kernel new enough" the code that sets up the binfmt files are also not related to qemu (the package) at all. it's entirely written by common-mk/platform2_test.py files i'm wondering if there's a race here somewhere. we tried to write the code to DTRT at each step, but maybe something changed with newer versions that violate our assumptions.
,
May 23 2018
,
May 24 2018
I think I see a bug in the logic. I'll post a CL and at the very least, add some debug output in case it goes wrong again.
,
Jun 6 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1a05b76cf1e92a2bdec4db65b9a675cf8df9c568 commit 1a05b76cf1e92a2bdec4db65b9a675cf8df9c568 Author: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Wed Jun 06 01:15:54 2018 Roll src/third_party/chromite 3c46387..784bb7f (5 commits) https://chromium.googlesource.com/chromiumos/chromite.git/+log/3c46387..784bb7f git log 3c46387..784bb7f --date=short --no-merges --format='%ad %ae %s' 2018-06-05 vapier@chromium.org cros lint: rework extension/linter lookups 2018-06-05 bmgordon@chromium.org cros_build_lib: Catch additional gethostbyaddr exception 2018-06-05 bpastene@chromium.org chromite: Roll third_party/gn_helpers forward. 2018-06-05 vapier@chromium.org qemu: fix bug in registration logic 2018-06-05 vapier@chromium.org qemu: add more logging around registration Created with: gclient setdep -r src/third_party/chromite@784bb7f The AutoRoll server is located here: https://chromite-chromium-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. BUG=chromium:None,chromium:None,chromium:824808,chromium:824808 TBR=chrome-os-gardeners@chromium.org Change-Id: I122486e1710bce26bd16709b9d5c6d401cdb9f38 Reviewed-on: https://chromium-review.googlesource.com/1087637 Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#564757} [modify] https://crrev.com/1a05b76cf1e92a2bdec4db65b9a675cf8df9c568/DEPS
,
Jun 6 2018
not sure why bugdroid didn't comment, but these two CLs should fix the issue i think: https://chromium-review.googlesource.com/1085859 https://chromium-review.googlesource.com/1085860 |
||||||||||||
►
Sign in to add a comment |
||||||||||||
Comment 1 by sha...@chromium.org
, Mar 22 2018