security_SandboxLinuxUnittests failing on 32 bit pfq-informational / pfq builders |
||||||||||
Issue description
,
Apr 25 2016
Odd. This failed @ 34a7a8cbbf500710c5867a5ecb7cdc6ba369e2cb, which was 389360, and seems super unlikely to be the culprit. I forgot to run at 389359 (the last known success) to confirm, but I think this may be a chromeos issue. +chromeos sheriffs.
,
Apr 25 2016
The failure should be one of the CLs here: https://uberchromegw.corp.google.com/i/chromiumos.chromium/builders/x86-generic-tot-chromium-pfq-informational/builds/8326
,
Apr 25 2016
Can't figure it out from eye-balling the CLs. I think hshi@ should be able to narrow this down? Adding Jorge in case he has some ideas.
,
Apr 25 2016
re:#3 the link only shows chromium CLs... while stevenjb mentioned in #2 that it may be a chromeos issue.
,
Apr 25 2016
Pay attention to the detailed error about libbase.so: 04:01:38 INFO | autoserv| FAIL security_SandboxLinuxUnittests security_SandboxLinuxUnittests timestamp=1461402097 localtime=Apr 23 04:01:37 sandbox_linux_unittests failed! Command <su chronos -c 'HOME=/tmp/tmpmAkum7 CR_SOURCE_ROOT=/usr/local/autotest/deps/chrome_test/test_src CHROME_DEVEL_SANDBOX=/opt/google/chrome/chrome-sandbox /usr/local/autotest/deps/chrome_test/test_src/out/Release/sandbox_linux_unittests '> failed, rc=127, Command returned non-zero exit status 04:01:38 INFO | autoserv| AUTOTEST_STATUS:: * Command: 04:01:38 INFO | autoserv| AUTOTEST_STATUS:: su chronos -c 'HOME=/tmp/tmpmAkum7 04:01:38 INFO | autoserv| AUTOTEST_STATUS:: CR_SOURCE_ROOT=/usr/local/autotest/deps/chrome_test/test_src 04:01:38 INFO | autoserv| AUTOTEST_STATUS:: CHROME_DEVEL_SANDBOX=/opt/google/chrome/chrome-sandbox /usr/local/autotest 04:01:38 INFO | autoserv| AUTOTEST_STATUS:: /deps/chrome_test/test_src/out/Release/sandbox_linux_unittests ' 04:01:38 INFO | autoserv| AUTOTEST_STATUS:: Exit status: 127 04:01:38 INFO | autoserv| AUTOTEST_STATUS:: Duration: 0.0221660137177 04:01:38 INFO | autoserv| AUTOTEST_STATUS:: 04:01:38 INFO | autoserv| AUTOTEST_STATUS:: stderr: 04:01:38 INFO | autoserv| AUTOTEST_STATUS:: /usr/local/autotest/deps/chrome_test/test_src/out/Release/sandbox_linux_unittests: error while loading shared libraries: libbase.so: cannot open shared object file: No such file or directory 04:01:38 INFO | autoserv| AUTOTEST_STATUS:: END FAIL security_SandboxLinuxUnittests security_SandboxLinuxUnittests timestamp=1461402097 localtime=Apr 23 02:01:37 "error while loading shared libraries: libbase.so: cannot open shared object file: No such file or directory" does not seem likely to be caused by a chromium CL
,
Apr 25 2016
You can also see the 3 chromeos CLs in that build. This build is the first breaking one, so either the 3 chromeos CLs, or the 7 chrome CLs are responsible. Here are the chromeos CLs: https://chromium-review.googlesource.com/#/c/340470/ https://chromium-review.googlesource.com/#/c/338212/ https://chromium-review.googlesource.com/#/c/340480/
,
Apr 25 2016
Ilja's CL looks suspicious? https://chromium-review.googlesource.com/#/c/340470/
,
Apr 25 2016
+ ihf (on https://chromium-review.googlesource.com/#/c/340470/) as the most likely culprit The following 2 chromeos CLs are unlikely to cause a test to fail to locate libbase.so: https://chromium-review.googlesource.com/#/c/338212/ "Add Samsung K4E6E304EB 4G LPDDR3 SDRAM for elm-rev1 SKU2" https://chromium-review.googlesource.com/#/c/340480/ " drm/i915: fix kernel WARN in i915_gem_obj_to_ggtt"
,
Apr 25 2016
I agree with c#9.
,
Apr 25 2016
I just got a cbuildbot failure on a version of chrome that has passed, so almost definitely a chromeos failure: https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/chromium-pfq-informational/builds/43
,
Apr 25 2016
I switched to component build on 32 bit Chrome OS. Which means there is a libbase.so now and it is in /opt/google/chrome/lib/libbase.so. But only on 32 bit. I assume sandbox_linux_unittests is a component build now as well. And of course it got separated from the lib directory. One should point it to /opt/google/chrome/lib/ some way, for instance by using LD_PRELOAD or creating a symbolic link?
,
Apr 25 2016
,
Apr 25 2016
Confirmed that this is occurring on 32 bit (x86- and ARM) builders.
,
Apr 25 2016
Do we have a plan here, or should we consider reverting the chromeos change or disabling the test?
,
Apr 25 2016
We have a plan. But building still takes time. I'd say disabling the affected tests is an option.
,
Apr 25 2016
Given that it is a sandbox test I am fine with waiting for a pending fix if we think we can have it in by tomorrow. I am reluctant to disable a sandbox test even if the current failure is not a security issue.
,
Apr 26 2016
Let's try not to disable this test since there's a small chance it stays disabled for a while. This particular test covers a large surface area.
,
Apr 26 2016
On the positive side my build finished and I have a repro.
,
Apr 26 2016
Adding the link on the DUT indeed fixes the test ln -s /opt/google/chrome/lib /usr/local/autotest/deps/chrome_test/test_src/out/Release/lib
,
Apr 26 2016
As a side, I just noticed that component build saves 11M. Not on the image, but at least during testing. Before component build: -rwxr-xr-x 1 root root 12M Mar 23 12:37 media_unittests -rwxr-xr-x 1 root root 4.0M Mar 23 12:37 video_decode_accelerator_unittest -rwxr-xr-x 1 root root 1.8M Mar 23 12:37 sandbox_linux_unittests -rwxr-xr-x 1 root root 1.7M Mar 23 12:37 video_encode_accelerator_unittest With component build: -rwxr-xr-x 1 root root 6.6M Apr 21 18:00 media_unittests -rwxr-xr-x 1 root root 1.2M Apr 21 18:00 sandbox_linux_unittests -rwxr-xr-x 1 root root 213K Apr 21 18:00 video_decode_accelerator_unittest -rwxr-xr-x 1 root root 193K Apr 21 18:00 video_encode_accelerator_unittest
,
Apr 26 2016
,
Apr 26 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/be44d6e917fddb73b8fdf6056027164fcc68ba62 commit be44d6e917fddb73b8fdf6056027164fcc68ba62 Author: Ilja H. Friedel <ihf@chromium.org> Date: Tue Apr 26 00:54:11 2016 chrome_binary_test: add link for component build. On 32 bit systems we use chrome with component build now. That means the *unittest binaries are smaller, but can't find their dependency. Use the setup in chrome_binary_test to create a symlink to /opt/google/chrome/lib if needed. Notice this requires a binary match between the installed chrome and unittests. Emit a reminder to the user that mixing and matching isn't meaningful anymore. Also minor cleanup. TEST=test_that $DUT security_SandboxLinuxUnittests ideo_VideoDecodeAccelerator.h264 video_VideoEncodeAccelerator.h264 BUG= chromium:606373 Change-Id: I297a2b93bdb7e3443b40f459f15270511f505da0 Reviewed-on: https://chromium-review.googlesource.com/340661 Trybot-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Haixia Shi <hshi@chromium.org> [modify] https://crrev.com/be44d6e917fddb73b8fdf6056027164fcc68ba62/client/cros/chrome_binary_test.py
,
Apr 26 2016
Issue 606639 has been merged into this issue.
,
Apr 26 2016
Issue 606126 has been merged into this issue. Issue 606129 has been merged into this issue. Issue 606130 has been merged into this issue. Issue 606131 has been merged into this issue. Issue 606139 has been merged into this issue. Issue 606145 has been merged into this issue. Issue 606151 has been merged into this issue. Issue 606161 has been merged into this issue. Issue 606166 has been merged into this issue. Issue 606170 has been merged into this issue. Issue 606171 has been merged into this issue. Issue 606174 has been merged into this issue. Issue 606176 has been merged into this issue. Issue 606179 has been merged into this issue. Issue 606180 has been merged into this issue. Issue 606183 has been merged into this issue. Issue 606188 has been merged into this issue. Issue 606195 has been merged into this issue. Issue 606203 has been merged into this issue. Issue 606212 has been merged into this issue. Issue 606220 has been merged into this issue. Issue 606223 has been merged into this issue. Issue 606224 has been merged into this issue. Issue 606227 has been merged into this issue. Issue 606253 has been merged into this issue. Issue 606256 has been merged into this issue. Issue 606257 has been merged into this issue. Issue 606298 has been merged into this issue. Issue 606332 has been merged into this issue. Issue 606369 has been merged into this issue. Issue 606433 has been merged into this issue. Issue 606438 has been merged into this issue. Issue 606490 has been merged into this issue. Issue 606551 has been merged into this issue. Issue 606591 has been merged into this issue. Issue 606623 has been merged into this issue. Issue 606628 has been merged into this issue.
,
Apr 26 2016
The second builder on the original report is green again: https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/x86-alex-tot-chrome-pfq-informational/builds/19822
,
Apr 27 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/43ac0331a240c9cdefee549a625b41f43f60fd83 commit 43ac0331a240c9cdefee549a625b41f43f60fd83 Author: Ilja H. Friedel <ihf@chromium.org> Date: Tue Apr 26 00:54:11 2016 chrome_binary_test: add link for component build. On 32 bit systems we use chrome with component build now. That means the *unittest binaries are smaller, but can't find their dependency. Use the setup in chrome_binary_test to create a symlink to /opt/google/chrome/lib if needed. Notice this requires a binary match between the installed chrome and unittests. Emit a reminder to the user that mixing and matching isn't meaningful anymore. Also minor cleanup. TEST=test_that $DUT security_SandboxLinuxUnittests ideo_VideoDecodeAccelerator.h264 video_VideoEncodeAccelerator.h264 BUG= chromium:606373 Change-Id: I297a2b93bdb7e3443b40f459f15270511f505da0 Reviewed-on: https://chromium-review.googlesource.com/340894 Reviewed-by: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> [modify] https://crrev.com/43ac0331a240c9cdefee549a625b41f43f60fd83/client/cros/chrome_binary_test.py
,
May 23 2016
Bulk verified
,
May 23 2016
bulk verified |
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by steve...@chromium.org
, Apr 25 2016