Massive exceptions on the Linux ChromeOS MSan Tests bot |
|||||
Issue descriptionhttps://build.chromium.org/p/chromium.memory.full/waterfall?builder=Linux%20ChromeOS%20MSan%20Tests has been reporting test crashes for a month already. Looks like the problem is in the missing libgraphite.so: ./compositor_unittests: error while loading shared libraries: libgraphite2.so.3: cannot open shared object file: No such file or directory Error: failed to symbolize snippets in JSON: Traceback (most recent call last): File "/b/swarm_slave/w/irDxc6lZ/tools/valgrind/asan/asan_symbolize.py", line 271, in <module> main() File "/b/swarm_slave/w/irDxc6lZ/tools/valgrind/asan/asan_symbolize.py", line 264, in main symbolize_snippets_in_json(args.test_summary_json_file, loop) File "/b/swarm_slave/w/irDxc6lZ/tools/valgrind/asan/asan_symbolize.py", line 220, in symbolize_snippets_in_json with open(filename, 'r') as f: IOError: [Errno 2] No such file or directory: '/b/swarm_slave/w/ioB6p_Jt/output.json' (see https://build.chromium.org/p/chromium.memory.full/builders/Linux%20ChromeOS%20MSan%20Tests/builds/4405/steps/compositor_unittests/logs/stdio)
,
Jan 13 2017
dpranke@ yes, Precise has libgraphite2.so.2 but Trusty has libgraphite2.so.3 which is needed. If the bot upgrade is taking too long, maybe we could statically link libgraphite?
,
Jan 13 2017
It should be easy enough to upgrade individual bots if they're currently failing. However, we need to be careful that we don't generally have requirements that anything chromeos-related requires jessie, since not everything is on jessie.
,
Jan 13 2017
Maybe I missed something but why is this failing now?
,
Jan 13 2017
I don't know ...
,
Jan 13 2017
In general, tying the build to the packages installed on the host leads to issues like this one. I thought all the ChromeOS stuff happened in a chroot to avoid these sorts of issues?
,
Jan 13 2017
This isn't a real CrOS builder, it's a "desktop CrOS builder", i.e., a variant of the Chromium Linux build. Clearing the R-V-G and Infra-Troopers flags, I don't think this is a trooper issue. Tom, maybe you can figure out either what to do here or ask someone?
,
Jan 14 2017
I'm going to take a guess as to what's happening. The script that downloads the prebuilt instrumented libraries always downloads the host distro packages. (third_party/instrumented_libraries/scripts/download_binaries.py) We have prebuilt libraries for Precise and Trusty, but the builder is on Trusty, so it will download those. What we really want is to download the prebuilt libraries for the platform that will actually run the tests (still Precise). glider@ says this broke about a month ago. That suspiciously sounds like the time when johnw@ did all of the Precise->Trusty upgrades.
,
Jan 16 2017
,
Jan 23 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build.git/+/1df4a5b1b6ccce9f56e863a48801bf3915affa5d commit 1df4a5b1b6ccce9f56e863a48801bf3915affa5d Author: Tom Anderson <thomasanderson@google.com> Date: Mon Jan 23 22:11:51 2017 Use Trusty platform when running MSan swarming tests BUG= 680967 R=dpranke@chromium.org Change-Id: Id0a79b532d6b179a3377165942176e2ea150bc0d Reviewed-on: https://chromium-review.googlesource.com/430829 Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Tom Anderson <thomasanderson@google.com> [modify] https://crrev.com/1df4a5b1b6ccce9f56e863a48801bf3915affa5d/scripts/slave/recipe_modules/chromium_tests/chromium_memory_full.py [modify] https://crrev.com/1df4a5b1b6ccce9f56e863a48801bf3915affa5d/scripts/slave/recipes/chromium.expected/full_chromium_memory_full_Linux_ChromeOS_MSan_Tests.json [modify] https://crrev.com/1df4a5b1b6ccce9f56e863a48801bf3915affa5d/scripts/slave/recipes/chromium.expected/full_chromium_memory_full_Linux_MSan_Tests.json
,
Jan 24 2017
It appears the exceptions are now fixed. https://build.chromium.org/p/chromium.memory.full/builders/Linux%20ChromeOS%20MSan%20Tests/builds/4818 However, now that the tests are actually being run, there are new MSAN failures that were introduced during this bot's downtime. I've opened bug 684175 for that issue. |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by dpranke@chromium.org
, Jan 13 2017