Android testers passing .isolate files are failing confusingly |
|||
Issue descriptionIn WebRTC, we suddenly started getting a lot of content_browsertests failures in our https://build.chromium.org/p/chromium.webrtc.fyi/ waterfall. First failing run: https://build.chromium.org/p/chromium.webrtc.fyi/builders/Android%20Tests%20%28dbg%29%20%28K%20Nexus5%29/builds/9205 The reason is this Angle roll: https://codereview.chromium.org/2228553002 that removes the angle.isolate file: https://chromium-review.googlesource.com/#/c/365234/ Because of this, our tests started failing like this: I 0.006s Main command: /b/c/b/Android_Tests__dbg___L_Nexus6_/src/build/android/test_runner.py gtest --suite content_browsertests --isolate-file-path /b/c/b/Android_Tests__dbg___L_Nexus6_/src/out/Debug/gen.runtime/content/test/content_browsertests.device.isolate --output-directory /b/c/b/Android_Tests__dbg___L_Nexus6_/src/out/Debug --gtest_filter=WebRtc* --blacklist-file /b/c/b/Android_Tests__dbg___L_Nexus6_/src/out/bad_devices.json --verbose --isolate_file_path=/b/c/b/Android_Tests__dbg___L_Nexus6_/src/content/content_browsertests.isolate --flakiness-dashboard-server=test-results.appspot.com --json-results-file /tmp/tmpWfZUbv.json --adb-path /b/c/b/Android_Tests__dbg___L_Nexus6_/src/third_party/android_tools/sdk/platform-tools/adb I 0.006s TimeoutThread-1-for-MainThread [host]> /b/c/b/Android_Tests__dbg___L_Nexus6_/src/third_party/android_tools/sdk/platform-tools/adb devices I 0.010s Main [host]> /b/c/b/Android_Tests__dbg___L_Nexus6_/src/third_party/android_tools/sdk/build-tools/23.0.1/aapt dump xmltree /b/c/b/Android_Tests__dbg___L_Nexus6_/src/out/Debug/content_browsertests_apk/content_browsertests-debug.apk AndroidManifest.xml I 0.089s Main [host]> /usr/bin/python /b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/isolate.py remap --isolate /b/c/b/Android_Tests__dbg___L_Nexus6_/src/content/content_browsertests.isolate --isolated /b/c/b/Android_Tests__dbg___L_Nexus6_/src/out/Debug/content_browsertests.isolated --outdir /tmp/tmpi17Ysh --path-variable DEPTH /b/c/b/Android_Tests__dbg___L_Nexus6_/src --path-variable PRODUCT_DIR /b/c/b/Android_Tests__dbg___L_Nexus6_/src/out/Debug --config-variable use_ozone 0 --config-variable msvs_version 0 --config-variable v8_use_external_startup_data 1 --config-variable branding Chromium --config-variable chromeos 0 --config-variable tsan 0 --config-variable kasko 0 --config-variable msan 0 --config-variable use_x11 0 --config-variable component static_library --config-variable CONFIGURATION_NAME Debug --config-variable use_instrumented_libraries 0 --config-variable use_prebuilt_instrumented_libraries 0 --config-variable icu_use_data_file_flag 1 --config-variable asan 0 --config-variable lsan 0 --config-variable enable_pepper_cdms 0 --config-variable use_custom_libcxx 0 --config-variable enable_plugins 0 --config-variable fastbuild 0 --config-variable OS android --config-variable target_arch arm C 0.978s Main STDERR: Traceback (most recent call last): File "/b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/isolate.py", line 1226, in <module> sys.exit(main(sys.argv[1:])) File "/b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/isolate.py", line 1212, in main return dispatcher.execute(parser, argv) File "/b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/third_party/depot_tools/subcommand.py", line 251, in execute return command(parser, args[1:]) File "/b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/isolate.py", line 974, in CMDremap complete_state = load_complete_state(options, cwd, None, options.skip_refresh) File "/b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/isolate.py", line 677, in load_complete_state options.extra_variables, options.blacklist, options.ignore_broken_items) File "/b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/isolate.py", line 506, in load_isolate self.saved_state.config_variables)) File "/b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/isolate_format.py", line 649, in load_isolate_for_config isolate = load_isolate_as_config(isolate_dir, eval_content(content), None) File "/b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/isolate_format.py", line 627, in load_isolate_as_config included = load_included_isolate(isolate_dir, include) File "/b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/isolate_format.py", line 557, in load_included_isolate None) File "/b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/isolate_format.py", line 627, in load_isolate_as_config included = load_included_isolate(isolate_dir, include) File "/b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/isolate_format.py", line 553, in load_included_isolate with fs.open(included_isolate, 'r') as f: File "/b/c/b/Android_Tests__dbg___L_Nexus6_/src/tools/swarming_client/utils/fs.py", line 120, in open return __builtin__.open(extend(path), *args, **kwargs) IOError: [Errno 2] No such file or directory: '/b/c/b/Android_Tests__dbg___L_Nexus6_/src/third_party/angle/angle.isolate' All our Chromium bots are using GN now, but it appears unexpected behavior is caused by that they still pass the --isolate_file_path flag, like this: --isolate_file_path=/b/c/b/Android_Tests__dbg___K_Nexus5_/src/content/content_browsertests.isolate jbudorick/agrieve: Can you update the test runner scripts so a clear error is thrown about it being deprecated (or just remove it entirely)?
,
Aug 9 2016
The following revision refers to this bug: https://chrome-internal.googlesource.com/chrome/tools/build_limited/scripts/slave/+/1f004e5e260487d896046327df7e3366e333a1a4 commit 1f004e5e260487d896046327df7e3366e333a1a4 Author: recipe-roller <recipe-roller@chromium.org> Date: Tue Aug 09 10:21:55 2016
,
Aug 9 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a606c0a3b4600432aa30d795d9034479789b8d35 commit a606c0a3b4600432aa30d795d9034479789b8d35 Author: recipe-roller <recipe-roller@chromium.org> Date: Tue Aug 09 12:23:11 2016 Roll recipe dependencies (trivial). This is an automated CL created by the recipe roller. This CL rolls recipe changes from upstream projects (e.g. depot_tools) into downstream projects (e.g. tools/build). More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug (or complain) build: https://crrev.com/daee037381ce9c7c3c92a6d2a095846d6c01a5ac WebRTC: Remove --isolate_file_path argument from content_browsertests (kjellander@chromium.org) R=kjellander@chromium.org,magjed@chromium.org BUG= 635818 TBR=martiniss@chromium.org,phajdan.jr@chromium.org Review-Url: https://codereview.chromium.org/2230453003 Cr-Commit-Position: refs/heads/master@{#410642} [modify] https://crrev.com/a606c0a3b4600432aa30d795d9034479789b8d35/infra/config/recipes.cfg
,
Aug 9 2016
--isolate-file-path isn't deprecated, but the checked-in .isolate files are. They've been replaced by generated .isolate files based on the data fields in dependent gn targets. I'll switch the webrtc bots over.
,
Aug 9 2016
I really need to stop looking at bugs before getting coffee. kjellander, it looks like your CL in #1 fixed this.
,
Aug 9 2016
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/71a3cf4008d2df4e0844d662c357177d4d1728ec commit 71a3cf4008d2df4e0844d662c357177d4d1728ec Author: recipe-roller <recipe-roller@chromium.org> Date: Tue Aug 09 14:19:21 2016 Roll recipe dependencies (trivial). This is an automated CL created by the recipe roller. This CL rolls recipe changes from upstream projects (e.g. depot_tools) into downstream projects (e.g. tools/build). More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug (or complain) build: https://crrev.com/daee037381ce9c7c3c92a6d2a095846d6c01a5ac WebRTC: Remove --isolate_file_path argument from content_browsertests (kjellander@chromium.org) https://crrev.com/0232a12f269f590f84ab5a11b4d56e5a221eb574 findit: prepare for running under remote_run (phajdan.jr@chromium.org) R=kjellander@chromium.org,magjed@chromium.org,phajdan.jr@chromium.org BUG= 593999 , 635818 ,584079 TBR=martiniss@chromium.org,phajdan.jr@chromium.org Review-Url: https://codereview.chromium.org/2228903003 [modify] https://crrev.com/71a3cf4008d2df4e0844d662c357177d4d1728ec/infra/config/recipes.cfg |
|||
►
Sign in to add a comment |
|||
Comment 1 by bugdroid1@chromium.org
, Aug 9 2016