New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 637145 link

Starred by 1 user

Issue metadata

Status: Duplicate
Owner: ----
Closed: Aug 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

mac_chromium_rel_ng flaky

Project Member Reported by chromium...@appspot.gserviceaccount.com, Aug 12 2016

Issue description

"hardware_accelerated_feature_tests (with patch)" is flaky.

This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label.

We have detected 55 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyOgsSBUZsYWtlIi9oYXJkd2FyZV9hY2NlbGVyYXRlZF9mZWF0dXJlX3Rlc3RzICh3aXRoIHBhdGNoKQw.



This flaky test/step was previously tracked in  issue 627155 .
 

Comment 1 by samli@chromium.org, Aug 12 2016

Labels: Infra-Troopers
Status: Available (was: Untriaged)
Summary: mac_chromium_rel_ng flaky (was: "hardware_accelerated_feature_tests (with patch)" is flaky)
https://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_ng

Fails to trigger or run the tests sometimes. Times out waiting for shard #0.

https://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_ng/builds/276392/steps/context_lost_tests%20on%20Intel%20GPU%20on%20Mac%20%28with%20patch%29%20on%20Mac-10.10/logs/stdio

Comment 2 by samli@chromium.org, Aug 12 2016

 Issue 637144  has been merged into this issue.

Comment 3 by sashab@chromium.org, Aug 15 2016

Labels: -Sheriff-Chromium
Judging by the test results, this looks like flaky infra rather than flaky tests. See where the latest bad run started: https://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_ng/builds/275303

upload_test_results seems to finish fine but the next set of tests all output:
Waiting for results from the following shards: 0
Waiting for results from the following shards: 0
Waiting for results from the following shards: 0
+------------------------------------------------------------------------+
| Shard 0  https://chromium-swarm.appspot.com/user/task/308e9de25ef3e810 |
+------------------------------------------------------------------------+

+------------------------------------------------------------------------+
| End of shard 0  Pending: N/A  Duration: N/A  Bot: N/A  Exit: N/A       |
+------------------------------------------------------------------------+

And then the results say:
TEST RESULTS WERE INVALID 

Assigning to infra and removing the Chromium-Sheriff label.
looks like a lot of the infra failures have error messages like:
"Missing or invalid gtest JSON file: /tmp/tmp8gh_4X_swarming/0/output.json"



Comment 5 by mar...@chromium.org, Aug 15 2016

Cc: kbr@chromium.org
That's likely a bug in the recipe that does not handle expired task?
https://chromium-swarm.appspot.com/user/task/308e9de25ef3e810 was definitely expired, due to OSX GPU fleet saturation. It's a transient problem but fixing the recipe to be helpful should be done.

Comment 6 by kbr@chromium.org, Aug 15 2016

Mergedinto: 637081
Status: Duplicate (was: Available)
As far as I can tell, this was the same outage as in Issue 637081, and I'm duplicating this into it.

If there's some improvement that should be made to the error handling output for isolated script tests in tools/build/scripts/slave/recipe_modules/swarming/api.py :
https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/swarming/api.py?q=_isolated_script_collect_step&sq=package:chromium&l=757&dr=CSs

then I'd appreciate it if someone from the Infra team could either tell me exactly what's needed, point me to an example, or, ideally, make the improvement and send it for review. Thanks.

Sign in to add a comment