android_n5x_swarming_rel try bot is flaky |
||||
Issue descriptionI'm trying to land https://codereview.chromium.org/2680403005/ that is iOS only change (only touches files built on iOS) and is code that is only used by the test suite that test it (part of a multi CL refactoring). However, android_n5x_swarming_rel has been failing flakily multiple times, preventing me from landing the CL. This is not the first time this bot has been flaking and preventing landing iOS only changes. See for example https://bugs.chromium.org/p/chromium/issues/detail?id=666293.
,
Feb 17 2017
I don't think we have any UI showing what's flaky on a specific builder, so it would be very hard to find an owner for this. From your CL looks like the following tests are failing: - org.chromium.android_webview.test.PostMessageTest#testCanUseReceivedAwMessagePortFromJS - benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.browse:news:cnn The flakes for them have been recorded by chromium-try-flakes: - http://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyYgsSBUZsYWtlIldvcmcuY2hyb21pdW0uYW5kcm9pZF93ZWJ2aWV3LnRlc3QuUG9zdE1lc3NhZ2VUZXN0I3Rlc3RDYW5Vc2VSZWNlaXZlZEF3TWVzc2FnZVBvcnRGcm9tSlMM - http://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyeQsSBUZsYWtlIm5iZW5jaG1hcmtzLnN5c3RlbV9oZWFsdGhfc21va2VfdGVzdC5TeXN0ZW1IZWFsdGhCZW5jaG1hcmtTbW9rZVRlc3Quc3lzdGVtX2hlYWx0aC5tZW1vcnlfbW9iaWxlLmJyb3dzZTpuZXdzOmNubgw First one is not flaky enough to have a bug filed for it yet, but the second is clearly very flaky. I am not sure why the bug wasn't filed yet, but will investigate.
,
Feb 17 2017
After looking at the logs, I now see what happened. Yesterday, I've fixed a bug in a recipe uploading test results, which helped chromium-try-flakes to file bugs for specific flaky tests instead of filing bugs for entire step. This prevents the problem when nobody wanted to own bugs for entire flaky step since it would mean they would have to fix all flaky tests there. Unfortunately, this also means that many bugs for flaky tests need to be filed instead of a few bugs that were used to track flaky steps containing them. As a result chromium-try-flakes quickly hit the limit of 10 bugs per day, which was set up to prevent chromium-try-flakes from filing hundreds of bugs when a single change breaks lots of tests, so this is WAI. Over the next coming days the backlog of bugs should be exhausted and chromium-try-flakes should be quickly filing bugs for flaky tests. Meanwhile, I recommend reaching out to the sheriffs or disabling the test yourself.
,
Feb 17 2017
I'm going to add Sheriff-Chromium label, so that this bug shows up on SoM.
,
Feb 17 2017
This is not infra flakiness, so removing corresponding components and adding Tests>Flaky.
,
Feb 17 2017
Looks like benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.browse:news:cnn has not flaked in the last ~170 builds. Marking this as WontFix, and as per #3 it looks like a separate bug will be auto-filed if this flakiness is observed again. |
||||
►
Sign in to add a comment |
||||
Comment 1 by sdefresne@chromium.org
, Feb 17 2017